feat(star6e): add live SuperFrame encoder controls

[etiaro]

Assisted-by: OpenAI GPT 5.6

Summary

• What changed and why.

Expose I/P frame-size and frame-loss thresholds as percentage-based video0 settings. Apply them against the effective bitrate and delivered FPS, refreshing limits after live bitrate, throttle, and FPS changes.

Add config, HTTP API, WebUI, documentation, and test coverage. Maruko reports the new fields as unsupported.

This allows for reducing the resulting stream burstiness by limiting maximum frame size and according to docs, 4 reencoding trials if that limit is exceeded.

My tests with below settings show that the limit is working well in "racing" resilience mode:

    "superframePFramePercent": 110,
    "superframeIFramePercent": 150,
    "superframeLossPercent": 250,
    "encFrmGaps": 1,

The superframePFramePercent is the knob that gives most control - at 110 it keeps bitrate at about 2/3 configured. At 200 the bitrate seems to stabilize around configured limit.
To be done: Measure the introduced latency

Validation

• Build/test commands run:
  • make build SOC_BUILD=star6e
  • make build SOC_BUILD=maruko (if applicable)
• Runtime smoke checks performed:
  • Star6E
  • Maruko (if applicable)

Checklist

• VERSION updated for this PR.
• HISTORY.md updated with this PR's user-visible changes.
• Documentation updated for changed behavior.

HTTP API Contract (required when HTTP behavior changes)

• documentation/HTTP_API_CONTRACT.md updated in this PR.
• Endpoint/payload/status changes match the contract exactly.
• Contract version updated appropriately (breaking vs non-breaking).

HTTP API Design Guardrails

• Endpoints are lean and focused on direct operational value.
• JSON payloads remain simple and descriptive.
• No unnecessary endpoint proliferation or generic/ambiguous fields were introduced.

Notes

• Star6E-first policy for SigmaStar API-touching functionality:
  implement and validate on Star6E first, then add explicit Maruko follow-up items.

[snokvist]

Hardware review: changes requested

I do not consider this merge-ready yet. The percentage controls have genuine burst-control value, but the current implementation conflates three different mechanisms, resets other live RC controls at startup, and the retry/monitoring claims do not match this firmware's observed behavior.

What was tested

PR head: 3c75676450db058a96ef86e3ed06cb699f8a4c70.

• Star6E: SSC338Q, IMX335, H.265 CBR, 1920x1080@60, 25 Mbps. Main IQ file: /etc/sensors/imx335_spike5_colortrans.bin; /etc/sensors/imx335.bin was also live-swapped during a scene-change attempt. waybeam_hub was stopped and venc sent direct UDP to the probe, so no external link controller was clamping bitrate. Tests covered resilience off, current racing, and the fork's sliceCount 1/9 integration.
• Maruko: SSC378QE, IMX335, H.265 CBR, 1280x720@30, 3 Mbps.
• Clean exact-PR binaries: Star6E SHA-256 3a157098536707f8ee6b3c9b62a9624af22b954bd7e523ea5221b8fbee4c387a; Maruko 7b09c87409966ab1cd51f0de895195f024600753aa9504664656587660a3e67e.
• Timing captures used an exact-control diagnostic Star6E binary, SHA-256 527db90f923a0f6be22e3e232e9627a6d7729007a9098715b7e93f86cffbc13a. Its only behavioral addition was opt-in /proc/mi_modules/mi_venc/mi_venc0 counters appended to the existing RTP sidecar after frame send. Probe SHA-256: 5c3388f1f00fe827eacdb1a14c037a25d641e80a939c379ccc3ee8879ece1f0b.
• A separate isolation binary (dc7ea832...) disabled the existing RC max-size/FRAMEBITS_FIRST layer under an environment flag. Results from that binary are identified below and are not presented as exact PR behavior.
• Fork integration binary (PR cherry-picked onto snokvist/master, including resilience and slice support): eeedc8c36537938ef6a815e52b4281edaf128ad55233180e676ccf93a7351903.
• Exact PR: 2,496 host assertions passed; Star6E and Maruko lint/build passed. The fork integration also passed host tests and Star6E lint/build.

Both devices were restored to their original binaries and configs after testing. Star6E is back at 60 fps with waybeam_hub running; Maruko is back on its production 0.60.1 binary. Device /tmp was not used for binaries or captures.

Claims/results

Percentage limits have real value

With legacy maxIBytes/maxPBytes set to zero in racing:

Setting	Wire rate	Largest frame	Encode mean / max	Delivered FPS	RTP gaps
SuperFrame off	22.84 Mbps	429,003 B	10.88 / 16.64 ms	60.0	59, all in the 429 KB frame
I=150%, P=110%, loss=250%	13.82 Mbps	55,507 B	14.36 / 16.38 ms	60.2	0
I=150%, P=200%, loss=250%	25.45 Mbps	66,460 B	10.92 / 11.75 ms	60.1	0

This confirms the direction of the PR text: P=110 materially undershoots the configured bitrate in this setup, while P=200 reaches it. It also confirms useful burst reduction. The measured P=110 rate was about 54% of configured bitrate, not a universal 2/3; that value is scene/config dependent.

The threshold formula and refresh paths were confirmed in /proc: at 25 Mbps/60 fps, I=150/P=110 produced 640,000/469,336-bit thresholds; a live change to 12.5 Mbps halved them to 320,000/234,672; a live change to 30 fps doubled them again.

Normal latency cost is pre-encoding shaping, not an observed retry

Mode	Off baseline mean/max	I=150/P=110 mean/max	Mean delta	SDK overshoot events
GOP, resilience off	8.94/10.48 ms	13.76/15.65 ms	+4.82 ms	0
Racing, no legacy caps	10.88/16.64 ms	14.36/16.38 ms	+3.48 ms	0
Current racing config with legacy caps	10.44/10.96 ms	13.53/15.41 ms	+3.09 ms	0

No successful retry was observed in ordinary scene changes, requested IDRs, either IMX335 bin, or the author settings. The PR programs the same derived ceilings into RC u32MaxISize/u32MaxPSize, selects FRAMEBITS_FIRST, and also programs SuperFrame REENCODE. In these successful tests, RC shaped the first encode below the threshold; vendor SuperFrame tracing reported no overshoot/retry.

Forced overshoot behavior does not match the four-retry claim

To force the event path, I used I=P=20% and constrained QP to 12..20 after SuperFrame was enabled. This is deliberately pathological and is a failure-path test, not a recommended setting.

On the fork integration in racing mode:

Slices	Delivered FPS	Encode mean / p95 / max	Output DropCnt delta	Vendor events
1	19.46	72.51 / 89.92 / 89.98 ms	+295	461 messages (~230 overshoot+abort pairs)
9	19.24	71.96 / 89.91 / 90.92 ms	+292	456 messages (~228 pairs)

Every classified overshoot was logged as times:0, immediately followed by Fail to re-encode/discard, generate next GOP. No times:1 or higher event was observed. The observed worst delivered-frame capture-to-ready latency was 90.92 ms; this is not claimed as a theoretical bound. Slice 1 and 9 were equivalent within this test, so there is no evidence here that slices add a separate penalty.

The diagnostic ReEncCnt is not a SuperFrame event counter on this firmware. It stayed flat across hundreds of vendor-classified overshoot/abort events, while requested IDRs incremented it. The contract text telling users to monitor ReEncCnt for this feature is therefore incorrect for the tested firmware.

An isolation test with RC size caps/FRAMEBITS_FIRST disabled still found no successful retry regime: with QP headroom the encoder pre-shaped below the threshold and emitted no event; with insufficient headroom it logged times:0 and aborted. I cannot confirm the PR body's “4 reencoding trials” claim from the tested SDK, binary, or added headers.

FrameLost is separate and active

With SuperFrame I/P disabled and superframeLossPercent=1:

• encFrmGaps=1: 30.09 fps, Output DropCnt +599 in 20 s (approximately every other frame).
• encFrmGaps=2: 40.12 fps, Output DropCnt +400 in 20 s (approximately every third frame).

With the complete author setting 150/110/250/1 over 30 s: 60.1 fps, no new drops, no vendor SuperFrame events, 14.18 ms mean / 16.26 ms max encode time. External scene motion was requested for this capture but was not independently instrumented. The 250% FrameLost threshold did not engage.

So superframeLossPercent/encFrmGaps expose a genuine periodic frame-skip mechanism, but it is distinct from SuperFrame retry and can directly reduce temporal resolution.

Maruko negative path is correct, with one schema issue

The exact PR Maruko binary streamed 360/360 frames at 30.1 fps with zero RTP gaps. All four capability entries reported supported:false; both a single-field write and a four-field batch returned HTTP 501; /etc/waybeam.json remained byte-identical. This is confirmed on device.

However, the unsupported fields are added to config/waybeam.default.maruko.json and appear in Maruko's active config response. Our agreed integration direction is hard removal of retired/unsupported controls on Maruko, without compatibility aliases or legacy trails.

Merge blockers

1. SuperFrame resets existing QP controls at startup. With config qpDelta=-12, minQp=12, maxQp=24, startup logs reported all three applied, then SuperFrame applied. /proc showed IPQPDelta=0, MaxQp=48, MinQp=12. The same reset occurred with the author settings (qpDelta=-12 became 0). A live multi-set containing QP bounds plus SuperFrame fields returned HTTP 200 for all fields while the encoder retained the old/default QP maximum. The current order is visible in src/star6e_runtime.c:841-857; the PR has no SuperFrame/FrameLost readback binding to catch the reset. Preserve/reapply all RC state after these SDK calls and verify the actual driver state before reporting success.

2. The PR combines three controls under one knob. src/star6e_controls.c:578-614 programs RC max I/P sizes, RC priority, SuperFrame REENCODE, and FrameLost together. Consequently ordinary “SuperFrame” latency/results mostly measure RC pre-shaping and do not exercise re-encode. Please make the intended mechanism unambiguous and test it independently. Per the previously agreed cleanup, replace and hard-remove the older overlapping maxIBytes/maxPBytes fields from the shared schema/defaults/API on both Star6E and Maruko; do not keep compatibility aliases or legacy config trails.

3. Do not implicitly enable FrameLost whenever either I/P percentage is nonzero. frame_lost_enabled = superframe_enabled || loss_percent > 0 at src/star6e_controls.c:578-581 turns on a separate frame-skip strategy even when superframeLossPercent=0. This repository removed FrameLost in 0.19.0 after prior device testing found it unsuitable as a bandwidth throttle (HISTORY.md, 0.19.0). The fresh results above show exactly how aggressively encFrmGaps can reduce FPS. Separate it from SuperFrame, justify its reintroduction, and give it explicit safe bounds/tests—or leave it out of this PR.

4. Add validation and failure coverage. The four API descriptors at src/venc_api.c:569-572 have no ranges. Startup JSON uses signed json_get_int() followed by unsigned casts at src/venc_config.c:631-638, so negative values become huge unsigned values. encFrmGaps=0 is deliberately passed through but its device semantics are not established. Extreme percentages can reach the later UINT32_MAX/8 rejection, and startup ignores the apply_max_frame_size() return at src/star6e_runtime.c:854-857. Add startup and live validation, documented ranges, callback-failure/rollback tests, and tests for zero/one-sided/extreme values.

5. Correct the documentation and stack the PR cleanly. documentation/HTTP_API_CONTRACT.md:464-466 incorrectly recommends ReEncCnt; the example config says encFrmGaps=0 while the default is 1. VERSION and HISTORY.md are unchanged. This commit also carries PR #112's unrelated bitrate-no-IDR hunk (src/star6e_controls.c:296-299), and both PRs currently claim contract 0.18.3. Rebase/stack after #112 and remove the duplicate hunk/version collision.

6. Tests currently cover schema plumbing only. The added tests check defaults, round-trip serialization, and one backend-support predicate. They do not test threshold math, live refresh, ordering/state preservation, API group rollback, overflow/ranges, FrameLost coupling, or SDK call failures. These need coverage before another hardware pass.

Fork inclusion

I cherry-picked the PR onto current snokvist/master in local branch review/pr113-fork-integration. Four textual conflicts required resolution (contract version/example, API version, generated WebUI, and the API support test list). The resolved tree passes host tests and Star6E lint/build, and the 1/9-slice hardware matrix above used that build. This proves the code can be integrated mechanically, but it should not be merged into the fork history until the blockers above are fixed. No review branch was pushed.

IMX445 is not needed to establish these defects. After fixes, we can repeat the device matrix on Star6E and do the Maruko negative regression for the author; an additional sensor would only broaden burst-scene coverage.

[etiaro]

RC max I/P sizes will be separated.

According to sigmastar documentation MI will increase the QP of the frame by 4 and reencode the super frame.If it still exceeds the threshold after repeating up to 4 times, MI will discard the frame.
frameLost is introduced as a means to control max dropped consecutive frames. It is to be tested if this mechanism applies here, I thought it does due to image explaining that bevavior was pasted in this superframes description.

[snokvist]

Heads-up: #114 (parity sync v0.65.2 → v0.67.1, 54 commits) is open and overlaps this one more than the others.

Two things worth knowing before rebasing:

1. This PR has no VERSION bump or HISTORY.md entry. The repo's PR template asks for both, and it makes the sync accounting work — after sync: bring upstream to fork parity v0.65.2 -> v0.67.1 #114 lands, the next free version is 0.67.2 and the next free contract version is 0.18.7 (sync: bring upstream to fork parity v0.65.2 -> v0.67.1 #114 takes the contract to 0.18.6).

2. web/dashboard.html needs its blob regenerated. src/venc_webui.c is a gzip of the dashboard and make webui-check gates the two against each other. sync: bring upstream to fork parity v0.65.2 -> v0.67.1 #114 regenerates it, so editing dashboard.html here will conflict binary-on-binary — the fix is to rebase, then re-run the webui build so the blob matches your HTML.

There's also genuine overlap in src/star6e_mi.c, src/star6e_runtime.c, src/venc_api.c and src/venc_config.c, since #114 adds video0.sliceCount and binds MI_VENC_SetH265SliceSplit in the same CreateChn→StartRecvPic window SuperFrame uses. Worth a look at how the two interact rather than a mechanical rebase.

Sorry for the churn — this bundle had been sitting on the fork a while. Happy to help if useful.

[snokvist]

#114 is merged, so this now shows as conflicting — this is the heaviest of the four, 14 files overlapping.

Start with the generated file. src/venc_webui.c is a gzip blob of web/dashboard.html and make webui-check gates them against each other, so it will conflict binary-on-binary and is not worth hand-resolving. Take upstream's src/venc_webui.c, rebase your web/dashboard.html edits on top, then regenerate the blob (python3 tools/build_webui.py) and confirm make webui-check passes.

The part worth real attention is src/star6e_mi.c / include/star6e_mi.h / include/star6e.h / src/star6e_runtime.c. The sync binds MI_VENC_SetH265SliceSplit in the same CreateChn → StartRecvPic window SuperFrame uses, with a Set/Get readback that fails startup on mismatch. Two vendor calls now share that window, so it is worth checking the ordering interacts the way you expect rather than doing a mechanical rebase — the SDK is order-sensitive there.

Bookkeeping: this PR has no VERSION bump or HISTORY.md entry — the template asks for both, and it is what keeps the sync accounting working. Next free is VERSION 0.67.2 and contract 0.18.7 (four sites: header near line 21, example body, divergence note, and the string in venc_api.c).

Happy to help if any of that is awkward — sorry for landing a large bundle on top of your PR.

[snokvist]

Let me know if you want to pursue this, otherwise we can cloose the PR and you return at a point when you have made further progress. @etiaro

[etiaro]

I have some version working nicely for racing (150% avg frame size limit, but lowest value capped at 5mbps) that I'll open in separate PR, but I don't really have time to tune it out for all resilience modes.