Skip to content

Keep chat prompt overlays outside streamed output - #943

Open
Gleek wants to merge 2 commits into
xenodium:mainfrom
Gleek:fix/chat-prompt-front-advance
Open

Gleek wants to merge 2 commits into
xenodium:mainfrom
Gleek:fix/chat-prompt-front-advance

Conversation

@Gleek

@Gleek Gleek commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

In chat mode, streaming responses cause frequent relabeling of the entire conversation. This is expensive in long sessions: my CPU profiles showed prompt discovery taking roughly half the sampled CPU time. With multiple such sessions running in parallel Emacs bogs down quite considerably.

21118  79% - timer-event-handler
21089  79%  - apply
12639  47%   - agent-shell-chat--relabel-buffer
12635  47%    - agent-shell-chat--relabel
11962  44%     - agent-shell-chat--label-prompts
11584  43%      - agent-shell-chat--prompt-runs
 8416  31%         next-single-property-change
 2907  10%         agent-shell-chat--prompt-face-p

Disclaimer: agent-shell is v0.87.10

From what I understand the prompt overlays include text inserted at their start.

Since streamed output is inserted immediately before the live prompt, those overlays can expand over the output and hide it. Relabeling adjusts their boundaries to reveal the output.

This change gives prompt overlays front-advance so they move with the prompt automatically.
Ordinary streaming skips prompt relabelling, while response headings continue to update.
Submissions, restores, etc, would retain full relabelling.

Checklist

  • I agree to communicate (PR description and comments) with the author myself (not AI-generated).
  • I've reviewed all code in PR myself and will vouch for its quality.
  • I've read and followed the Contributing guidelines.
  • I've filed a feature request/discussion for a new feature.
  • I'm making visual changes, so I'm including screenshots so you can view and discuss.
  • I've added tests where applicable.
  • I've updated documentation where necessary.
  • I've run M-x checkdoc and M-x byte-compile-file.

Update: agent-heading relabelling

Updated the PR with the optimisation for agent relabelling as well. Ordinary streaming appends now skip both prompt and agent-heading relabelling.

The same 600-update benchmark took 25.84 seconds on v0.96.1 and now completes in about 6 ms (0.0061 seconds). The CPU profiles recorded 10,131 samples before and four after, with no samples in either labelling pass in the updated run.

This measures append-time label maintenance on the same three buffer snapshots, with GC enabled. Initial heading creation and the full streaming/rendering pipeline are excluded.

@Gleek
Gleek marked this pull request as ready for review November 12, 2026 15:24
@xenodium

Copy link
Copy Markdown
Owner

Thanks for profiling! Your profile is from v0.87.10. Since then, #905 (v0.89.2) made prompt scanning much cheaper. Are you confident the changes are still needed?

@Gleek

Gleek commented Nov 16, 2026

Copy link
Copy Markdown
Contributor Author

Got it! I'll update this week and see if there's still issue without my change.

@Gleek

Gleek commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor Author

I compared this implementation on the lastest version as well (v0.96.1) by inserting a single text in different buffer sizes.

Buffer size Upstream With the change
159 KB 2.43 ms 1.11 ms
2.1 MB 6.70 ms 3.55 ms
32.5 MB 28.65 ms 18.06 ms

When running 600 updates and we get about half the samples with the change than without it.
These are the profiles:
original:

        2031  87% - normal-top-level
        2031  87%  - command-line
        2031  87%   - command-line-1
        2031  87%    - load-with-code-conversion
        2031  87%     - let
        2031  87%      - let
        2031  87%       - unwind-protect
        2031  87%        - let
        2031  87%         - while
        2031  87%          - let
        2031  87%           - let
        2031  87%            - agent-shell-perf--stream-updates
        2030  87%             - agent-shell-chat--relabel
        1078  46%              + agent-shell-chat--label-responses
         952  41%              + agent-shell-chat--label-prompts
         288  12%   Automatic GC
           0   0%   ...

after the change:

        1108  89% - normal-top-level
        1108  89%  - command-line
        1108  89%   - command-line-1
        1108  89%    - load-with-code-conversion
        1108  89%     - let
        1108  89%      - let
        1108  89%       - unwind-protect
        1108  89%        - let
        1108  89%         - while
        1108  89%          - let
        1108  89%           - let
        1108  89%            - agent-shell-perf--stream-updates
        1105  89%             - agent-shell-chat--label-section-response
        1105  89%              + agent-shell-chat--label-responses
         127  10%   Automatic GC
           0   0%   ...

Notice the agent-shell-chat--label-prompts branch going away.

On a side note, after this profile I realise that relabelling is still taking quite a lot of time most probably because we are still scanning all the agent headings in the --label-responses which takes in the whole buffer to process the labels. Let me see if I can add the optimisation for this as well.

@Gleek

Gleek commented Nov 16, 2026

Copy link
Copy Markdown
Contributor Author

Agent headings now update when a fragment is created, rather than being rescanned on every streamed append.

Updated the PR with the optimisation for agent relabelling as well. The repeated labelling work during ordinary streaming appends is now gone.

The same 600 update benchmark that took about 25 seconds on v0.96.1 now completes in about 6 ms, with only four CPU samples recorded. This measures append-time label maintenance, excluding initial heading creation and the full rendering pipeline.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants