Repository navigation
feat: review in the document's own language, and release extension 0.3.0 - #28
Merged
Merged
Conversation
The analysis prompt now tells every AI client to detect the document's language from the text (it is never supplied or set by the user), review in it without translating, keep its spelling variant, leave passages in other languages alone, and write notes in the main language. Also: Level 2 is described as sentence structure and clarity instead of the translated term "sentence construction"; quoted text, names, technical terms, code-like text and deliberate fragments are protected; fewer, confident suggestions are preferred; one worked example is included; the response shape is described once, by the schema; and the output rule is repeated after the document. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The skill works out the draft's language and spelling variant from the text, reviews by that language's own rules without translating, writes notes in it, and names it in the reply so the author can correct it. changes.json gains an optional, validated `language` BCP 47 tag, which the page sets on its text panes for screen readers and spellcheck. Level 2 is renamed to "sentence structure and clarity", and quoted text, names, technical terms, code-like text and deliberate fragments are left alone, matching the extension's prompt. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
"Leave passages in another language unchanged" made Codex skip a document's whole second language, and notes followed whichever language came first. The prompt now reviews each passage by its own language's rules, applies both levels to every language, and leaves only short insertions (quotations, loanwords, names) unchanged. Also tightened after testing: each note is in the same language as its from text, with a bilingual worked example showing it; Level 1 errors are always listed at Level 1 even when a Level 2 rewrite covers the sentence; and very short or common words get a neighbouring word so their anchor is unambiguous. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Matches the extension's prompt: every passage is reviewed by its own language's rules, only short foreign insertions are left alone, each note is in the language of the text it is about, and `language` is the language most of the prose is in. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`npm run eval:languages` runs the signed-in CLIs, several times each, on four mixed-language documents with an answer key of planted errors: Hungarian/Spanish in both orders, Swedish/English/Hungarian, and the closely related Swedish/Norwegian/Danish. It scores recall, text that must not change (quotations, names, UK spellings, control sentences), Level 1 rewrites, and notes or replacements in the wrong language. Raw responses are saved so a run can be re-scored with --rescore. It makes real model calls and is not part of npm test or CI; its scoring and language detector have unit tests that are. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Testing showed Codex answering an English request in Hungarian, Spanish or Swedish, extending the notes-in-the-draft's-language rule to its reply. Step 5 now says to reply in the language the author writes in. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… eval
A correct fix of a short word ("was" -> "were") changes most of it, so
the rewrite check now skips anchors under 6 characters.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Nobbettt
marked this pull request as ready for review
September 29, 2026 19:37
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
sharp-pen now reviews every document in its own language, detected automatically from the text, including documents that mix languages. This PR also releases VS Code extension 0.3.0.
Extension
CHANGELOG.mdentry.Skill
changes.json: gains an optional, validatedlanguagetag, such assvoren-GB. The page sets it as thelangof its text panes.Tests and docs
npm run eval:languages: runs the real AI CLIs on four mixed-language documents: Hungarian/Spanish in both orders, Swedish/English/Hungarian, and Swedish/Norwegian/Danish. It scores recall and flags protected text that changed, Level 1 rewrites, and notes in the wrong language. It isn't part of CI; its scoring has unit tests that are.After merging
v0.1.6publishes automatically.extension-v0.3.0onmainto publish 0.3.0 to the VS Code Marketplace.Test plan
npm test, 270 tests, pass.npm run eval:languages, 3 runs per AI tool and document. Claude, Codex and OpenCode pass all four documents. Copilot passes two; it fails both Hungarian/Spanish documents at 86% recall, because its default model misses the Hungarian comma before "aki" and some Spanish agreement. No tool changed protected text or mixed up languages.build.py, 2 runs per tool and document. Claude and Codex caught 100% of planted errors, Copilot 88–100%.build.py: validateslanguage, and the page setslangon its text panes.