Self heal session#282
Open
vipin-bs wants to merge 6 commits intobrowserstack:mainfrom
Open
Conversation
- Added interfaces for healing logs and reports in selfheal-utils. - Implemented fetchSelfHealingReportByBuild to retrieve self-healing reports from BrowserStack. - Introduced flattenHealedSelectors to simplify selector extraction from reports. - Enhanced getSelfHealSelectors to support fetching test code alongside selectors. - Created prepareSelfHealingPlanTool to generate structured plans for applying healed selectors. - Added input validation and error handling for self-healing tools. - Implemented tests for fetchSelfHealSelectorTool and prepareSelfHealingPlanTool to ensure functionality and error messaging. - Included handling for various input shapes and credential requirements in the tools.
Contributor
There was a problem hiding this comment.
Pull request overview
Expands the self-heal tooling to support build-scoped self-healing reports and enrich responses with per-session test source code, plus adds a new “plan preparation” tool to help the calling LLM apply locator edits safely.
Changes:
- Add
buildUuidmode tofetchSelfHealedSelectors, including fetching the build self-healing report + test-code context and emitting a prominent warning banner when test code is unavailable. - Introduce
prepareSelfHealingPlantool to normalize multiple input shapes and generate a structured plan (with optional test-code enrichment) without modifying files. - Add new Observability test-code fetch utilities and auth override resolution; update self-heal log parsing to support both Automate and App Automate sessions.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/tools/selfheal.test.ts | Adds comprehensive unit tests for new validation paths, plan output, banner ordering, and fetch-issues guidance text. |
| src/tools/selfheal.ts | Implements buildUuid/sessionId modes, warning banners, session normalization, and registers the new prepareSelfHealingPlan MCP tool. |
| src/tools/selfheal-utils/selfheal.ts | Adds build-scoped self-healing report fetch + report types; updates session log parsing to capture locator types and support app-automate. |
| src/tools/selfheal-utils/fetch-test-code.ts | New helper module to fetch test code per session, classify outcomes, and format context + LLM guidance notes. |
| src/lib/get-auth.ts | Adds resolveBrowserStackAuth to safely merge arg overrides with server config and avoid hard failures when creds are absent. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+46
to
+59
| function friendlyApiError(error: unknown, context: string): string { | ||
| const message = error instanceof Error ? error.message : String(error); | ||
| if (/\b401\b|Unauthorized/i.test(message)) { | ||
| return `Authentication with BrowserStack failed while ${context}. The username/access key is incorrect or the user does not have access to this resource. Ask the user to re-check the credentials.`; | ||
| } | ||
| if (/\b404\b|Not Found/i.test(message)) { | ||
| return `BrowserStack returned 404 while ${context}. The identifier is likely invalid or the build/session has no self-healing data. Ask the user to verify the ID.`; | ||
| } | ||
| if (/\b403\b|Forbidden/i.test(message)) { | ||
| return `BrowserStack returned 403 while ${context}. The provided credentials do not have permission to access this resource.`; | ||
| } | ||
| return `Error ${context}: ${message}`; | ||
| } | ||
|
|
There was a problem hiding this comment.
friendlyApiError is declared but never used. With the repo’s ESLint config (typescript-eslint recommended), this will trigger a no-unused-vars failure and break CI. Either remove this helper or use it in the error path (e.g., when building the tool’s error text) so API failures return the friendlier guidance this function encodes.
Suggested change
| function friendlyApiError(error: unknown, context: string): string { | |
| const message = error instanceof Error ? error.message : String(error); | |
| if (/\b401\b|Unauthorized/i.test(message)) { | |
| return `Authentication with BrowserStack failed while ${context}. The username/access key is incorrect or the user does not have access to this resource. Ask the user to re-check the credentials.`; | |
| } | |
| if (/\b404\b|Not Found/i.test(message)) { | |
| return `BrowserStack returned 404 while ${context}. The identifier is likely invalid or the build/session has no self-healing data. Ask the user to verify the ID.`; | |
| } | |
| if (/\b403\b|Forbidden/i.test(message)) { | |
| return `BrowserStack returned 403 while ${context}. The provided credentials do not have permission to access this resource.`; | |
| } | |
| return `Error ${context}: ${message}`; | |
| } |
Comment on lines
+731
to
+735
| return { | ||
| content: [ | ||
| { | ||
| type: "text", | ||
| text: `Error preparing self-healing plan: ${errorMessage}`, |
There was a problem hiding this comment.
The prepareSelfHealingPlan tool handler’s catch block returns an error message but does not set isError: true. Most other tools in this repo mark failures with isError, and omitting it can cause clients to treat the response as a success. Add isError: true to this error return for consistency and correct downstream handling.
Collaborator
ruturaj-browserstack
left a comment
ruturaj-browserstack
left a comment
There was a problem hiding this comment.
Check copilot comments as well , they are relevant
|
|
||
| tools.prepareSelfHealingPlan = server.tool( | ||
| "prepareSelfHealingPlan", | ||
| "Builds a self-healing edit plan and returns it to the calling LLM as " + |
Collaborator
There was a problem hiding this comment.
Can we optimize these prompts? They are being sent on every chat turn by the MCP client.
…tter context handling
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.