Skip to content

Add Trustabl Agent Scanner to CI - #892

Open
joshua-trustabl wants to merge 1 commit into
rennf93:masterfrom
joshua-trustabl:add-trustabl-action
Open

Add Trustabl Agent Scanner to CI#892
joshua-trustabl wants to merge 1 commit into
rennf93:masterfrom
joshua-trustabl:add-trustabl-action

Conversation

@joshua-trustabl

Copy link
Copy Markdown

We came across your repo and we like that you're building a unique platform for AI agent collaboration and development. We scanned the repo, and noticed agent runtime reliability findings that might be worth reviewing.

  1. [LOW] Tool has no description
    File: roboco/agent_sdk/intake_driver.py, The Claude Agent SDK uses the tool's docstring as the description shown to the model.

Recommendations are based on our understanding of agent runtime reliability, some findings may be intentional. Please let us know if this was intentional or if our findings are helpful so we can improve the accuracy of the scanner.

Best,
Trustabl.ai
Open-source AI agent reliability scanner (runs locally, GitHub Action)

@github-actions

Copy link
Copy Markdown

Thanks for opening your first pull request on RoboCo!

Quick checklist before review (most of these are enforced by CI, but worth a glance):

  • make quality — ruff format check, ruff check, mypy, pytest (≥80% coverage), and the rest of the gate
  • Panel changes pass pnpm lint and pnpm exec tsc --noEmit (run from panel/)
  • No # noqa / # type: ignore shortcuts; pre-existing violations in touched files are fixed
  • Added an entry under ## [Unreleased] in CHANGELOG.md
  • Signed the CLA (the bot will prompt you on this PR)
  • Signed your commits — master requires verified signatures (SSH signing setup)
  • Updated any affected docs under docs/

See CONTRIBUTING.md for the full workflow and the Code of Conduct for the community standards we follow.

Welcome aboard — a maintainer will review shortly.

@github-actions

Copy link
Copy Markdown


Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

@github-actions github-actions Bot added the ci GitHub Actions and CI configuration label Aug 12, 2026
@rennf93 rennf93 self-assigned this Aug 12, 2026

@roboco-app roboco-app Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

This PR adds a third-party security scanner (trustabl/trustabl-action) to CI. The workflow is well-structured in places — minimal top-level permissions, job-scoped write grants, SHA-pinned actions, persist-credentials: false — but the core issue is supply-chain trust: it introduces an unvetted third-party action with security-events:write and pull-requests:write into CI, running on every PR including external forks, while the action fetches a mutable-tagged tool artifact (version: v0.1.6) at runtime. The existing workflows use only first-party actions (actions/, github/). Several convention inconsistencies with the repo's established CI patterns (checkout version drift, a concurrency group that doesn't collapse push+PR, no paths filter) should also be addressed before this is mergeable. Requesting changes pending maintainer vetting of the action and alignment with repo conventions.

Findings

File Line Severity Expected → Actual
.github/workflows/trustabl.yml 30 major Third-party GitHub Actions added to CI are vetted by the org and requested by a maintainer; all existing workflows use only first-party actions (actions/, github/). → Introduces trustabl/trustabl-action@973f666d... from an org outside the repo's trust boundary, granted security-events:write and pull-requests:write so it can upload to the Security tab and post PR comments. SHA-pinning prevents tag mutation but does not vet the action's own code.
.github/workflows/trustabl.yml 32 major Runtime-fetched tool artifacts referenced by CI actions should be pinned to an immutable digest, not a mutable tag. → The action's version: v0.1.6 input is a mutable tag. The SHA-pinned action fetches this version at runtime, so a republished v0.1.6 artifact would execute with the job's write permissions — reintroducing the mutable-tag supply-chain risk inside the pinned action.
.github/workflows/trustabl.yml 25 minor actions/checkout version consistent with the rest of the repo (every existing workflow pins actions/checkout@v7). → Pins actions/checkout@11bd71901bbe...# v4.2.2, a different version than every other workflow, leaving two checkout versions in CI.
.github/workflows/trustabl.yml 13 minor Concurrency group that collapses duplicate push+pull_request runs for the same branch, per the repo's established pattern: ${{ github.workflow }}-${{ github.head_ref
.github/workflows/trustabl.yml 5 minor Triggers scoped with paths: filters so CI doesn't burn runners on irrelevant changes (every existing workflow uses paths:). → Bare pull_request: and push: branches: [master] with no paths: filter runs on every PR and every master push, including docs-only/CHANGELOG-only changes — wasted CI minutes against the org's monthly budget cap.
.github/workflows/trustabl.yml 22 nit An advisory scanner that is meant to surface findings should produce a visible signal when it runs or breaks. → continue-on-error: true with no required-check or status surface means a broken or empty scanner is invisible — no signal that it's working or producing output.

Verdict

changes requested

@rennf93 rennf93 moved this from Backlog to In progress in RoboCo Kanban Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci GitHub Actions and CI configuration

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

2 participants