fix: render standard-test assertion messages correctly

[devin-ai-integration]

Summary

Two assertion messages in the standard test suites never rendered their values, so test_spec / test_check failures printed literal braces instead of the counts that explain the failure.

Resolves (in part) https://github.com/airbytehq/airbyte-internal-issues/issues/17132:

• https://github.com/airbytehq/airbyte-internal-issues/issues/17132

# connector_base.py — implicit concatenation where only the FIRST fragment had `f`
-    f"Expected exactly one CONNECTION_STATUS message. "
-    "Got: {result.connection_status_messages!s}"
+    "Expected exactly one CONNECTION_STATUS message. "
+    f"Got: {result.connection_status_messages!s}"

# source_base.py — no `f` prefix, and the message was a TUPLE
-    "Expected exactly 1 spec message but got {len(result.spec_messages)}",
-    result.errors,
+    f"Expected exactly 1 spec message but got {len(result.spec_messages)}. "
+    f"Errors: {result.errors!s}"

Behavior called out for review: in source_base.py the assert message was a tuple, which is always truthy as an assert message and printed as a tuple repr. It is now a single interpolated string that still includes result.errors. This goes slightly beyond the literal reported defect (the missing f), so flagging it explicitly rather than burying it — happy to split if you'd rather keep the tuple.

The new unit_tests/test/test_assertion_messages.py is a regression guard rather than a behavioral test: it AST-walks every module under airbyte_cdk/test/standard_tests and fails on any assert message that is a tuple, or that contains a {placeholder}-looking substring in a fragment missing the f prefix. Verified it fails on both pre-fix lines (connector_base.py:117, source_base.py:98) and passes after.

The other three defects in the issue are in airbytehq/airbyte (poe-tasks/*.toml) and are handled by a separate PR linked from the issue.

Test plan: poetry run pytest unit_tests/test -q → 124 passed; poetry run ruff check / ruff format --check clean on the touched paths.

Link to Devin session: https://app.devin.ai/sessions/7effc4c9e1fa40f1bf8f6c7aadac0d62
Open in Devin Desktop: https://app.devin.ai/desktop/session/7effc4c9e1fa40f1bf8f6c7aadac0d62?variant=devin

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[github-actions]

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1788275255-fix-standard-test-assert-messages#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1788275255-fix-standard-test-assert-messages

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

• /autofix - Fixes most formatting and linting issues
• /poetry-lock - Updates poetry.lock file
• /test - Runs connector tests with the updated CDK
• /prerelease - Triggers a prerelease publish with default arguments
• /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
• /poe <command> - Runs any poe command in the CDK environment

📚 Show Repo Guidance

Helpful Resources

• CDK API Reference

📝 Edit this welcome message.

[Copilot]

Pull request overview

This PR fixes two assertion messages in the CDK standard test suites that previously printed literal {...} instead of interpolated values, improving the usefulness of test_spec / test_check failure output. It also adds a unit-test guard to prevent similar assertion-message rendering mistakes from being introduced in airbyte_cdk/test/standard_tests.

Changes:

• Fix connector_base.py assertion message by ensuring the fragment containing {result.connection_status_messages!s} is an f-string.
• Fix source_base.py assertion message by replacing a tuple assert-message with a single interpolated string including result.errors.
• Add an AST-based regression test to detect tuple assert messages and placeholder-like {...} substrings that would render literally.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
unit_tests/test/test_assertion_messages.py	Adds an AST-based regression guard against non-renderable assert messages in standard tests.
airbyte_cdk/test/standard_tests/source_base.py	Fixes test_spec assertion message to interpolate counts/errors correctly and avoids tuple messages.
airbyte_cdk/test/standard_tests/connector_base.py	Fixes test_check assertion message so the “Got:” portion interpolates correctly.

Suppressed comments (1)

unit_tests/test/test_assertion_messages.py:64

• Same as above: reporting node.lineno will point at the assert statement rather than the start of the msg expression, making failures easier to locate.

            problems.append(
                f"{source_file.name}:{node.msg.lineno}: assert message contains "
                f"{placeholder} but the fragment is missing the `f` prefix"
            )

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

PyTest Results (Fast)

4 372 tests  +9   4 360 ✅ +9   9m 30s ⏱️ -17s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit d21b65e. ± Comparison against base commit 64bdfc9.

[github-actions]

PyTest Results (Full)

4 375 tests   4 363 ✅  11m 33s ⏱️
    1 suites     12 💤
    1 files        0 ❌

Results for commit d21b65e.

[devin-ai-integration]

CI note for reviewers: all required checks are green (Pytest 3.10–3.13, MyPy, Ruff lint/format, Build & Inspect). The one failure is the downstream Check: destination-motherduck connector test, which fails on check with a MotherDuck credential error rather than anything this diff touches:

AirbyteConnectionStatus(status=FAILED, message="An exception occurred: PermissionError(13, 'Permission denied')")
... Request failed: Your request is not authenticated. Please check your MotherDuck token.

The same job also failed on an unrelated recent CDK PR branch (devin/1787707125-movingwindow-honor-ratelimit-headers), so it looks environmental (secret/token availability) — but I can't fully rule it out from outside, so flagging it rather than dismissing it.

Written by Devin