[AIT-846] Universal Test Suite (UTS) groundwork#2213
Merged
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughIntroduces a new ChangesUTS Swift Testing Infrastructure
Sequence Diagram(s)sequenceDiagram
participant Test as UTSTestCase (test body)
participant UTSTestCase as UTSTestCase (makeRealtime)
participant MockWebSocketTransportFactory
participant MockWebSocketFactory
participant MockWebSocketProvider
participant ARTWebSocketTransport
participant MockWebSocket
Test->>UTSTestCase: installMock(wsProvider)
Test->>UTSTestCase: makeRealtime { configure options }
UTSTestCase->>ARTWebSocketTransport: init with MockWebSocketTransportFactory
ARTWebSocketTransport->>MockWebSocketFactory: create(request:workQueue:)
MockWebSocketFactory->>MockWebSocket: init
MockWebSocketFactory->>MockWebSocketProvider: register(ws)
MockWebSocketFactory-->>MockWebSocketProvider: onConnectionAttempt(ws) async on workQueue
MockWebSocketProvider-->>Test: handler receives MockWebSocket
Test->>MockWebSocket: respondWithSuccess()
MockWebSocket->>ARTWebSocketTransport: delegate.webSocketDidOpen (async)
Test->>MockWebSocket: send(ProtocolMessage.connected(...))
MockWebSocket->>ARTWebSocketTransport: delegate.didReceiveMessage (async)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
temporarily deployed
to
staging/pull/2213/markdown-api-reference
github-actions
Bot
temporarily deployed
to
staging/pull/2213/markdown-api-reference
github-actions
Bot
temporarily deployed
to
staging/pull/2213/markdown-api-reference
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.claude/skills/uts-to-swift/SKILL.md:
- Around line 23-54: Add language tags to plain-text code fences to resolve
markdownlint issues. In .claude/skills/uts-to-swift/SKILL.md at lines 23-54, add
the text language tag to each of the usage and error-message fences (the three
backtick blocks containing Usage and Error messages). In Test/UTS/README.md at
lines 8-23, add the text language tag to the directory-tree fence. In
.claude/skills/uts-to-swift/SKILL.md at lines 493-505, add the text language tag
to the decision-tree fence. For each fence, change the opening backticks from
``` to ```text.
In `@Test/UTS/Harness/MockTimeProvider.swift`:
- Around line 149-159: The fireBlocksDue function extracts blocks by checking
isCancelled under lock but then dispatches them asynchronously without
re-verifying the isCancelled flag. If cancel() is called between extraction and
execution, the block will still run despite being cancelled. Fix this by adding
an isCancelled check in the for loop where scheduled blocks are executed (the
loop that calls scheduled.queue.async) to ensure cancelled blocks are skipped
even after async dispatch. Additionally, add a regression test that schedules a
block, advances time to trigger extraction, calls cancel() before the async
queue drains, and verifies the block does not execute.
In `@Test/UTS/Tests/rest/unit/TimeTests.swift`:
- Line 34: The test in TimeTests.swift at line 34 uses truncation-based
conversion with Int(date.timeIntervalSince1970 * 1000) which can cause
off-by-1ms failures due to floating-point rounding inconsistencies. Replace the
truncating Int() conversion with explicit rounding (such as using round()
function) before converting to Int to ensure deterministic millisecond
comparison. This will make the equality check against serverTimeMs consistent
and reliable across different system floating-point representations.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: d42e36ee-9dad-4541-9b9f-620340489177
📒 Files selected for processing (20)
.claude/skills/uts-to-swift/SKILL.md.swiftpm/xcode/xcshareddata/xcschemes/ably-cocoa.xcschemePackage.swiftSource/ARTRealtime.mSource/ARTRest.mSource/ARTTestClientOptions.mSource/PrivateHeaders/Ably/ARTTestClientOptions.hTest/Ably.xctestplanTest/UTS/Harness/Captured.swiftTest/UTS/Harness/CapturingLog.swiftTest/UTS/Harness/MockHTTPClient.swiftTest/UTS/Harness/MockTimeProvider.swiftTest/UTS/Harness/MockWebSocket.swiftTest/UTS/Harness/NoOpReachability.swiftTest/UTS/Harness/ProtocolMessage.swiftTest/UTS/Harness/UTSTestCase.swiftTest/UTS/README.mdTest/UTS/Tests/realtime/unit/ConnectionRecoveryTests.swiftTest/UTS/Tests/rest/unit/TimeTests.swiftTest/UTS/deviations.md
maratal
force-pushed
the
AIT-846-uts-groundwork
branch
from
3f262a2 to
d5621d2
Compare
github-actions
Bot
temporarily deployed
to
staging/pull/2213/markdown-api-reference
lawrence-forooghian
force-pushed
the
AIT-846-mock-for-time
branch
2 times, most recently
from
209f1e9 to
d43144e
Compare
maratal
force-pushed
the
AIT-846-mock-for-time
branch
from
d43144e to
24f15a0
Compare
maratal
force-pushed
the
AIT-846-uts-groundwork
branch
from
d5621d2 to
d0c4888
Compare
github-actions
Bot
temporarily deployed
to
staging/pull/2213/markdown-api-reference
github-actions
Bot
temporarily deployed
to
staging/pull/2213/markdown-api-reference
maratal
force-pushed
the
AIT-846-uts-groundwork
branch
from
d1148f6 to
5247491
Compare
maratal
force-pushed
the
AIT-846-uts-groundwork
branch
from
d2f5443 to
280a570
Compare
github-actions
Bot
temporarily deployed
to
staging/pull/2213/markdown-api-reference
maratal
force-pushed
the
AIT-846-uts-groundwork
branch
from
280a570 to
73fe058
Compare
github-actions
Bot
temporarily deployed
to
staging/pull/2213/markdown-api-reference
maratal
force-pushed
the
AIT-846-uts-groundwork
branch
from
73fe058 to
d692e32
Compare
github-actions
Bot
temporarily deployed
to
staging/pull/2213/markdown-api-reference
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
Test/UTS/Tests/realtime/unit/connection/ConnectionRecoveryTests.swift (1)
489-492: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueConsider using failable String initializer for defensive coding.
SwiftLint flags the use of
String(decoding:as:)on line 491, suggesting the failableString(bytes:encoding:)initializer. In this test helper context where we just serialized valid JSON, the data is guaranteed to be valid UTF-8, so the non-failable API is safe. However, using the failable initializer would be more defensive against future changes.♻️ Proposed defensive fix
func toJSONString(_ object: [String: Any]) throws -> String { let data = try JSONSerialization.data(withJSONObject: object) - return String(decoding: data, as: UTF8.self) + guard let string = String(bytes: data, encoding: .utf8) else { + throw NSError(domain: "ConnectionRecoveryTests", code: -1, userInfo: [NSLocalizedDescriptionKey: "Failed to decode JSON data as UTF-8"]) + } + return string }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@Test/UTS/Tests/realtime/unit/connection/ConnectionRecoveryTests.swift` around lines 489 - 492, The toJSONString method currently uses the non-failable String(decoding:as:) initializer on line 491. Replace it with the failable String(bytes:encoding:) initializer to be more defensive. Since the function already throws, use a guard statement to unwrap the optional String result and throw an appropriate error if the UTF-8 decoding fails, ensuring the function maintains its error handling contract.Source: Linters/SAST tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.claude/skills/uts-to-swift/SKILL.md:
- Around line 410-418: The awaitTime function has a control flow issue where the
guard let date else block does not exit, allowing execution to continue to
continuation.resume(returning: date) with a potentially nil date value, which
violates the CheckedContinuation<Date, Never> contract that requires a non-nil
Date. Add a return statement inside the guard-else block after the
Issue.record() call to exit the closure when date is nil, ensuring that
continuation.resume is only reached and called with a valid non-nil date value.
---
Nitpick comments:
In `@Test/UTS/Tests/realtime/unit/connection/ConnectionRecoveryTests.swift`:
- Around line 489-492: The toJSONString method currently uses the non-failable
String(decoding:as:) initializer on line 491. Replace it with the failable
String(bytes:encoding:) initializer to be more defensive. Since the function
already throws, use a guard statement to unwrap the optional String result and
throw an appropriate error if the UTF-8 decoding fails, ensuring the function
maintains its error handling contract.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 76216180-73bd-46af-a3c3-927d63f57990
📒 Files selected for processing (3)
.claude/skills/uts-to-swift/SKILL.mdTest/UTS/Tests/realtime/unit/connection/ConnectionRecoveryTests.swiftTest/UTS/Tests/rest/unit/TimeTests.swift
🚧 Files skipped from review as they are similar to previous changes (1)
- Test/UTS/Tests/rest/unit/TimeTests.swift
lawrence-forooghian
force-pushed
the
AIT-846-mock-for-time
branch
from
24f15a0 to
66dce54
Compare
ARTTestClientOptions is the place to inject test dependencies into clients. Add a reachabilityClass property (defaulting to ARTOSReachability) and have ARTRealtimeInternal read it at init, so tests can install an alternative ARTReachability implementation through testOptions instead of the post-construction setReachabilityClass setter (which remains for existing tests that still use it). Addresses #2210 (comment) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lawrence-forooghian
force-pushed
the
AIT-846-uts-groundwork
branch
from
d692e32 to
78d4eec
Compare
Collaborator
github-actions
Bot
temporarily deployed
to
staging/pull/2213/markdown-api-reference
Collaborator
|
@maratal my comments from the previous review have been addressed, thanks. As mentioned in the previous review, there's a lot of detail here that I didn't get into (comparing the translated tests, looking at some of the mock implementations in detail). I think that what we have here is a good start and hopefully any issues in it will be revealed once we start writing a broader set of LiveObjects tests. Happy to approve once commits sorted out. |
Standalone Swift Testing target (Test/UTS) derived from the language-neutral specs in ably/specification (uts/): mocked WebSocket (real ARTWebSocketTransport over a faked ARTWebSocket), mocked HTTP, MockTimeProvider and Captured/CapturingLog/NoOpReachability/ProtocolMessage helpers. Tests: ConnectionRecoveryTests (RTN16) and TimeTests (RSC16). Built in the Swift 6 language mode. Also adds the /uts-to-swift translation skill and registers UTS in the SwiftPM scheme and xctestplan. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
maratal
force-pushed
the
AIT-846-uts-groundwork
branch
from
78d4eec to
7203852
Compare
lawrence-forooghian
approved these changes
Jun 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Second of two PRs split out from the original combined PR #2210. Stacked on top of #2210 (base branch
AIT-846-mock-for-time) — review/merge that one first.This PR adds the Universal Test Suite (UTS) groundwork: the
Test/UTStarget (harness + mocks + the first translated example tests) and the/uts-to-swifttranslation skill. It builds on theARTTimeProviderabstraction from #2210 to drive deterministic time in tests.Contents
Test/UTS/Harness/):UTSTestCasebase,MockWebSocket/MockWebSocketProvider,MockHTTPClient,MockTimeProvider,NoOpReachability,CapturingLog,Captured<T>, SendableProtocolMessagebuilders.Test/UTS/Tests/): mirror the spec'suts/layout —realtime/unit/ConnectionRecoveryTests.swift,rest/unit/TimeTests.swift..claude/skills/uts-to-swift/): translates a UTS pseudocode spec into Swift tests.ARTTestClientOptions(reachabilityClass,httpExecutor).Builds: SDK + tests ✓ ·
swift test --filter UTS11/11 ✓.Review
Addresses the comments from review 4485155284; per-comment commit links are posted as replies on #2210.
🤖 Generated with Claude Code
Summary by CodeRabbit
Release Notes
New Features
Tests
Documentation
Chores