feat[uts]: Pub/Sub unified test suite added#2247
Merged
Merged
Conversation
|
Important Review skippedToo many files! This PR contains 165 files, which is 15 over the limit of 150. To get a review, narrow the scope: Upgrade to a paid plan to raise the limit. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (165)
You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tighten weakened assertions, add missing test coverage, and improve test infrastructure: - Fix accommodate-both patterns in annotations, token_renewal, batch_presence, and publish tests to assert spec-correct behavior - Add fallback host tests: status code variants, cache expiry, endpoint routing, option conflict detection - Add batch_presence, request_endpoint, and additional push/presence tests - Add trackClient safety net for automatic client cleanup - Fix FakeClock: non-zero initial time, setTimeout yield in tickAsync - Update deviations.md with all documented non-conformances - Add msgpack test stubs (pending mock infrastructure support) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Import Ably from internal source types instead of untyped require() so
the test suite compiles cleanly under strict TypeScript checking.
- Replace require('../../build/ably-node') with internal source imports
(DefaultRest, DefaultRealtime, ErrorInfo, ProtocolMessage)
- Add explicit type annotations to eliminate noImplicitAny errors
(captured arrays, callback parameters, catch clauses)
- Add non-null assertions and as-any casts for test mock patterns
- Fix Platform.Config.clearTimeout casts in source files to use
ReturnType<typeof setTimeout> instead of number/NodeJS.Timeout
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace unconditional this.skip() in 27 deviation tests with `if (!process.env.RUN_DEVIATIONS) this.skip()` so that each deviation can be reproduced on demand: RUN_DEVIATIONS=1 npx mocha --grep "RSA7b" test/uts/rest/auth/client_id.test.ts Normal test runs are unchanged (464 passing, 37 pending). With RUN_DEVIATIONS=1: 464 passing, 10 pending, 27 failing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add MockWebSocket infrastructure and realtime client test suite: - mock_websocket.js: MockWebSocket, PendingWSConnection, MockWSInstance - client_options.test.js: RSC1, RSC1a, RSC1b, RSC1c, RTC12 (9 tests) - realtime_client.test.js: RTC1a-f, RTC2-4, RTC13, RTC15-17 (22 tests) - realtime_stats.test.js: RTC5, RTC5a, RTC5b (3 tests) - realtime_request.test.js: RTC9 (5 tests) - realtime_timeouts.test.js: RTC7 (4 tests) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… RTN25) Add 10 new test files covering connection ping, open failures, resume/retry, fallback hosts, heartbeat idle timer, error reason, auto connect, connection ID/key, server-initiated reauth, update events, and whenState. Fix FakeClock zero-time bug (JS falsy 0 broke ably-js idle timer check) and add checkConnectivity() to mock HTTP for fallback host tests. 516 tests passing across full UTS suite. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Apply the same type annotation fixes as the REST tests: - Add `as any` casts to partial connectionDetails objects - Add `as any` casts to Ably.Realtime constructor options - Cast `(Ably as any).RealtimeAnnotations` for plugin access - Add `!` non-null assertions for errorReason property access - Add `: any` type annotations to callback parameters - Type array declarations as `any[]` - Fix prettier formatting across all 36 files Build passes (0 errors), 749 tests passing, 39 pending. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
925 passing, 52 pending (skipped deviations), 0 failing. Adds tests for all realtime UTS specs including presence, auth, channels, connection, and delta decoding. Updates deviations.md with correct classification of all skipped and failing tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace all real setTimeout/setInterval delays with flushAsync() — a setImmediate-based event-loop yield. Test suite drops from 16s to 2s. - Add flushAsync() helper to helpers.ts - Replace ~190 await setTimeout patterns across 28 test files - Convert callback-based "prove a negative" tests to async - Replace delayed mock responses with setImmediate - Track all Platform.Config timers and cancel orphans in restoreAll() to work around ably-js connectWs() timer handle overwrite bug - Skip 5 remaining failing deviation tests (RTC1a, RTL5k, RTN24, RTP17e, TM4) with RUN_DEVIATIONS guard 925 passing, 52 pending, 0 failing in 2s. Process exits cleanly. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
25 new test files covering REST integration (auth, publish, history, presence, pagination, push admin, revoke tokens, mutable messages, batch presence, time/stats) and realtime integration (auth, channels, connection, presence, mutable messages, delta decoding). Also updates deviations.md with integration test deviations: RSC10 (token renewal infinite loop) and RSH1b2 (push pagination missing Link headers). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sandbox HTTP requests (app provisioning and deletion) now have a 30s AbortSignal timeout. Previously these had no timeout, so a stuck sandbox request would silently consume the entire mocha suite timeout. Teardown deletion is also wrapped in try/catch since it's best-effort cleanup — sandbox apps auto-expire. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Suite timeouts must accommodate the sum of all tests plus setup and teardown. 60s was too tight for suites with many tests under sandbox load, causing spurious failures when individual operations stalled. Individual operations already have their own bounded timeouts (30s for sandbox HTTP, 10-15s for connection waits), so the suite timeout now serves as a generous outer bound. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The test was using client-side Date.now() to define time boundaries between "early" and "late" message batches. This failed reliably because publishes completed within the same millisecond, making the boundary meaningless. The test now retrieves server-assigned timestamps from the messages and derives the boundary from those. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move all unit tests under realtime/unit/ and rest/unit/ directories to mirror the specification repo's top-level unit vs integration breakdown. Update import paths and describe() names accordingly. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The test proxy is a Go binary that sits between the SDK and the Ably sandbox, forwarding WebSocket and HTTP traffic while allowing programmable fault injection. Each test creates a proxy session with rules that match specific protocol messages and apply actions: suppress, replace, inject, delay, or refuse connections. The proxy also supports imperative actions (e.g. disconnect the current WebSocket) and exposes an event log for post-hoc verification of traffic patterns. Infrastructure: - helpers/proxy.ts: TypeScript client for the proxy's REST control API. Exports createProxySession(), ProxySession (addRules, triggerAction, getLog, close), and waitForProxy(). - helpers/run-proxy-tests.sh: Builds the Go proxy if needed, starts it, runs mocha against proxy/**/*.test.ts, and kills it on exit. - package.json: test:uts excludes proxy/ tests (they require the proxy binary); test:uts:proxy runs them via run-proxy-tests.sh. Tests (19 test cases across 5 files): - connection_open_failures (RTN14a/b/c/d/g): Fatal errors, token errors, timeouts, and connection refused during the opening handshake. - connection_resume (RTN15a/b/c6/c7/h1/h3): Disconnect/resume, resume preserving connectionId, failed resume with new id, token vs non-token DISCONNECTED errors. - heartbeat (RTN23a): Suppress server frames to starve heartbeats, verify idle timeout fires and SDK reconnects. - channel_faults (RTL4f/5f/13a/14): Attach timeout, detach timeout, unsolicited DETACHED triggering reattach, channel ERROR causing FAILED. - rest_faults (RSC10/RSC15m/REC2c2/RTL6): Token renewal on HTTP 401, 503 with fallback disabled, end-to-end publish+history passthrough. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
7 new proxy integration tests matching the UTS specs: connection_resume.test.ts (3 new tests): - Test 21 (RTN15j): Fatal ERROR mid-session → FAILED state + channels FAILED - Test 22 (RTN15g/g2): connectionStateTtl expiry prevents resume - Test 23 (RTN19a/a2): Unacked messages resent after successful resume channel_faults.test.ts (2 new tests): - Test 24 (RTL12): ATTACHED with resumed=false triggers channel UPDATE event - Test 25 (RTL3d): Both channels reattach after connection recovery auth_reauth.test.ts (1 new test): - Test 26 (RTN22/RTC8a): Server-initiated AUTH triggers re-authentication presence_reentry.test.ts (1 new test): - Test 27 (RTP17i/RTP17g): Presence auto re-enter on non-resumed ATTACHED Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
New test files: - RTB1: backoff coefficient and jitter validation - RSA4a: token expiry with non-renewable tokens (FAILED state) - RSA4c/d/f: auth callback errors (timeout, 403, invalid type, oversize) - RTN16: connection recovery key, msgSerial, channelSerials - RTN20: network change events (skipped — browser-only in Node.js) - RSF1/RTF1: forwards compatibility for unknown fields/actions - RSH7: push channel subscriptions (unit + integration, integration skipped) Extended test files: - RTL22/MFI: message filter subscriptions (name, refTimeserial, clientId) - RTN7e: publish error matches connection.errorReason - CHD2/CHM2: all ChannelMetrics fields including objectPublishers/Subscribers - RTN16d/RTN16l: proxy-based two-phase recovery and recovery failure Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
proxy.ts now builds and spawns the Go proxy binary automatically via ensureProxy() — no need to start it externally before running tests. The proxy is built on first use (skipped if binary is up to date), polled via /health until ready, and killed on process exit. Removes the separate test:uts:proxy script; proxy tests now run as part of the standard test:uts suite. The test:uts:unit script excludes both proxy/ and integration/ directories. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…uccess Tests the race condition where a slow request to a cached fallback host completes after fallbackRetryTimeout has expired. The late success must not re-establish the fallback as the preferred host. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Verifies that when the proxy closes the underlying TCP connection without sending a WebSocket close frame, ably-js detects the TCP FIN and transitions to disconnected with the same minimal delay as the close-frame case. Corresponds to specification issue #464. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…om source Removes the Go toolchain dependency for running proxy integration tests. The binary is fetched from github.com/ably/uts-proxy releases, verified against SHA-256 checksums, and cached in node_modules/.cache/uts-proxy/. Also removes the unused run-proxy-tests.sh script. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ason ably-js behavior is correct — when auth renewal fails while CONNECTED, errorReason should NOT be set because the connection is healthy and the existing token is still valid. Remove deviation skip and update assertions. Refs: ably/specification#466 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
TM4 is about Message constructors (name, data) and (name, data, clientId), not serialization. Replace toJSON deviation test with constructor tests. Remove TM4 deviation entry — ably-js was correct. Refs: #2210 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…, RSA17c) With X-Ably-Version >= 3 (ably-js sends version 6), the server returns a BatchResult envelope with HTTP 200 for all batch responses. Updated all mock responses from legacy format (plain arrays / batchResponse with HTTP 400) to the new format. Removed corresponding deviation entries — these were not real deviations. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…l, RSC15l4) - Fix mock shouldFallback to handle statusCode 408 (request timeout) - Fix mock respond_with_timeout to use code: 'ETIMEDOUT' (matching Node) - Remove RSC15l deviation — test now passes (mock was wrong, not ably-js) - Add proxy integration tests for RSC15l2 (request timeout) and RSC15l4 (CloudFront header). RSC15l2 passes; RSC15l4 skipped as deviation. - Update deviations.md: narrow to RSC15l4 only Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds tests for unreachable endpoint (ECONNREFUSED), connection drop (http_drop), 5xx with/without error body, 4xx not retried, and RSL1k4 idempotent publish (skipped pending proxy response modification support). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Every ably-js UTS test now has a `// UTS: <test-id>` comment mapping it to its corresponding UTS spec. All 1120 UTS Test IDs are covered (1226 total JS tests, with some specs covered by multiple tests via .N suffixes). New tests added across 13 files: - Fallback/endpoint config (REC): 10 tests in fallback.test.ts - REST request handling (RSC): 8 tests across request/rest_client/channel - Batch publish (RSC22): 6 tests in batch_publish.test.ts - Auth token lifecycle (RSA16): 4 tests in token_details.test.ts - Presence get/history (RSP): 14 tests in rest_presence.test.ts - Pagination (TG): 3 tests in paginated_result.test.ts - Presence message types (TP3): 2 tests in presence_message_types.test.ts - Realtime reconnect (RTN23b): 1 test in heartbeat.test.ts - Msgpack deviations: 4 skip stubs across encoding/presence/channel/auth - Integration (RSA7): 1 test in auth.test.ts 18 tests are it.skip() documenting deviations (msgpack not supported, fallbackHostsUseDefault not implemented, connectivity check not accessible, addRequestIds not implemented, switch-to-basic-auth not supported). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Unit tests: endpoint: 'sandbox' → 'test' (not a real environment) - Update host/fallback assertions to match (test.realtime.ably.net) - Proxy helper: use nonprod sandbox hostnames Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add describeEachProtocol helper and update all 10 data-path integration test files to run with both protocol variants. Each test receives the protocol and sets useBinaryProtocol accordingly. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Verify decode and round-trip of all 8 ably-common msgpack fixtures (strings, binary, JSON array/object at various sizes). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Added optional parameter support to multiple functions to ensure compatibility with `ably.d.ts` - Updated tests to include stricter type annotations and address cases with potential type conflicts. - Fixed minor logging inconsistencies, ensuring proper initialization of logger.
github-actions
Bot
temporarily deployed
to
staging/pull/2247/bundle-report
github-actions
Bot
temporarily deployed
to
staging/pull/2247/bundle-report
This was referenced Jun 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is complete UTS changes with polished types, new GitHub workflow and full Pub/Sub coverage, based on:
#2217
#2216
#2212
#2211
#2191