test(e2e): simulated Jellyfin server and Android E2E suite - #45
Merged
Conversation
The two existing instrumented tests both depended on a real external Jellyfin server, so the layer that only exists once the service, Media3 session, Jellyfin SDK and ExoPlayer are wired together had no automated coverage. PaginatedBrowseTest ended in assumeTrue() and silently skipped without a signed-in server with a large library — including in CI, which ran only unit tests — so the pagination fix in 21f0a39 was unguarded. PinnedMediaPathTest hit a live production host. Add a FakeJellyfinServer (MockWebServer) that runs in the app's own process on 127.0.0.1. Instrumentation shares that process, so the service can reach it with no host networking, and onLogin() already re-points the live ApiClient at whatever URL the stored account holds — so a test seeds an account and sends the existing LOGIN_COMMAND, which doubles as the per-test reset. No production code changes. Responses are built from the SDK's own BaseItemDto and encoded with its ApiSerializer, the same Json the app decodes with, so field names, enum spellings and UUID format cannot drift. Requests are answered by filtering one flat item list with the parameters Jellyfin itself accepts, so a single pipeline covers every browse call; unhandled routes return 501 and are recorded rather than surfacing as a mystery empty list. Audio is a WAV tone generated in code, served with real Range support, so no binary fixture is checked in. 38 tests covering the browse tree, pagination, playback, audiobook chapter/position round-trips, failure modes and TLS pinning. Tests that assert audio actually plays bring an app activity to the foreground first: Android 15's audio focus hardening denies focus to an app whose only running component is a bound service, and Media3 then forces playWhenReady to false. Also adds a CI job on an x86_64 automotive image — an automotive system image is mandatory since the manifest requires that hardware feature. This job has not been exercised on a runner; if emulator boot proves flaky there, make it non-blocking rather than leaving main red. Claude-Session: https://claude.ai/code/session_01RnZ7gm5on6xa6kn8SdkEL5
The hand-rolled emulator steps hung: the job sat on "Boot the emulator" for 30+ minutes with no failure. The boot-completed poll was guarded by `timeout 600`, and it never fired — so execution was stuck earlier, on `adb wait-for-device`, which has no timeout and blocks until the job limit when the emulator never registers with adb. The steps were also undiagnosable: the emulator's own output went to $RUNNER_TEMP/emulator.log and was never uploaded, so a boot failure left nothing to read. reactivecircus/android-emulator-runner does support automotive targets — the earlier claim that it did not was wrong; `target` accepts android-automotive and `system-image-api-level` accepts 35-ext15. It owns AVD creation, boot with a real timeout, retries and teardown, so a stuck boot fails the job instead of burning an hour. Give the AVD 4 cores, 4GB RAM and an 8GB disk, and a 900s boot budget: AAOS images are heavier than phone images. Claude-Session: https://claude.ai/code/session_01RnZ7gm5on6xa6kn8SdkEL5
The README had a Building section but nothing about testing, so there was no way to discover either suite from the project's front door — including the fact that the end-to-end tests need no Jellyfin server, which is the main thing a new contributor would otherwise assume they were missing. Add a Testing section covering both commands, the automotive emulator requirement for the instrumented suite (the manifest requires android.hardware.type.automotive, so a phone image will not install the APK), where reports land, and where the test sources and the simulated server live. Point Contributing at it. Also correct the stated prerequisite from "JDK 11 or later" to JDK 17: the module sets jvmToolchain(17) and Java 17 source/target compatibility, so a JDK 11 toolchain does not build the project. Claude-Session: https://claude.ai/code/session_01RnZ7gm5on6xa6kn8SdkEL5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Both existing instrumented tests depended on a real external Jellyfin server, so the layer that only exists once the service, Media3 session, Jellyfin SDK and ExoPlayer are wired together had no automated coverage:
PaginatedBrowseTestended inassumeTrue(unpaged.size > 40)and silently skipped without a signed-in server holding a large library — including in CI, which ran only unit tests. The pagination fix in 21f0a39 was therefore unguarded.PinnedMediaPathTesthit the live hostdiotify.dedyn.io:4433.How
A
FakeJellyfinServer(MockWebServer) running in the app's own process on127.0.0.1. Three facts made this hermetic without touching production code:10.0.2.2.usesCleartextTrafficis already set.onLogin()already re-points the liveApiClientat whatever URL the stored account holds. A test seeds an account and sends the existingLOGIN_COMMAND— which doubles as the per-test reset.userId(it targets Jellyfin ≥ 10.11's token-derived-user routes), so the fake needs no user model.Responses are built from the SDK's own
BaseItemDtoand encoded with itsApiSerializer— the sameJsonthe app decodes with — so field names, enum spellings and UUID format cannot drift. Requests are answered by filtering one flat item list with the parameters Jellyfin itself accepts, so a single pipeline covers every browse call. Unhandled routes return 501 and are recorded, so a missing endpoint fails loudly instead of appearing as an empty browse list. Audio is a WAV tone generated in code and served with realRangesupport, so no binary fixture is checked in.No production code changes. Deliberately no
@HiltAndroidTest— that swapsDashTuneApplicationforHiltTestApplicationand would stop this being end-to-end.Coverage — 38 tests
BrowseTreeE2eTestbrowse_categoriespreference, Latest, album drill-down, favourites, the A–Z index, genres, playlists, folder browse, album art through the ContentProvider, auth header on every callPaginatedBrowseTestPlaybackE2eTestAudiobookE2eTest/UserViewsbooks lookup, collection → book → chapters, AAOS completion extras, book expansion into an ordered chapter queue, chapter resume from the server, position persisted back via the UserData APIFailureModeE2eTestPinnedMediaPathTestVerification
Locally on the
Automotive_PortraitAVD (API 35, arm64):./gradlew :automotive:assembleDebug— passes./gradlew :automotive:assembleRelease— passes (thepackagingblock applies to all variants)./gradlew :automotive:testDebugUnitTest— 362 tests pass./gradlew :automotive:connectedDebugAndroidTest— 38 tests pass, ~48sIn CI on an x86_64 automotive image — both jobs green:
unit-tests2m19s,instrumented-tests6m11s, with the log confirmingStarting 38 tests/Finished 38 testsonemulator-5554rather than a vacuous pass.Worth a reviewer's attention
Playback needs the app in the foreground. Android 15's audio focus hardening (
AS.HardeningEnforcer: Focus request DENIED) refuses focus to an app whose only running component is a bound service; Media3 then forcesplayWhenReadyto false and nothing plays. Tests asserting real audio setforegroundForPlayback = true, which launches the app's ownSettingsActivityfirst. Browse-only tests skip that cost.The CI job went through two revisions (see 59a0365). The first attempt hand-rolled the emulator steps and hung for 30+ minutes on
adb wait-for-device, which has no timeout and blocks until the job limit when the emulator never registers — and it wrote the emulator log somewhere never uploaded, so the failure was undiagnosable. It now usesreactivecircus/android-emulator-runner, which does support automotive targets; my initial note claiming otherwise was wrong. The action owns AVD creation, boot with a real timeout, retries and teardown, so a stuck boot fails fast instead of burning an hour.https://claude.ai/code/session_01RnZ7gm5on6xa6kn8SdkEL5