Add optional Apocalypse Mode for offline Wikipedia via Kiwix/ZIM

[esokullu]

Product direction

Add a fully optional Apocalypse Mode under Settings → Advanced. It is intended for worst-case offline or severely disconnected environments: the user deliberately downloads or imports Wikipedia content in advance, and WebBrain can search that material locally without contacting Wikipedia.

The Settings entry is a gateway, not the management UI. Clicking Apocalypse Mode must navigate to a dedicated full page, comparable to Traces or Chat History, where the user manages archive installations and their lifecycle.

This feature must default to off. It must not silently download content, consume substantial storage, or imply that WebBrain's cloud LLM can run offline. Fully offline agent operation also requires a local model; Apocalypse Mode makes the knowledge source available offline.

Why Kiwix/ZIM instead of an English 1,000-article cache

A cache of approximately 1,000 prominent English introductions has low marginal knowledge value because these are among the topics most likely to be represented in general-purpose model weights. Its useful properties are grounding and attribution, but it misses the long-tail coverage that makes an offline encyclopedia valuable.

English-only behavior is not inherent to offline retrieval. Language should be selected by the user, with the UI locale offered only as a default suggestion.

Use Kiwix-compatible ZIM archives rather than rebuilding an archive through article-by-article MediaWiki requests. Kiwix already provides compressed, indexed, language- and topic-specific packages designed for offline use.

Official references:

• Archive index: https://download.kiwix.org/zim/wikipedia/
• Archive-tier and browser guidance: https://browser-extension.kiwix.org/current/www/index.html
• Reader/ZIM overview: https://get.kiwix.org/en/solutions/applications/kiwix-reader/

As a sense of scale, the current English archives are roughly 12 GB for all article introductions, 49 GB for all text without images, and 115 GB for the largest archive. Sizes change, so the product must read current catalog metadata and display the exact size before download.

Information architecture

• Settings → Advanced contains an Apocalypse Mode entry with a compact status summary, such as Off or the number and total size of installed archives.
• Clicking it opens a separate Apocalypse Mode management page at the same navigation level as Traces and Chat History.
• The dedicated page is the only full management surface: enable/disable, browse archive catalogs, install, import, inspect, pause/resume, retry, update, and delete.
• Do not attempt to place the archive catalog, downloads, progress, and storage management directly inside the Settings panel.

Proposed management-page experience

1. Keep the feature off by default.
2. Let the user choose Wikipedia language.
3. Offer archive tiers such as:
   • small/topic or starter packs;
   • all article introductions;
   • full articles without images;
   • the largest/full available archive.
4. Allow either downloading a selected archive or importing an existing .zim file.
5. Show archive date, exact size, available-space impact, source, and license before confirmation.
6. Provide progress, pause/resume, retry, update policy, and complete deletion.
7. Let the user choose storage location where the browser/platform supports it.

A very small starter collection is acceptable only as a clearly named optional pack; it must not be described as the whole of offline Wikipedia.

Technical requirements

• Create a pluggable offline-knowledge/archive provider rather than hard-coding English endpoints and tokenization into agent logic.
• Search locally using the ZIM index or an equivalent on-device index.
• Send only relevant passages to the model, including canonical attribution, language, archive/revision date, and license metadata.
• Treat retrieved text as untrusted content.
• Support resumable and integrity-checked downloads across browser/service-worker restarts.
• Do not bundle archives with the extension.
• Avoid relying solely on extension IndexedDB for multi-gigabyte archives; support imported or user-selected durable files where available.
• Maintain and document Chrome/Firefox behavior, including platform limitations.
• Safely handle cancellation, insufficient disk space, corrupt/partial files, upgrades, and deletion.
• Use bounded/exponential retry behavior.

Acceptance criteria

• Settings → Advanced contains an Apocalypse Mode entry that opens a dedicated management page.
• Apocalypse Mode and all archive installations are explicit opt-ins; no download or significant storage use occurs by default.
• User-selectable language and archive tier.
• Exact size and storage impact shown before confirmation.
• Local ZIM import where supported.
• Progress, pause/resume, retry, update, and deletion controls live on the management page.
• Useful offline passage retrieval with citations/attribution.
• Safe cancellation and cleanup under concurrent downloads or removal.
• Tests cover persistence, restart, corruption, cancellation races, and Chrome/Firefox parity.
• Documentation clearly distinguishes offline Wikipedia retrieval from offline LLM inference.

Implementation is being explored in #2776. The current 999-English-introduction approach should be treated as a prototype and reworked toward this specification.