Repository navigation
Conversation
Contributor
Preview Deployment
Previews update automatically with new commits to this PR. No cleanup needed: previews are version aliases on the existing workers. Commit 36a6d57 |
… write Publication kept its own guarded puts and retry loops, about sixty lines: `keepingLocal`, `putOver`, two read-twice loops, and a `KEPT_CHANGING` cause. `merge` already decides `epicenter.json` as the writer's whole file with `local` the union of both sides, and `.gitignore` is ordinary text, so both now go through `writeFile` over the host's storage. `epicenter.json` is written as a create that joins the file there (`joining`), so the union keeps every `local` pattern the file lists and a file already holding the result is not written. `.gitignore` is written from the file as it was read to that file plus the missing patterns, so a line another writer adds meanwhile merges in. Retries are the write's own, up to `WRITE_ATTEMPTS`, and a failure is still logged without refusing the write that triggered it; its cause is now the write's error, such as `InvalidPath` for a directory at `epicenter.json`, or `Io` around a lost reply. Replacing a different declaration is logged as any write that replaced another writer's file is. Invariant: every write of `epicenter.json`, by a handle's publication, `folder.write`, Sync, or the merge driver, is decided by `merge`.
…merges instead of stopping Sync Sync stopped and said to move the file when the person made a file, after the checkpoint, at a path the remote adds. A second Sync would merge it, and moving it lost it from where the app expects it: a settings row has a fixed stem, so the first use on two devices at once collided here. Sync now records the file and merges again, as it does an edit to a file the remote removes, and the two files merge as an addition on both sides. An ignored file there still stops Sync, which never takes one into history.
Finishing writes the change from the old head to the published tree into the files as they are, so an arrived insertion the person edited after the cut comes back beside the edit. The record claimed the next Sync always ends where an uninterrupted one would.
fix(todos): notes a save failed on are tried again as the todo's view closes
A handle's publication kept every `local` pattern `epicenter.json` listed, including one a Sync brought from another device, but wrote `.gitignore` from its own definition's patterns alone. After such a Sync, `.gitignore` lacked the pattern, so a hand-run `git add -A` could commit a file Epicenter's own commits leave out. `publishDefinition` now answers the patterns the file lists once published, or the definition's own when the publication failed, and `.gitignore` names those. Invariant: after a handle's publication, `.gitignore` names every pattern its `epicenter.json` lists.
`presentFolder` was exported from opfs-store.ts, and only tests called it: production folders come from `folderIn`, which makes the directory on the first change. It now lives beside the in-memory OPFS stand-in. Invariant: everything opfs-store.ts exports has a production caller or is a type a test names.
Todos and Whispering each kept a byte-identical opfs-holds.test-support.ts, and Capture's browser folder support inlined the same function. The one copy now lives in the folder package's OPFS test support, beside the stand-in it wraps.
…stops Sync, naming it Counting person-made files as ones to record also counted a file whose read failed, and recording it failed with a reason that named nothing. It stays a stop that names the path.
The shell's grant, file, and history commands were faked three times: in tauri-directory.test.ts, in Capture's granted-directory.test-support.ts, and in Whispering's granted-folder.test-support.ts, each with its own error text, expectation check, and hooks. `granted-shell.test-support.ts`, beside the shell history support, is now the one fake: `createGrantedShell(root)` answers every folder plugin command over `node:fs`, runs history through the CLI's engine, records calls, and holds the controls the three suites used (`grant`, `beforeChange`, `afterChange`, `refuseNext`, `loseReplies`, `failReads`). The folder package's opener test, Capture's host leaf test, and Whispering's desktop tests use it; Whispering answers its recorder plugin's `recover` itself.
The closing test slept 20 ms and then required the first save to be held at its rename. On a loaded CI machine the save had not reached it yet, so the test failed with nothing wrong. It now waits for the save to arrive, up to two seconds.
file-browser.ts defined its own basename, dirname, and ancestorsOf, which behave exactly as paths.ts's basename, dirname, and ancestors. The `@epicenter/folder/file-browser` entry now exports the paths.ts helpers, and Honeycrisp and Whispering call `ancestors`. Invariant: one implementation of each path helper.
The desktop opener prepared the definition and the lens only to refuse a bad one before asking for a grant, then dropped the results, and `openDirectoryFolder` prepared both again. Its `lens` was documented as prepared but arrived raw. `openDirectoryFolder` and `openGrantedFolder` now take the prepared `schema` (or `'declared'`) and a prepared `lens`, as `openAtRoot` does. Each opener prepares once: the desktop opener before the grant, as before, and Bun's openers after checking the root, as before. Invariant: every opener prepares its definition and lens exactly once, and every layer below takes them prepared.
The browser folder has no ready marker to claim and no download; `openAtRoot` and the directory adapter still named them. `hashBytes` said it ran before storage transactions that cannot await crypto, which was true of the IndexedDB storage. The Todos host leaf said its page runs the system `git`, where the `epicenter` CLI runs it for the page.
The Git lab's workspace kept its own text editor: `from`, `buffer`, `dirty`, a `saving` promise, and a save loop that advanced `from` to what each landed save sent. That is `createTextEditor` from `@epicenter/folder/svelte`, which every other app edits a file's text with. Each selected file now gets an editor whose `current` is the text as the workspace last read or wrote it and whose `write` is `folder.write`. `flush` saves through it and still rejects when the typed text could not be saved, so Sync, Git actions, and switching files still stop for a failed save. The workspace keeps what is its own: the listing, change reports, the messages it shows, and the Git output.
fix(folder): a file moved or made while Sync runs is recorded and merged, not lost or in the way
`folder$.move` had no app caller. Honeycrisp's and Whispering's rename mutations each chose between `table.rename` for a row and `folder.move` for any other file, though both are the same move: a row's bytes put at the path of its new stem, the old path removed while it holds them. Both now run `folder$.move` for every file, after their own checks and waits, so a rename queues behind writes to its file in the folder's own turn and its paths' queries are read again, as a write's are. The apps keep what is theirs: which names may change, waiting for typed text, and following the open item to its new path. Invariant: every app rename is the query layer's one move mutation.
When a write landed at a move's source after its bytes reached the destination, the move carries that write over, guarded by the bytes it put there. If another writer had changed the destination meanwhile, the move answered `KeptChanging` for the source with `attempts: 3`, though neither was true: the source changed once, and the destination is what changed under it. It now answers `InvalidPath` naming the destination, as a move onto an occupied path does, which is the state it leaves: the file stays at its source, and the destination is the other writer's. A test covers the race through `table.rename`. Invariant: `KeptChanging` from a move means its source changed on every attempt.
Every entry re-exported OpfsDirectoryHandle, OpfsEnvironment, OpfsFileHandle, and OpfsLocks, and nothing imported them from an entry: the tests that build stand-in handles import them from opfs-store.ts. `BrowserFolderOptions.opfs` still names its type. Invariant: the contract exports no type that only test code names.
# Conflicts: # packages/folder/src/browser-storage.test-support.ts # packages/folder/src/bun.ts # packages/folder/src/directory-adapter.ts # packages/folder/src/storage.ts
…local pattern Publishing the patterns through the one write merged the addition into the file as it was at the write, so a pattern another writer removed between publication's read and that write stayed removed while the publication succeeded, and a later plain git add could commit a local file. Publication reads again after each write, up to WRITE_ATTEMPTS, as the guarded loop it replaced did.
…port The one host skeleton computed the process's engine id with crypto.randomUUID when boundary.ts loaded. An insecure origin, such as an http LAN dev server, has no randomUUID, so @epicenter/folder/browser threw at import and the apps failed to boot, where opening used to answer NoFileSystem. The id is made when the first file is staged, as before.
…unning save It waited for a running save and then discarded, so the editor sent what was typed during that save first, though the comment said nothing typed is sent after it. Dropping the text first lets the running save end its loop.
…ey race Concurrent registrations collided on the primary key, which an ON CONFLICT clause naming the client ID (or the resource identifier) does not cover. Inserts now name no conflict target, so every unique key is an arbiter, and a client's declared fields are written by a separate update.
Delete the in-place create kept for Chrome before 102, which could leave a new file empty after a cut write. Opening now answers NoFileSystem with FileSystemFileHandle.move missing, beside the createWritable check, and every create stages and moves into place.
No released build staged under the earlier names, and a leftover one is inert inside .git/epicenter/scratch.
No released build published one. A dev folder that still holds "kv": {}
is refused by name until an app's publication replaces the declaration or
the key is deleted. defineFolder no longer points a kv key at settings.
A row is a regular file directly in its table's directory whose name is a nonempty id that does not start with a dot, then exactly .md; the id is the name without .md. rowStemOfName in paths.ts is that predicate, and listing, get, set, delete, rename, check, query, and change events all read it: rowAddressProblem refuses the empty id and a dot id, a listing offers no directory as a row, and a table's listing is stale only when a row or the table's directory changed. .md, .hidden.md, a.MD, and nested files are never rows.
A table is a top-level directory holding a .epicenter.json, and that file is the table's JSON Schema: what each table entry of the root epicenter.json held. readDefinition discovers tables from these files, so readFolder, epicenter check, epicenter query, and every opener read them, and a schema that cannot be used names its own path. The root epicenter.json is neither read nor written: an app's own folder publishes each table's schema before its first write, epicenter generate writes the schema files and adds local patterns to .gitignore, and no file carries a folder title or local. History reads no schema; the desktop page hands its definition's local patterns to the CLI beside appLocal, and the desktop Folders window lists a folder that holds a table, titled by its directory name.
…it does" This reverts commit f4bf10b.
perf(folder): load each pack once per Git operation and never read ignored files
The desktop and CLI passed a definition's local patterns as appLocal, which protects only untracked files, while the browser and in-process hosts treated them as local, which also untracked committed files: the same definition behaved differently by host, and a desktop Sync could delete a replica's audio. Every host now reads .gitignore and nothing else, as Git does. A tracked file stays tracked whatever .gitignore says, an untracked file it excludes is never committed or read, and a negated pattern is honoured. appLocal, --app-local, the shell commands' app_local argument, isLocal, localMatcher, and sourceFiles are deleted; history and merge take no definition. A definition's local patterns remain only as lines an app's own folder's .gitignore gets before its first write. A publication that fails is tried again before the next write instead of being kept as done.
…r's write boundary epicenter generate read .gitignore, built the new text, and wrote it back with node:fs, so a line another writer added in between was lost, and any read failure counted as an empty file. It now writes each schema with the hash-guarded put and .gitignore with the merging write, under the folder's save lock; only a missing file reads as absent, and a write that fails is the command's error.
…efusing the folder One unusable .epicenter.json, or a schema in a directory that cannot name a table, made every reader of the folder fail. readDefinition now passes over dot directories and names no folder path may hold, and reports each unusable schema as a problem while the other tables read and open. readFolder also names each file directly in a table that looks like a row but is not one (.md, .b.md, a.MD). epicenter check reports both as findings, and query leaves them out by name.
A failed .gitignore write was logged and the change it preceded landed anyway, and on the desktop a recording's audio is written by the shell before the page's first write publishes anything, so audio could be committed unignored. Opening an app's own folder now makes its root .gitignore name the definition's local patterns before the handle exists, and the open is IgnoreUnwritten when it cannot. Each change through the handle checks the lines first and is refused while they cannot be written; schema publication stays best effort. Lines are compared literally, so ' audio/' no longer stands for 'audio/'.
…der an excluded directory Sync read an untracked file where the remote adds one before asking whether .gitignore excludes it, and equal bytes made an ignored file silently tracked. Colliding untracked paths are now classified by the rules first, and an ignored one stops Sync unread, whatever it holds. Capture read every .gitignore, so an unreadable one inside an excluded directory failed the commit; rule files are now read from the root down, passing over untracked ones beneath an excluded directory, as Git does. The native checkpoint's read of an ignored .gitattributes that governs recorded files is the one deliberate exception, now documented.
… is missing Table directories whose names differ only by case, such as Notes and notes, made the whole folder unreadable. Each is now a schema problem and the other tables read. A definition that reads a table whose schema cannot be used is refused with that schema's reason, not only missing table.
Reading rule files from the root down passed over an untracked .gitignore that a pattern such as **/.gitignore names, so its own rules went unapplied and a browser commit took private/secret.key. A rule file is now passed over only when its directory, or one above it, is excluded; one that names itself still applies, and is still no source.
…ote after a failed fetch Opening an app's own folder now writes .gitignore, so a fresh folder's first Sync checkpoints it as a parentless commit before fetching. When that fetch failed, the retry saw no commit of its own making and answered Unrelated forever. The Sync that makes a never-fetched folder's first commit now marks it (refs/epicenter/joining); while the folder has still never fetched, a Sync joins the remote's history over an empty base, and the first successful fetch clears the mark. A folder whose history a person made, or that has fetched, still refuses an unrelated history unless asked.
… its remote after a failed fetch" This reverts commit e624d28.
…de/per-table-schema # Conflicts: # packages/folder/src/git/history.ts # packages/folder/src/git/scope.ts # packages/folder/src/source-scope.test.ts
feat(folder)!: per-table .epicenter.json schemas, one row predicate, and the id column
Once a repository existed, the native backend ignored the captured .gitignore map and asked `git check-ignore`, which rereads the live rule files. A .gitignore emptied between rule capture and selection let an ignored file be read and committed beside the earlier rule bytes that excluded it. Every host now evaluates the captured map with one matcher. `readRules` moves to scope.ts and `ignoredPaths` leaves the backend interface; the native checkpoint reads its rules the same way. `.git/info/exclude` and `core.excludesFile` are no longer honored: per-device policy a browser cannot see and a clone does not carry, against ADR-0592's one rule.
…against captured rules A `.GITIGNORE` on a case-folding disk is the file Git reads, but both hosts skipped it and selected what it ignored. Rule files now compare case-insensitively, as their patterns do; the exact spelling wins when a case-sensitive disk holds both. The native checkpoint decided which special files were ignored from the live rules before capture, so a rule removed in between let an ignored executable be captured and written as plain 100644. A commit now runs the check again with the captured rule map before writing the tree.
A case-folding disk opens `.GITIGNORE` as the folder's rules and a case-sensitive one never reads it, so no single rule decides what it would ignore. Recognizing it case-insensitively made a tracked `.GitIgnore` hide files on a case-sensitive host where Git ignores it. Only a file named exactly `.gitignore` is a rule. Any other spelling is refused from the listing, before anything is read, naming the file and asking to rename it: a commit is blocked, status fails, Sync stops at its checkpoint or as an unrepresentable tree, and `epicenter check` reports it as a file to fix.
…urce Git never reads a rule file inside an ignored directory, so a .GITIGNORE under node_modules/ no longer blocks every commit. Commits, status and epicenter check now look for misspellings only among files the exact .gitignore rules keep.
fix(folder): select native source from the captured .gitignore bytes
A folder whose branch has no commit now fetches before it records anything. Against a remote branch, its files merge with the remote's tree over an empty base into a first commit whose one parent is the remote head; when they add nothing, it takes the remote head as it is. Against no branch, the checkpoint's root commit is pushed as before. A failed fetch on a first sync leaves nothing recorded, so the retry is not stranded as Unrelated. The first-run unrelated-history allowance and its two-parent bootstrap merge are deleted: histories that share no commit merge only with allowUnrelatedHistories. The publishing marker's tree now records the files' tree when the publish began, so an interrupted first publish resumes against what the files held. Proposes ADR-0615, amending ADR-0576 at step 1.
The first commit of a folder with no commit was pushed before it was published. While the branch still had none, any retry (an edit the publish found, a close between push and publish) captured again and merged the files with the commit just sent over an empty base: text merged twice, extra file~local names, deletes undone. recordFirst now captures, merges, and publishes in one hold of the backend's protection, so no version saved meanwhile can land as a root. The commit is a checkpoint whose parent is the remote head; once the branch holds it, Sync sends it as any local commit ahead of the remote. The separate pre-check, push, and moved handling of the first commit are gone. lastCommit reports it. CommitOutcome.parent is deleted: only tests read it, and they now read the parents from Git. The merge's unreachable throw is gone. ADR-0615 states the order, the empty-remote race, the clean break for old markers, and an open decision on ADR-0609's template commit.
fix(folder): fetch before the first checkpoint
The browser backend now speaks smart HTTP itself (git/transfer.ts) and uses isomorphic-git only for object work. A fetch offers the remote-tracking commit and the branch head as haves. isomorphic-git offered only the branch head, which the server does not know whenever the folder holds an unpushed checkpoint, so every such Sync received and stored the whole history again. A push sends the commits no advertised ref reaches, and of each only the tree entries no parent holds at the same path. isomorphic-git walked to the root whenever a merge's other parent was behind the remote head and sent old blob versions with it. With 2,000 files and 50 commits: a fetch with an unpushed commit went from 2,207 objects to 4, and Sync's merge push from 211 to 7. ADR-0616 records the decision.
The push took new commits newest first, so a later commit could mark a tree sent after expanding only the part its parent lacked; the earlier commit that introduced the rest then found it marked, and the remote refused the pack as missing objects. New commits are now taken parents first, so every parent is held by the remote or already entirely in the pack when a child's tree is compared with it. A test reproduces the case, and a fixed set of random histories must pass git fsck --strict. The fetch goes back to git.fetch with singleBranch and the tracking ref as ref, which isomorphic-git sends as its one have. The size test asserts that have line and the object count, so an upgrade that changes it fails. The custom push remains, in git/push.ts. The push refuses a protocol version 2 advertisement and a line that names no ref. setRemote refuses a URL with a username or password. The commit walk keeps a count of unheld queued commits instead of scanning the queue, and inserts by binary search.
fix(folder): browser fetch and push send only what the other side lacks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Epicenter kept data in two models at once: a Yjs store synced through Durable Objects, and file folders opened by a separate Rust owner on desktop. Every folder feature was built twice, desktop folders could not Sync, and Markdown a person or agent edited was a second copy of the truth. This branch makes one model the only one.
A data folder is a directory of Markdown rows in Git. Rows are
<table>/<stem>.md, settings arekv.json, andepicenter.jsonis the folder's schema: a generated, portable contract that an application writes from its code, orepicenter generatewrites from the folder's ownepicenter.ts. History is plain Git, committed only on Sync or Save version (ADR-0529).One TypeScript engine opens a folder everywhere (ADR-0531, ADR-0533): OPFS in a browser,
node:fsin Bun, and a directory the desktop shell grants. The Rust folder owner is deleted. On desktop, the page reads and writes through the folder plugin, which checks every path, compares every save against what the page read, and publishes it under one mutex per folder, so several windows can edit one folder and a save another window overtook isConflict(ADR-0535).Two openers, by who owns the schema (ADR-0535):
The Yjs store is removed (ADR-0534):
@epicenter/app,@epicenter/sync,@epicenter/app-shell,@epicenter/skills, and the apps built only on them, with the store sync's Durable Objects.Sync is plain Git to a remote in the folder's Git config (ADR-0532, ADR-0521). Signed-in folders sync to private hosted Git on Cloudflare Artifacts; the GitHub connector and the account-partitioned folders are deleted.
The Epicenter app gets a Folders window: it lists the data folders under the Epicenter root and edits their rows in a grid, merging a save against the row as the edit began.
Tools work on any data folder by path (ADR-0520):
epicenter checknames every file that does not conform;epicenter queryanswers one read-only SQL statement over the files that conform and names the ones it left out;epicenter generatewritesepicenter.jsonfromepicenter.ts. A table can close withadditionalProperties: falseso undeclared keys are reported (ADR-0536). The SQL projection lives in@epicenter/folderfor the CLI and the Folders window alike.Folds in #2528 (partial query, closed tables, shared projection, Honeycrisp property budget) and the spec for the Folders window replacing Matter.
Changelog
epicenter queryanswers from the files that conform and lists the ones it left outadditionalProperties: falseso undeclared frontmatter keys are reportedNeed help on this PR? Tag
@codesmith-botwith what you need.