Design: preserve complete keyring state across daemon restarts

[caniko]

Problem

I hit a lifecycle gap while using oo7 with PAM.

PAM unlocked the Login collection, and oo7-daemon-login delivered the
password through its sealed memfd. Later, a package update restarted
oo7-daemon without ending the desktop session. The helper had already exited,
and no systemd credential was configured, so the new daemon loaded the
collection locked. The user had not logged out or requested Lock().

I previously suggested retaining the helper memfd in systemd's file descriptor
store in #506.
After reviewing the wider lifecycle, I no longer think oo7 should retain the raw
login password for the whole session. The fdstore still looks like a suitable
optional transport, but it should carry keyring-specific restart state instead.

Proposed ownership

Credential acquisition should remain outside oo7. PAM, the desktop,
credentiald, or another provider supplies a secret or key. oo7 validates it,
unlocks its own keyrings, and retains only the state needed to resume those
keyrings after its process restarts.

The split would be:

• The file backend owns validated key-based unlocking and should retain derived
  keys rather than source passwords.
• The server owns a versioned snapshot of persistent keyring and lock state.
• An optional Linux/systemd backend carries that snapshot through the service
  fdstore.
• Desktop and distro integration owns the logout boundary.

Non-systemd behavior should remain unchanged.

First release contract

I think the first release advertising restart continuity should implement the
complete state model rather than a Login-only or collection-only subset:

• Every persistent encrypted and unencrypted file keyring is represented.
• Collection and per-item lock state is restored exactly.
• The raw PAM or prompt password is discarded after key derivation.
• Graceful and crash-triggered daemon restarts restore state.
• Explicit locks remain effective after restart.
• Malformed, stale, mismatched, or oversized state fails locked without partial
  restoration.
• Single-keyring and multi-keyring password changes have crash-consistent
  recovery.
• Restoration finishes before org.freedesktop.secrets is acquired.
• An explicit stop, logout, user-manager exit, or reboot discards state.

The work can still land as small preparatory PRs. The shipped service should not
enable continuity until the complete contract is tested.

Key-based file API

UnlockedKeyring currently retains the source Secret and derives its Key
lazily. I propose deriving the key during unlock and retaining only the key.

The narrow API needed for this is:

• LockedKeyring::unlock(secret), which derives a key and delegates to a
  validated key path.
• An unlock_with_key(key) path for restart restoration.
• An UnlockedKeyring that contains the derived key, not the source password.
• Password rotation that derives the replacement key and drops the source
  password before returning.

This follows the direction discussed in
#506,
where callers may provide either a secret or an already-derived key.

For non-empty keyrings, restart restoration should authenticate every encrypted
item with the supplied key. Empty keyrings have no encrypted content to test,
so their restart entry must instead be bound to the exact keyring file.

Restart state

A snapshot would describe the complete persistent keyring inventory:

RestartState {
    version
    generation
    keyrings: [
        {
            canonical relative file identity
            exact file digest
            format version
            collection lock state
            derived key or explicit unencrypted state
            locked item indexes
        }
    ]
}

Item indexes are safe only when bound to the digest of the exact keyring file.
If the file is unchanged, item order is deterministic. This preserves item lock
state without requiring an on-disk item-ID migration.

The parser should bound the total size and counts, reject unknown versions,
duplicates, invalid key lengths, path traversal, truncation, trailing data, and
stale file digests, and apply restoration all-or-nothing.

The snapshot must not contain passwords, temporary session-keyring data, D-Bus
sessions, client state, or absolute paths.

State updates

The server should rebuild the complete snapshot after successful durable
changes:

• Collection and item lock or unlock.
• Persistent collection creation or deletion.
• Completed migration.
• Persistent item writes or deletion.
• Password rotation.

Failed operations must leave the previous snapshot unchanged. Locking one
collection must not discard continuity for unrelated collections.

Optional systemd transport

On systemd systems, the server could serialize the snapshot into a bounded,
sealed memfd and submit it with FDSTORE=1.

The unit would eventually add:

NotifyAccess=main
FileDescriptorStoreMax=2
FileDescriptorStorePreserve=restart

Two transient slots avoid a loss window while replacing state:

1. Publish generation N+1 under a generation-specific FDNAME.
2. Call sd_notify_barrier() to confirm receipt.
3. Remove generation N.
4. Return to one descriptor at steady state.

If a crash leaves two generations, oo7 can try newest to oldest and select the
newest one matching the current keyring files.

At startup, oo7 should consume descriptors using
sd_listen_fds_with_names(true) semantics, validate and restore them, register
all service objects and aliases, and only then acquire
org.freedesktop.secrets.

Type=dbus can remain. Fdstore notifications only require access to
NOTIFY_SOCKET.

With FileDescriptorStorePreserve=restart, systemd preserves state for an
explicit restart or automatic crash restart and discards it after a full stop.

Password changes

A single-keyring password change should stage re-encryption separately and
replace the live object only after the new file is durable.

A PAM password change may affect several keyrings. Updating them sequentially
can leave a permanent old/new mixture after a crash. I think this needs a
narrow rotation journal:

1. Prepare and fsync every replacement file.
2. Write and fsync a journal containing paths and old/new file digests, but no
   passwords or keys.
3. Commit the journal.
4. Rename every prepared file into place.
5. Publish restart state with the new keys and original lock states.
6. Remove the journal after all files and restart state are committed.

Before journal commit, recovery keeps all keyrings on the old password. After
journal commit, recovery rolls all of them forward. A reboot without fdstore
would finish the disk transaction but start encrypted keyrings locked.

Logout boundary

Restart continuity must not extend beyond the desktop session. A desktop that
treats logout as the end of the unlock lifetime should bind oo7 to its graphical
session, for example with:

PartOf=graphical-session.target

Logout also needs to wait for that target to stop when the user manager remains
alive because lingering is enabled. I see this as desktop/distro policy rather
than portable oo7 core behavior.

Suggested implementation sequence

I would keep this reviewable with at most one small PR open from this series at
a time:

1. Delay D-Bus name acquisition until initialization is complete.
2. Derive keys eagerly and remove retained passwords from UnlockedKeyring.
3. Drain and zeroize PAM secrets after processing and startup replay.
4. Finish durable atomic keyring replacement, including directory fsync and
   temporary-file cleanup.
5. Make single-keyring password rotation transactional.
6. Agree on and implement the bounded restart-state format.
7. Add validated key-based restore.
8. Capture and restore complete state from bytes without systemd.
9. Keep that state synchronized after successful mutations.
10. Add the optional systemd fdstore transport.
11. Add crash recovery for multi-keyring password rotation.
12. Add the release-level restart and fault-injection matrix.
13. Enable fdstore in the shipped unit.

The early PRs are useful correctness and security improvements on their own and
would not add dormant fdstore scaffolding.

Questions

• Does oo7 owning derived restart state fit the intended boundary, while the
  desktop continues to own credential acquisition?
• Is a complete file-bound snapshot an acceptable way to preserve exact item
  lock state without changing the keyring file format?
• Does an optional systemd-specific server path fit oo7's portability goals?
• Is the proposed first-release contract the right point at which to enable the
  unit settings?

If this direction is acceptable, I can start with the D-Bus readiness change
and keep the implementation to one focused PR at a time.

[bilelmoussaoui]

I appreciate the long issue description even if it feels llm generated.

In any case, adding initial APIs that adds support for an already derived key is definitely something we should add, and wouldn't be problematic.

For the rest, the whole restart state, we can discuss that once the initial parts are done.

[caniko]

Opened PR #559 for the already-derived-key file APIs discussed here. Restart-state work remains out of scope and deferred.

[caniko]

PR #559 has merged as 990ce28. The next proposed PR is limited to delaying acquisition of org.freedesktop.secrets until the current startup initialization and buffered PAM replay are complete. It would not add restart state, fdstore, systemd-unit changes, PAM changes, or password-lifetime changes. Does that remain the preferred next step?

[bilelmoussaoui]

I don't think we should do that, given that a user could not have pam setup to pass the secret at all. Or even, use a pam method that won't send us a secret.