Skip to content

Make graph views attributes live-update - #325

Merged
JoOkuma merged 17 commits into
royerlab:mainfrom
cmalinmayor:graph-views
Sep 10, 2026
Merged

Make graph views attributes live-update#325
JoOkuma merged 17 commits into
royerlab:mainfrom
cmalinmayor:graph-views

Conversation

@cmalinmayor

@cmalinmayor cmalinmayor commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Add an option for what mode the view is in:

  • Write through (existing, default behavior) - attribute updates propagate to root and are written to the current view, but don't get updates from sibling views
  • Live - attribute updates are written to root, and this view is registered with the root to stay synchronized with any updates to the root
    For a totally detached view, just call subgraph and detach.

Every Live view adds work for the root update, regardless of listeners on the view. Write-through views do not, and are fully updated in the case where there is a single view, so they are preferred for any use case other than multi-view synchronization.

Next steps:

  • add structural updates (adding or removing nodes or edges) to the live view updates - my next step
  • Simplify/clarify what "sync" means - I left all references as intact as possible, other than requiring LIVE views to always "sync", since I wasn't quite sure what sync=False meant
  • Live filtering/dynamic membership based on attribute values

cmalinmayor and others added 3 commits July 28, 2026 16:14
A GraphView was a one-time snapshot: writes to the root were not propagated, so
a view could return stale attribute values and its listeners were never told
anything changed. SQL-rooted views diverged silently; rustworkx-rooted ones only
looked correct because they share the root's attribute dicts.

Roots now track their views and apply changes to them directly rather than
through the signal system. A view is queryable in its own right, so keeping it
current is an invariant — it cannot depend on whether an observer happens to be
subscribed. Views absorb the change first and emit second, so a listener never
sees the root and view disagree.

Also fixes SQLGraph not calling super().__init__(), and pickling, which the
view registry broke for every graph.
Resolves royerlab#324.

A GraphView used to block its root's signal, mutate both graphs, then replay the
root's signal itself, so that a listener on either graph would see both graphs
updated. That coupled two unrelated things: a listener subscribes to one graph
and only needs that graph to be current.

Each graph now emits for itself, once, as soon as it is up to date. Writes
through a view delegate to the root and let the normal view-maintenance path
update the view, so there is a single implementation of "absorb a change"
instead of one per direction. The temporary detach that kept those two
implementations from both running is gone with them.

This also removes a real inconsistency: the replayed payload was rebuilt from a
fixed key list, so a root listener received a different set of attribute keys
depending on whether the write went through a view.
@TeunHuijben

Copy link
Copy Markdown
Contributor

@cmalinmayor, I pushed a fix to a bug that was introduced in this branch, namely:

A view does not necessarily contain all node|edge attributes that the root contains. When the attributes in the view are updated because they come from the root, we should check which attributes are actually present on the view, as implemented in my last commit.

Details:

_apply_root_node_attrs and _apply_root_edge_attrs forwarded every changed key into the view's local store. A view built with an explicit node_attr_keys/edge_attr_keys has no local column for the keys it left out, so the write was rejected:

ValueError: Node attribute key 'bar' not found in graph.
            Expected '['t', 'area']'

Intersect the changed keys with the view's own key list before writing through. Only affects views over a non-rustworkx root; a rustworkx-rooted view shares the root's attribute dicts and never takes this path.

@TeunHuijben

Copy link
Copy Markdown
Contributor

@cmalinmayor, I made this PR more "symmetric" by also propagating the removal of node|edge attributes (remove_node|edge_attr_key) from the root to all its views

@TeunHuijben

Copy link
Copy Markdown
Contributor

Point of concern is the regression in the benchmarks (across all backends), because keeping views up to date is just expensive and will always be slower compared to not doing it...

bit deeper:
GraphMutationsBenchmark.setup() always adds a listener to the view. Before, this was harmless because operations on the root didn't care about viewers/listeners. However, it now does, and every test that was intended only to test something on a root without views/listeners is slower because of this listener. Claude suggests to put listened_view into its own benchmark class, so it only runs for the listener benchmark

@cmalinmayor
cmalinmayor marked this pull request as ready for review September 2, 2026 12:14
@TeunHuijben
TeunHuijben requested a review from JoOkuma September 2, 2026 15:49
@TeunHuijben

Copy link
Copy Markdown
Contributor

Hi @JoOkuma, from a motile_tracker perspective, this PR does what we want it to do. Can you have a look and let me know what we need to change to have it ready to merge? 🙏

@JoOkuma JoOkuma left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @cmalinmayor, thanks for the major cleanup and organization of the codebase you have done with this PR.

I opened cmalinmayor#2 with some changes. Can you review it?

I'm worried about the runtime with listeners, as the benchmark shows.

What do you think of having the communication from root to views off by default?
If we go for that, I think I can also refactor what sync means in GraphView and its behavior.

Comment thread src/tracksdata/graph/_mapped_graph_mixin.py Outdated
Comment thread src/tracksdata/graph/_mapped_graph_mixin.py Outdated
@cmalinmayor cmalinmayor changed the title WIP: Make graph views live-update Make graph views live-update Sep 3, 2026
cmalinmayor and others added 2 commits September 3, 2026 11:54
Co-authored-by: Jordão Bragantini <jordao.bragantini@gmail.com>
@cmalinmayor

cmalinmayor commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author
class ViewMode(Enum):
    """
    How a `GraphView` relates to its root graph after creation.

    UNLINKED
        No relationship to the root at all: never registers with it, so it is
        invisible to the root's per-write maintenance -- writing to the root,
        or to any other view, costs nothing extra because this view exists.
        Writing through this view (`update_node_attrs`, `add_node`, etc.)
        never touches the root either: it behaves like a plain, independent
        `RustWorkXGraph` that happens to remember where it came from.
    WRITE_THROUGH
        Does not register with the root either, so it is still invisible to
        the root's per-write maintenance and stays silently stale with
        respect to writes made elsewhere (directly on the root, or through a
        sibling view). It stays current for its own writes only: writing
        through this view (`update_node_attrs`, `add_node`, etc.) writes the
        root and then patches this view's own local copy in the same call,
        matching the view's pre-registration behavior.
    LIVE
        Registers with the root (`root._views`). The root pushes every write
        -- from anywhere -- into this view, so it is always current, at the
        cost of the root doing maintenance work for this view on every write
        regardless of whether anything is listening. The
        only mode that keeps a view consistent with writes made outside of
        it.
    """

    UNLINKED = "unlinked"
    WRITE_THROUGH = "write_through"
    LIVE = "live"

Started to workshop this and hit enough edge cases that it was worth getting feedback before I went further. Is this like what you were thinking @JoOkuma ?

@cmalinmayor

Copy link
Copy Markdown
Contributor Author

(Noticed that we didn't wire up any actual changes to graph structure yet, just attributes, but we will need that for multi-user edits, and I don't think it's a huge change once we decide on the framework)

@JoOkuma

JoOkuma commented Sep 3, 2026

Copy link
Copy Markdown
Member

@cmalinmayor, yes, that's exactly it.

There are a few edge cases with graph structure that are ambiguous. For example, when you delete an element of the subgraph, are they deleted from the root graph or not (if we assume it's a candidate graph, "not" is the right answer)?

@TeunHuijben TeunHuijben mentioned this pull request Sep 3, 2026
Use shared helpers for updating the local rustworkx view graph,
just sometimes called from root, and sometimes from view.
@cmalinmayor cmalinmayor changed the title Make graph views live-update Make graph views attributes live-update Sep 6, 2026
@cmalinmayor

Copy link
Copy Markdown
Contributor Author

@JoOkuma I think I added back in the old behavior without any performance regressions and with the same functionality 🤞

@JoOkuma

JoOkuma commented Sep 9, 2026

Copy link
Copy Markdown
Member

Hi @cmalinmayor, I'm back from a few days off. I gotta catch up with work, and I'll review this in a few days.

@JoOkuma

JoOkuma commented Sep 10, 2026

Copy link
Copy Markdown
Member

LGTM, @cmalinmayor. Thanks for your patience.

@JoOkuma
JoOkuma merged commit 7523bb3 into royerlab:main Sep 10, 2026
7 checks passed
@TeunHuijben

Copy link
Copy Markdown
Contributor

Thanks @JoOkuma!!

@cmalinmayor
cmalinmayor deleted the graph-views branch September 11, 2026 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants