feat: Copying Databases

[saul-jb]

It would be great if there was a way to copying databases built in, preferably in a generic way that works with any database type. Here are some potential use cases for this:

• Make a copy of a public database with your own rules keeping history.
• Modify the manifest configuration for your own database.
• Create a snapshot of the state to reduce merkle-crdt size.

I'm not sure how feasible the last one is for generic databases since one would need to know what operations cancel each other out but maybe there is a way to encode the state map as the root of the log?

[tabcat]

I agree, have had this in mind. Have thought it could be useful for updating access control as well, which i guess would go under modifying the manifest configuration use case you mention.

Let's talk about what this might look like. There could be a function called fork. What does it take as parameters and is it a method on some object? Maybe it should be a method on welo that takes a db and a manifest and returns a new manifest with a parent field whose values are the CIDs of the previous manifest and the current root of the db's index? May be worth allowing writers to add entries form the previous database made after the snapshot, if they have write access in bother manifests.

Wondering if this would benefit from a special entry/operation that shows where identities forked the database.

One of the downsides of needing to move to a new database to accomplish these use cases is that the address of the database is different. So if someone has linked to their database from a system like ENS, it will need to be changed. Any usecases requiring frequent migrations . You could add another layer in between which points to the latest db but this would'nt be ideal either.

[saul-jb]

Have thought it could be useful for updating access control as well, which i guess would go under modifying the manifest configuration use case you mention.

Yeah, this would be the most useful thing to do with this but it also has potential problems to address, for one: if we remove an identity from static access are the forked entries invalidated? Are they copied and re-signed with the forkers ID?

Maybe it should be a method on welo that takes a db and a manifest and returns a new manifest with a parent field whose values are the CIDs of the previous manifest and the current root of the db's index?

Yes, I think that adding a field on the manifest that points to the heads it was based on and the manifest that holds the rules that chain is based on could work and solve the above problems. If this is how it would work then I think Welo.fork(Database, Manifest): Manifest should be fine.

May be worth allowing writers to add entries form the previous database made after the snapshot, if they have write access in bother manifests.

I don't see a reason to prevent adding entries to the old database but if this behavior is desired I would expect the user to manage both - unless there is a good use case for having both automatically managed by Welo, I just can't think of one right now.

Wondering if this would benefit from a special entry/operation that shows where identities forked the database.

It would have to be optional of course but a special 301 style pointer would definitely be useful.

One of the downsides of needing to move to a new database to accomplish these use cases is that the address of the database is different. So if someone has linked to their database from a system like ENS, it will need to be changed. Any usecases requiring frequent migrations . You could add another layer in between which points to the latest db but this would'nt be ideal either.

I think the change in address is to be expected, at least in the case of a fork but I can see this will be a bit of a problem for systems like ENS if this is the cost of a snapshot. Perhaps we can separate the snapshot use-case from the rest and find a way to do it without needing a change in address.

At first I thought that we could handle the snapshot case at the same time but after considering the above it might be easiest to handle that completely separately.

[saul-jb]

Another idea for how to handle this is to handle it as a merge rather than a fork. You would create a new database and add a special entry that indicates to include the heads and manifest of a different database to be included, this way you get the additional functionality of merge and a way to perform multiple merges after the creation of a database. Along with the special 301/fork operation in the old database this can provide a fairly wide range of possibility.

This of course ignores the snapshot use case for the time being.