diff --git a/book/src/SUMMARY.md b/book/src/SUMMARY.md index 9d130158ca9..5d918c949c7 100644 --- a/book/src/SUMMARY.md +++ b/book/src/SUMMARY.md @@ -74,6 +74,7 @@ - [Ranked Index Examples](drive/ranked-index-examples.md) - [Time-Range Index TTL](drive/time-range-ttl.md) - [Index-Only Document Types](drive/index-only-document-types.md) +- [Keep-History Documents](drive/keep-history-documents.md) # Testing diff --git a/book/src/contributing/coding-conventions.md b/book/src/contributing/coding-conventions.md index 19b6fda27e1..8b7a38607cb 100644 --- a/book/src/contributing/coding-conventions.md +++ b/book/src/contributing/coding-conventions.md @@ -92,6 +92,77 @@ the method's number in the new protocol version's tables only. Duplication between generations is the accepted cost; it is cheaper than a drift-prone flag. +### A new input changes the signature, not the method count + +When the new generation of a method needs something its callers do not pass +yet (the block, the signer, a mode), add the parameter to the method and +update the callers. Do not add a second method beside it (`*_with_lifecycle`, +`*_with_block`), a second operation variant beside the existing one +(`DeleteDocumentWithLifecycle` next to `DeleteDocument`), or a wrapper that +fabricates the missing value on the way in. One method per directory means +one signature, the one the current generation needs. + +The shipped generation keeps its own signature. The dispatcher hands it the +part of the new input it always had and drops the rest: + +```rust +pub fn delete_document_for_contract_operations( + &self, + document_id: Identifier, + contract: &DataContract, + document_type: DocumentTypeRef, + block_info: &BlockInfo, // was `block_time_ms: u64` + deleter_id: Option, // new: whom the lifecycle record credits + previous_batch_operations: Option<&mut Vec>, + estimated_costs_only_with_layer_info: &mut Option< + HashMap, + >, + transaction: TransactionArg, + platform_version: &PlatformVersion, +) -> Result, Error> { + match platform_version.drive.methods.document.delete.delete_document_for_contract_operations { + // The shipped generation never knew a deleter; it reads the time off + // the block exactly as it read `block_time_ms` before. + 0 => self.delete_document_for_contract_operations_v0( + document_id, + contract, + document_type, + previous_batch_operations, + estimated_costs_only_with_layer_info, + block_info.time_ms, + transaction, + platform_version, + ), + 1 => self.delete_document_for_contract_operations_v1( + document_id, + contract, + document_type, + block_info, + deleter_id, + previous_batch_operations, + estimated_costs_only_with_layer_info, + transaction, + platform_version, + ), + version => Err(Error::Drive(DriveError::UnknownVersionMismatch { .. })), + } +} +``` + +Why: a twin method is a second name for the same behaviour with the version +decision split across two dispatchers, and callers that reach the old name +silently get the old capability at the new protocol version, or a value +somebody made up to satisfy the old signature. Changing the signature makes +the compiler find every caller, and each one then states what it passes. + +How: change the dispatcher's signature, re-point the callers, and let +generation 0 keep receiving what it received before. If a wrapper's new +generation would only forward the new parameter to the method that has the +behaviour, it is not a new generation: forward it from the wrapper's existing +body and leave the wrapper's version slot alone. The one exception is a +public client-facing API that must stay source-compatible across a release; +there, the SDK's builder pattern absorbs the new input. + ### Table versions follow protocol-version boundaries, not PRs Version-table constants (`DRIVE_ABCI_VALIDATION_VERSIONS_V10`, diff --git a/book/src/drive/keep-history-documents.md b/book/src/drive/keep-history-documents.md new file mode 100644 index 00000000000..1d5c4cf66cd --- /dev/null +++ b/book/src/drive/keep-history-documents.md @@ -0,0 +1,292 @@ +# Keep-History Documents + +> **Status:** implemented and gated at protocol version 15 (meta-schema v4 / +> parser generation 4). The storage layout, the delete and erase paths and +> the fee accounting are pinned against a real grovedb by +> [`lifecycle/tests.rs`](https://github.com/dashpay/platform/blob/v4.3-dev/packages/rs-drive/src/drive/document/lifecycle/tests.rs) +> in rs-drive; the full ABCI pipeline (transitions, validation, proofs, +> refunds) by the `keep_history`, `deletion`, `erase` and +> `lifecycle_contracts` modules under rs-drive-abci's batch tests; and the +> whole story against a running network by +> [`KeepHistoryDocument.spec.js`](https://github.com/dashpay/platform/blob/v4.3-dev/packages/platform-test-suite/test/functional/platform/KeepHistoryDocument.spec.js). + +A document type declared with `documentsKeepHistory: true` retains every +revision of every document instead of overwriting the previous one. This +chapter is the reference for how those revisions are stored, how a +keep-history document is deleted and later erased, what the four lifecycle +states mean, and how `getDocumentHistory` reports all of it. + +## Storage layout + +Every document type owns three reserved single-byte keys under its +document-type tree. Index trees are keyed by property names, which cannot +start below `0x30`, so no index can collide with them. + +```text +[DataContractDocuments, contract_id, 1, , 0, ] + → Reference to the current revision (the primary-key tree) + +[DataContractDocuments, contract_id, 1, , 1, ] + → Item() (the lifecycle tree) + +[DataContractDocuments, contract_id, 1, , 2, ] + → ProvableCountTree (the history tree) + [] → Item() +``` + +For a type that does not keep history the primary-key tree holds the +serialized document itself and keys `1` and `2` do not exist. For a +keep-history type: + +- **The history tree** holds every retained revision of one document, keyed + by the block time it was written at followed by its revision, both + big-endian `u64`. Two revisions written in one block therefore stay + distinct. It is a provable count tree, so how many revisions a document + retains is a single hash-bound read, and its absence is provable. +- **The primary-key entry** is a reference to the newest revision, so every + ordinary by-id read and every secondary index resolve through it. For a + summable type the reference also carries the document's sum contribution. +- **The lifecycle record** exists exactly while the document is deleted or + erasing. It is what tells a deleted document apart from an id that was + never used: both are absent from the primary-key tree. The record is + `DocumentLifecycleRecord` in `rs-dpp` (`packages/rs-dpp/src/document/lifecycle`), + a versioned enum with the ordinary platform-serialization derives, stored + with the storage flags of the identity that deleted the document. + +The record carries the block time the document was deleted at, the revision +it carried then, how many revisions the history retained at that moment, +and, once an erasure has been authorized, the block time the erasure started +and the newest revision that was retained when it started. The revision +count matters because a history written before protocol 15 can have gaps +(two writes in one block overwrote each other's revision); a by-revision +read is only meaningful when the retained revisions number one through the +deleted revision without a gap, and the record is where that is decided. + +The reserved key `1` and the whole lifecycle tree are per type and +unflagged, so no single deleter pays for structure nobody refunds; each +record inside it carries the deleter's flags and is what an erase refunds. + +## Contract grammar + +Protocol 15's parser generation 4 adds two rules to keep-history types: + +| Keyword | Meaning | Constraints | +|---|---|---| +| `canBeDeleted: true` | The owner may delete a document. Previously refused on a keep-history type, since the storage layer could not remove one. | Ordinary keyword, now allowed together with `documentsKeepHistory`. | +| `canBeErased: true` | The retained revisions of a deleted document may be purged. | Requires `documentsKeepHistory` and `canBeDeleted`; defaults to `false`; immutable across contract updates. | + +Declaring `canBeErased` on a type that keeps no history, or whose documents +can never be deleted, is a contract-structure error rather than an ignored +default: a type whose declared behaviour and reachable behaviour disagree +is an authoring mistake. The check runs even without full validation, +because the flag governs an irreversible operation. + +A keep-history type may **not** carry a contested index. A contested +resource is awarded by block processing, outside transition validation, at +an id derived from the winner rather than from the contested values, and on +a keep-history type that award could land on an id whose retained history +already exists. Lifting this needs changes to the contested machinery +itself; until then the parser refuses the combination. + +Types registered under protocol 14 or earlier keep their frozen grammar: a +keep-history type that allows deletion is still refused by the generation 3 +parser, so nothing that exists today can enter the lifecycle without a +contract update at protocol 15. + +## The four lifecycle states + +```text + delete erase (first chunk, erase (terminal chunk) + Active ──────────► Deleted ──────────────────────► Erasing ──────────────────────► Absent + │ by the owner) by anyone + │ + └── erase of a history that fits in one chunk goes straight to Absent +``` + +| State | Primary-key entry | Lifecycle record | History tree | +|---|---|---|---| +| **Active** | reference | none | every revision | +| **Deleted** | none | deleted-at only | every revision retained at deletion | +| **Erasing** | none | deleted-at and erasing-from | shrinking, newest revisions removed first | +| **Absent** | none | none | none; the id is free again | + +`DocumentLifecycleState` in `packages/rs-drive/src/drive/document/lifecycle/fetch` +is the one read that classifies a document. It is deliberately not folded +into the by-id fetch every document action performs: only stateful +validation of an action on a keep-history type needs the extra reads, and +they are ordered by how likely they are to settle the question. The +ordinary by-id read answers for every live document; only a miss pays for +the lifecycle record; only a deleted document pays a third read, for the +newest revision it retains, which is where its owner is read from. + +## Delete + +A delete on a keep-history document removes the primary-key reference and +every index reference, and writes the lifecycle record in their place. The +revisions are untouched, so `getDocumentHistory` keeps serving them. + +Two properties follow from the design: + +- **Delete never escalates.** It has no code path that removes a revision, + so a second delete of the same document is a paid consensus error + (`DocumentNotFoundError`, exactly as for an id that holds nothing) rather + than a deeper removal. +- **A deleted document keeps its id.** Creating over it is refused in + transition validation and, independently, in the storage writer: + appending to a history that still retains revisions would silently merge + a new document into a deleted one's record, and the writer's check is + what covers callers that never pass through validation. + +Every Drive delete entry point takes the block the delete belongs to and the +deleter's identity, because the record has to name both. + +## Erase + +An erase purges the retained revisions of a document that has already been +deleted. It is a separate transition kind, `DocumentTransition::Erase`, +appended to the shipped enum the same way every earlier kind was, carrying +nothing beyond the base transition. Its structure validation requires the +type to keep history and to declare `canBeErased`; its state validation +depends on the lifecycle state: + +| Lifecycle state | Result | +|---|---| +| Active | refused; deleting is a separate intent with its own permission and cost | +| Deleted | allowed only for the document's owner, read from the newest retained revision | +| Erasing | allowed for **any** identity | +| Absent | `DocumentNotFoundError` | + +**Erasure is authorized once.** The owner commits the document to erasure +with the first chunk; the record that commitment leaves in state is then the +evidence that destruction was authorized, so any identity may submit the +remaining chunks and pay for them. An owner who loses their keys, their +funds or their permission cannot strand a half-erased document. For the same +reason an erase carries no token payment: the deletion it follows was +charged when the document was deleted, and a payment on a transition anyone +may submit would let a continuation move tokens. + +**Chunks.** One transition removes at most +`max_document_revisions_erased_per_transition` revisions (100 from +`SYSTEM_LIMITS_V5`), newest first, so a partial erasure leaves the oldest +content behind and the retained sequence stays contiguous from one. The +enumeration reads one revision more than it may remove so that it knows, +before emitting anything, whether this chunk is the last one: + +- a **non-terminal first chunk** overwrites the record with the erasure it + authorizes; +- a **terminal chunk** removes the record and the now empty history subtree, + which GroveDB accepts only because the revision deletes are in the same + batch and refuses otherwise. + +The two never happen together, so one batch never carries two operations on +the record's key. + +**Accounting.** Removing a revision credits whoever paid for it, through the +storage flags every revision carries, in balance updates applied after the +transition's fee result is formed. The admission estimate cannot know how +many revisions a document retains, so it prices a full chunk of the type's +largest documents plus both endings; every erase is admitted against the +same worst-case estimate and the actual fee is what it removed. GroveDB's +query surface has no key-only result shape over a range, so the enumeration +reads revision bodies it discards, and the estimate charges for that read +honestly. + +## The `getDocumentHistory` query + +The query reads a page of a keep-history document's retained revisions +together with the document's lifecycle as of the same block. + +**Request.** A contract id, document type name and document id, an optional +`limit` (at most 10, the default), and exactly one selector: + +| Selector | Meaning | +|---|---| +| `startAtMs` | inclusive lower bound on block time | +| `startAfter { timeMs, revision }` | exclusive cursor returned with the previous page; the revision is needed because several revisions can share a block time | +| `startAtRevision` | inclusive lower bound on revision; refused on a history with gaps | +| `revision` | exactly one revision; requires limit one | + +**Response.** A page of entries, each the block time, revision and +serialized document, and a `Lifecycle` message: + +| Field | Meaning | +|---|---| +| `state` | `ACTIVE`, `DELETED`, `ERASING` or `ABSENT` | +| `remaining_revisions` | how many revisions the history still retains | +| `deleted_at_ms` | zero unless the document has been deleted | +| `erasing_started_at_ms` | zero unless an authorized erasure has started | +| `erasing_from_time_ms`, `erasing_from_revision` | the newest revision retained when the erasure started | + +The state is derived identically in Drive's fetch and in the proof verifier, +and the verifier checks every claimed field against the proof. + +**Proof.** The proved response carries one proof object whose grovedb proof +holds two GroveDB proofs in a small versioned envelope, because they answer +two queries GroveDB cannot merge: the offset-paginated proof over the +document's history tree, and the exact-key proof over the current pointer, +the lifecycle record and the history tree's count. Both commit to the same +root hash, signed once. + +## Client surfaces + +**Rust SDK.** `DocumentHistory::fetch` with a `DocumentHistoryQuery` from +`dash-platform-queries` reads a page. `DocumentEraseTransitionBuilder` and +`Sdk::document_erase` submit one chunk; the result is named for what its +proof authenticates, which is that the document is absent by id, something +that was already true before the erase ran. That is why the call takes the +affected-state wait rather than the strict one. `Sdk::document_current_lifecycle` +is the separately named read for how much history is left. + +**JavaScript.** The evo-sdk `documents` facade gains `history()`, +`historyWithProof()` and `erase()`; the WASM SDK exposes them as +`getDocumentHistory`, `getDocumentHistoryWithProofInfo` and `documentErase`. +Every history selector is an exact `u64`: a `number` is accepted only up to +`Number.MAX_SAFE_INTEGER`, anything larger must be a `bigint`. + +```typescript +// Delete, then erase in chunks until the history is gone. The first erase +// is signed by the owner; later ones may be signed by any identity. +await sdk.documents.delete({ document, identityKey: ownerKey, signer }); + +let lifecycle; +do { + await sdk.documents.erase({ + document: { id, ownerId, dataContractId, documentTypeName }, + identityKey: ownerKey, + signer, + }); + ({ lifecycle } = await sdk.documents.history({ + dataContractId, documentTypeName, documentId: id, startAtMs: 0n, limit: 1, + })); +} while (lifecycle?.state === 'ERASING'); +``` + +## Versioning + +Everything above is selected only from `PLATFORM_V15`. Released tables gain +dormant slots, and protocols 12 through 14 replay unchanged: a keep-history +delete still ends in `InvalidDeletionOfDocumentThatKeepsHistory` at 12 and +13 and in a paid rejection at 14, and an erase transition is refused at +basic-structure validation below 15 by its per-kind bounds slot. A +keep-history type registered under protocol 14 can never become deletable, +so the migrated storage layout and the lifecycle are only ever exercised +together on types updated at 15. + +## Rules and guidelines + +**Do:** +- Read a keep-history document's state through `fetch_document_lifecycle` + whenever the difference between deleted and never-existed matters. An + ordinary by-id read cannot tell them apart. +- Pass the block and the deleter to every Drive delete entry point; the + record needs both. +- Treat an erase's proof as an observation of the affected state and read + the lifecycle separately to learn what is left. + +**Do not:** +- Declare `canBeErased` without `documentsKeepHistory` and `canBeDeleted`, + or a contested index on a keep-history type. The parser refuses both. +- Add a token cost to an erase. Any identity may submit a continuation. +- Rely on a by-revision selector over a history written before protocol 15 + without checking `remaining_revisions` against the deleted revision; a + gapped history refuses it. diff --git a/book/src/evo-sdk/state-transitions.md b/book/src/evo-sdk/state-transitions.md index 4526d7085db..aadb2810db3 100644 --- a/book/src/evo-sdk/state-transitions.md +++ b/book/src/evo-sdk/state-transitions.md @@ -84,12 +84,15 @@ await sdk.documents.create({ }); ``` -### Replace, delete, transfer +### Replace, delete, transfer, erase The `sdk.documents` facade also provides `replace()`, `delete()`, -`transfer()`, `purchase()`, and `setPrice()` methods. See the +`transfer()`, `purchase()`, `setPrice()` and, from protocol version 15, +`erase()` for purging the retained revisions of a deleted keep-history +document, with `history()` to read what is left. See the [API reference](https://dashpay.github.io/evo-sdk-website/docs.html) for -parameters. +parameters and the [Keep-History Documents](../drive/keep-history-documents.md) +chapter for the lifecycle those calls drive. ## Token operations diff --git a/packages/dapi-grpc/protos/platform/v0/platform.proto b/packages/dapi-grpc/protos/platform/v0/platform.proto index 4ac2d07bf38..50b9cb615f5 100644 --- a/packages/dapi-grpc/protos/platform/v0/platform.proto +++ b/packages/dapi-grpc/protos/platform/v0/platform.proto @@ -1857,10 +1857,21 @@ message GetDocumentHistoryResponse { enum State { ACTIVE = 0; ABSENT = 1; + DELETED = 2; + ERASING = 3; } State state = 1; // Revisions the history still retains for the document. uint64 remaining_revisions = 2 [ jstype = JS_STRING ]; + // Zero unless the document has been deleted. + uint64 deleted_at_ms = 3 [ jstype = JS_STRING ]; + // Zero unless an authorized erasure has started. + uint64 erasing_started_at_ms = 4 [ jstype = JS_STRING ]; + // Timestamp of the newest revision retained when the erasure started. + uint64 erasing_from_time_ms = 5 [ jstype = JS_STRING ]; + // History sequence of the newest revision retained when the erasure + // started. + uint64 erasing_from_revision = 6 [ jstype = JS_STRING ]; } // A page of retained revisions and the document's lifecycle as of the // same block. diff --git a/packages/js-evo-sdk/src/documents/facade.ts b/packages/js-evo-sdk/src/documents/facade.ts index 928c4a68758..a1a70a0392f 100644 --- a/packages/js-evo-sdk/src/documents/facade.ts +++ b/packages/js-evo-sdk/src/documents/facade.ts @@ -121,6 +121,19 @@ export class DocumentsFacade { return w.documentDelete(options); } + /** + * Erases a chunk of the retained revisions of an already deleted document. + * + * A document with more retained revisions than one transition may remove + * needs several calls. The first must be signed by the document's owner and + * commits the document to erasure; any identity may sign the ones after it. + * Read `history` to see how many revisions are left. + */ + async erase(options: wasm.DocumentEraseOptions): Promise { + const w = await this.sdk.getWasmSdkConnected(); + return w.documentErase(options); + } + async transfer(options: wasm.DocumentTransferOptions): Promise { const w = await this.sdk.getWasmSdkConnected(); return w.documentTransfer(options); diff --git a/packages/js-evo-sdk/tests/unit/facades/documents.spec.ts b/packages/js-evo-sdk/tests/unit/facades/documents.spec.ts index f2dcb4edefa..337c0a67750 100644 --- a/packages/js-evo-sdk/tests/unit/facades/documents.spec.ts +++ b/packages/js-evo-sdk/tests/unit/facades/documents.spec.ts @@ -26,6 +26,7 @@ describe('DocumentsFacade', () => { let documentCreateStub: SinonStub; let documentReplaceStub: SinonStub; let documentDeleteStub: SinonStub; + let documentEraseStub: SinonStub; let documentTransferStub: SinonStub; let documentPurchaseStub: SinonStub; let documentSetPriceStub: SinonStub; @@ -92,6 +93,7 @@ describe('DocumentsFacade', () => { documentCreateStub = this.sinon.stub(wasmSdk, 'documentCreate').resolves(); documentReplaceStub = this.sinon.stub(wasmSdk, 'documentReplace').resolves(); documentDeleteStub = this.sinon.stub(wasmSdk, 'documentDelete').resolves(); + documentEraseStub = this.sinon.stub(wasmSdk, 'documentErase').resolves(); documentTransferStub = this.sinon.stub(wasmSdk, 'documentTransfer').resolves(); documentPurchaseStub = this.sinon.stub(wasmSdk, 'documentPurchase').resolves(); documentSetPriceStub = this.sinon.stub(wasmSdk, 'documentSetPrice').resolves(); @@ -291,6 +293,37 @@ describe('DocumentsFacade', () => { }); }); + describe('erase()', () => { + it('should erase the retained revisions of a deleted document', async () => { + const options = { + document, + identityKey, + signer, + }; + + await client.documents.erase(options); + + expect(documentEraseStub).to.be.calledOnceWithExactly(options); + }); + + it('should accept document identifiers instead of a Document instance', async () => { + const options = { + document: { + id: '4mZmxva49PBb7BE7srw9o3gixvDfj1dAx1K6z4A7P9Ah', + ownerId: '5mjGWa9mruHnLBht3ntBi8CZ6sNk3hZZsQMgTvgQobjS', + dataContractId: 'GWRSAVFMjXx8HpQFaNJMqBV7MBgMK4br5UESsB4S31Ec', + documentTypeName: 'note', + }, + identityKey, + signer, + }; + + await client.documents.erase(options); + + expect(documentEraseStub).to.be.calledOnceWithExactly(options); + }); + }); + describe('transfer()', () => { it('should transfer document ownership to another identity', async () => { const recipientId = '6o4vL6YpPjamqnnPNpwNSspYJdhPpzYbXvAJ4PYH7Ack'; diff --git a/packages/platform-test-suite/test/functional/platform/KeepHistoryDocument.spec.js b/packages/platform-test-suite/test/functional/platform/KeepHistoryDocument.spec.js new file mode 100644 index 00000000000..9f6029adc5c --- /dev/null +++ b/packages/platform-test-suite/test/functional/platform/KeepHistoryDocument.spec.js @@ -0,0 +1,242 @@ +const Dash = require('dash'); +const { expect } = require('chai'); + +const createClientWithFundedWallet = require('../../../lib/test/createClientWithFundedWallet'); +const waitForSTPropagated = require('../../../lib/waitForSTPropagated'); +const { getEvoSdkForNetwork } = require('../../../lib/test/createPlatformProofVerifier'); + +const { + Errors: { + StateTransitionBroadcastError, + }, +} = Dash; + +/** + * The identity's first key and a signer holding its private key, in the shape + * the Evo SDK's write methods expect. The legacy client owns the wallet, so the + * key has to be lifted out of it. + * + * @param {Object} evo + * @param {Object} evoSdk + * @param {Object} client + * @param {Object} identity + * @returns {Promise<{ identityKey: Object, signer: Object }>} + */ +// Key 1 is the identity's HIGH authentication key: document transitions +// require a HIGH or CRITICAL key, and key 0 is the MASTER key. +const DOCUMENT_SIGNING_KEY_ID = 1; + +async function evoSignerFor(evo, evoSdk, client, identity) { + const account = await client.getWalletAccount(); + const { privateKey } = account.identities.getIdentityHDKeyById( + identity.getId().toString(), + DOCUMENT_SIGNING_KEY_ID, + ); + + const signer = new evo.IdentitySigner(); + signer.addKeyFromWif(privateKey.toWIF()); + + const fetched = await evoSdk.identities.fetch(identity.getId().toString()); + const identityKey = fetched.getPublicKeyById(DOCUMENT_SIGNING_KEY_ID); + + return { identityKey, signer }; +} + +describe('Platform', () => { + describe('KeepHistoryDocument', function main() { + this.timeout(900000); + + let client; + let evo; + let evoSdk; + let identity; + let dataContract; + let note; + + /** + * The revision count and lifecycle metadata Platform authenticates for the + * note, read through the Evo SDK's history query. + */ + const readHistory = async () => { + const result = await evoSdk.documents.history({ + dataContractId: dataContract.getId().toString(), + documentTypeName: 'note', + documentId: note.getId().toString(), + startAtMs: 0, + limit: 10, + }); + + return result.lifecycle; + }; + + before(async () => { + client = await createClientWithFundedWallet(400000000); // 4 Dash + + identity = await client.platform.identities.register(200000000); + + // Additional wait time to mitigate testnet latency + await waitForSTPropagated(); + + ({ evo, sdk: evoSdk } = await getEvoSdkForNetwork(process.env.NETWORK)); + }); + + after(async () => { + if (client) { + await client.disconnect(); + } + }); + + it('should register a contract whose notes keep history and may be erased', async () => { + dataContract = await client.platform.contracts.create({ + note: { + type: 'object', + documentsKeepHistory: true, + documentsMutable: true, + canBeDeleted: true, + canBeErased: true, + properties: { + message: { + type: 'string', + maxLength: 256, + position: 0, + }, + }, + required: ['message'], + additionalProperties: false, + }, + }, identity); + + await client.platform.contracts.publish(dataContract, identity); + + // Additional wait time to mitigate testnet latency + await waitForSTPropagated(); + + client.getApps().set('notes', { + contractId: dataContract.getId(), + contract: dataContract, + }); + + const fetched = await client.platform.contracts.get(dataContract.getId()); + expect(fetched.toObject().documentSchemas.note.canBeErased).to.be.true(); + }); + + it('should keep every revision of a note that is edited', async () => { + note = await client.platform.documents.create( + 'notes.note', + identity, + { message: 'first' }, + ); + + await client.platform.documents.broadcast({ create: [note] }, identity); + + // Additional wait time to mitigate testnet latency + await waitForSTPropagated(); + + for (const message of ['second', 'third']) { + // The SDK signs `revision + 1` from the document it is handed but never + // bumps that local copy, so every replace starts from a fresh fetch. + const [stored] = await client.platform.documents.get( + 'notes.note', + { where: [['$id', '==', note.getId()]] }, + ); + stored.set('message', message); + await client.platform.documents.broadcast({ replace: [stored] }, identity); + + // Additional wait time to mitigate testnet latency + await waitForSTPropagated(); + } + + const lifecycle = await readHistory(); + expect(lifecycle.state).to.equal('ACTIVE'); + expect(lifecycle.remainingRevisions).to.equal(3n); + expect(lifecycle.deletedAtMs).to.equal(0n); + }); + + it('should hide a deleted note while keeping its revisions readable', async () => { + await client.platform.documents.broadcast({ delete: [note] }, identity); + + // Additional wait time to mitigate testnet latency + await waitForSTPropagated(); + + const [found] = await client.platform.documents.get( + 'notes.note', + { where: [['$id', '==', note.getId()]] }, + ); + expect(found).to.be.undefined(); + + const lifecycle = await readHistory(); + expect(lifecycle.state).to.equal('DELETED'); + expect( + lifecycle.remainingRevisions, + 'a delete removes no revision', + ).to.equal(3n); + expect(lifecycle.deletedAtMs > 0n).to.be.true(); + expect(lifecycle.erasingStartedAtMs).to.equal(0n); + }); + + it('should refuse to delete the same note twice', async () => { + let broadcastError; + + try { + await client.platform.documents.broadcast({ delete: [note] }, identity); + } catch (e) { + broadcastError = e; + } + + expect(broadcastError).to.be.an.instanceOf(StateTransitionBroadcastError); + // DocumentNotFoundError: a document nothing can read is not there to be + // deleted again, and a delete never escalates into removing a revision. + expect(broadcastError.code).to.equal(40101); + + // The refused delete still consumed a contract nonce in a block. Let every + // node commit it before the erase fetches that nonce, or a lagging node + // hands out the old value and its mempool drops the erase as a duplicate. + await waitForSTPropagated(); + }); + + it('should erase the retained revisions of a deleted note', async () => { + const { identityKey, signer } = await evoSignerFor(evo, evoSdk, client, identity); + + await evoSdk.documents.erase({ + document: { + id: note.getId().toString(), + ownerId: identity.getId().toString(), + dataContractId: dataContract.getId().toString(), + documentTypeName: 'note', + }, + identityKey, + signer, + }); + + // Additional wait time to mitigate testnet latency + await waitForSTPropagated(); + + const lifecycle = await readHistory(); + expect(lifecycle.state).to.equal('ABSENT'); + expect(lifecycle.remainingRevisions).to.equal(0n); + }); + + it('should refuse to erase a note that has already been erased', async () => { + const { identityKey, signer } = await evoSignerFor(evo, evoSdk, client, identity); + + let eraseError; + + try { + await evoSdk.documents.erase({ + document: { + id: note.getId().toString(), + ownerId: identity.getId().toString(), + dataContractId: dataContract.getId().toString(), + documentTypeName: 'note', + }, + identityKey, + signer, + }); + } catch (e) { + eraseError = e; + } + + expect(eraseError, 'an id that holds nothing is not erasable').to.exist(); + }); + }); +}); diff --git a/packages/rs-dpp/schema/meta_schemas/document/v4/document-meta.json b/packages/rs-dpp/schema/meta_schemas/document/v4/document-meta.json new file mode 100644 index 00000000000..89538910301 --- /dev/null +++ b/packages/rs-dpp/schema/meta_schemas/document/v4/document-meta.json @@ -0,0 +1,979 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://github.com/dashpay/platform/blob/master/packages/rs-dpp/schema/meta_schemas/document/v1/document-meta.json", + "$comment": "EDITABLE UNTIL THE RELEASE CARRYING PROTOCOL V15 SHIPS — FROZEN AFTER. This v4 document meta-schema activates with protocol v15 (CONTRACT_VERSIONS_V7). It is v3 plus the doctype-level canBeErased keyword, which lets a keep-history document type that allows deletion also allow the erasure of a deleted document's retained revisions, and admits every v15+ contract written to disk. v3 stays in place for protocol v14, where canBeErased still fails the document type's `additionalProperties: false`. Once the release carrying protocol v15 ships, mutating it would change historical validation results and break consensus replay. After release, any new top-level property or rule MUST go in a newer meta-schema version (v5+). The $id above deliberately still names the v1 path: v1, v2, v3 and v4 all share that identity, and it is the exact string `enrich_with_base_schema` injects as every PV12+ document schema's `$schema`, so bumping it here would be a wire-visible change rather than a documentation fix.", + "type": "object", + "$defs": { + "documentProperties": { + "type": "object", + "patternProperties": { + "^[a-zA-Z0-9-_]{1,64}$": { + "type": "object", + "allOf": [ + { + "$ref": "#/$defs/documentSchema" + } + ], + "unevaluatedProperties": false + } + }, + "propertyNames": { + "pattern": "^[a-zA-Z0-9-_]{1,64}$" + }, + "minProperties": 1, + "maxProperties": 100 + }, + "documentSchemaArray": { + "type": "array", + "minItems": 1, + "items": { + "type": "object", + "allOf": [ + { + "$ref": "#/$defs/documentSchema" + } + ], + "unevaluatedProperties": false + } + }, + "documentSchema": { + "type": "object", + "properties": { + "$id": { + "type": "string", + "pattern": "^#", + "minLength": 1 + }, + "$ref": { + "type": "string", + "pattern": "^#", + "minLength": 1 + }, + "$comment": { + "$ref": "https://json-schema.org/draft/2020-12/meta/core#/properties/$comment" + }, + "description": { + "$ref": "https://json-schema.org/draft/2020-12/meta/meta-data#/properties/description" + }, + "examples": { + "$ref": "https://json-schema.org/draft/2020-12/meta/meta-data#/properties/examples" + }, + "multipleOf": { + "$ref": "https://json-schema.org/draft/2020-12/meta/validation#/properties/multipleOf" + }, + "maximum": { + "$ref": "https://json-schema.org/draft/2020-12/meta/validation#/properties/maximum" + }, + "exclusiveMaximum": { + "$ref": "https://json-schema.org/draft/2020-12/meta/validation#/properties/exclusiveMaximum" + }, + "minimum": { + "$ref": "https://json-schema.org/draft/2020-12/meta/validation#/properties/minimum" + }, + "exclusiveMinimum": { + "$ref": "https://json-schema.org/draft/2020-12/meta/validation#/properties/exclusiveMinimum" + }, + "maxLength": { + "$ref": "https://json-schema.org/draft/2020-12/meta/validation#/properties/maxLength" + }, + "minLength": { + "$ref": "https://json-schema.org/draft/2020-12/meta/validation#/properties/minLength" + }, + "pattern": { + "$ref": "https://json-schema.org/draft/2020-12/meta/validation#/properties/pattern" + }, + "maxItems": { + "$ref": "https://json-schema.org/draft/2020-12/meta/validation#/properties/maxItems" + }, + "minItems": { + "$ref": "https://json-schema.org/draft/2020-12/meta/validation#/properties/minItems" + }, + "uniqueItems": { + "$ref": "https://json-schema.org/draft/2020-12/meta/validation#/properties/uniqueItems" + }, + "refersTo": { + "type": "object", + "properties": { + "type": { + "enum": [ + "identity", + "contract", + "token", + "permanentDocument", + "identityPublicKey" + ] + }, + "contractId": { + "description": "The id of the data contract the referenced document lives in, as a base58 string or a 32-byte array; when absent the reference targets the declaring contract itself", + "oneOf": [ + { + "type": "string", + "minLength": 32, + "maxLength": 44, + "pattern": "^[123456789A-HJ-NP-Za-km-z]{32,44}$" + }, + { + "type": "array", + "minItems": 32, + "maxItems": 32, + "items": { + "type": "integer", + "minimum": 0, + "maximum": 255 + } + } + ] + }, + "documentType": { + "description": "The name of the referenced document type; it must forbid deletion (canBeDeleted: false)", + "type": "string", + "minLength": 1, + "maxLength": 64, + "pattern": "^[a-zA-Z0-9-_]{1,64}$" + }, + "keyIdProperty": { + "description": "The property of the same document type whose value carries the referenced key id; the reference property's value carries the identity id", + "type": "string", + "minLength": 1, + "maxLength": 256, + "pattern": "^[a-zA-Z0-9-_]{1,64}(\\.[a-zA-Z0-9-_]{1,64})*$" + }, + "propertyAgreement": { + "description": "permanentDocument references only: each { referring property: referenced property } pair must hold as an equality between the referring document's value and the referenced document's value, enforced by consensus at document write time; both properties must exist and share one property type, validated at contract registration", + "type": "object", + "minProperties": 1, + "maxProperties": 10, + "propertyNames": { + "pattern": "^[a-zA-Z0-9-_]{1,64}(\\.[a-zA-Z0-9-_]{1,64})*$" + }, + "additionalProperties": { + "type": "string", + "minLength": 1, + "maxLength": 256, + "pattern": "^[a-zA-Z0-9-_]{1,64}(\\.[a-zA-Z0-9-_]{1,64})*$" + } + } + }, + "required": [ + "type" + ], + "additionalProperties": false, + "allOf": [ + { + "if": { + "properties": { "type": { "const": "permanentDocument" } }, + "required": ["type"] + }, + "then": { + "required": ["type", "documentType"] + }, + "else": { + "properties": { + "contractId": false, + "documentType": false + } + } + }, + { + "if": { + "properties": { "type": { "const": "identityPublicKey" } }, + "required": ["type"] + }, + "then": { + "required": ["type", "keyIdProperty"] + }, + "else": { + "properties": { + "keyIdProperty": false + } + } + } + ] + }, + "contains": { + "$ref": "https://json-schema.org/draft/2020-12/meta/applicator#/properties/contains" + }, + "maxProperties": { + "$ref": "https://json-schema.org/draft/2020-12/meta/validation#/properties/maxProperties" + }, + "minProperties": { + "$ref": "https://json-schema.org/draft/2020-12/meta/validation#/properties/minProperties" + }, + "required": { + "$ref": "https://json-schema.org/draft/2020-12/meta/validation#/properties/required" + }, + "additionalProperties": { + "type": "boolean", + "const": false + }, + "properties": { + "$ref": "#/$defs/documentProperties" + }, + "dependentRequired": { + "type": "object", + "minProperties": 1, + "additionalProperties": { + "$ref": "https://json-schema.org/draft/2020-12/meta/validation#/$defs/stringArray" + } + }, + "const": true, + "enum": { + "type": "array", + "items": true, + "minItems": 1, + "uniqueItems": true + }, + "type": { + "$ref": "https://json-schema.org/draft/2020-12/meta/validation#/properties/type" + }, + "format": { + "$ref": "https://json-schema.org/draft/2020-12/meta/format-annotation#/properties/format" + }, + "contentMediaType": { + "$ref": "https://json-schema.org/draft/2020-12/meta/content#/properties/contentMediaType" + }, + "byteArray": { + "type": "boolean", + "const": true + }, + "position": { + "type": "integer", + "minimum": 0 + }, + "requiredSince": { + "type": "integer", + "minimum": 1, + "maximum": 4294967295 + } + }, + "dependentSchemas": { + "byteArray": { + "description": "should be used only with array type", + "properties": { + "type": { + "type": "string", + "const": "array" + } + } + }, + "contentMediaType": { + "if": { + "properties": { + "contentMediaType": { + "const": "application/x.dash.dpp.identifier" + } + } + }, + "then": { + "properties": { + "byteArray": { + "const": true + }, + "minItems": { + "const": 32 + }, + "maxItems": { + "const": 32 + } + }, + "required": [ + "byteArray", + "minItems", + "maxItems" + ] + } + }, + "pattern": { + "description": "prevent slow pattern matching of large strings", + "properties": { + "maxLength": { + "type": "integer", + "minimum": 0, + "maximum": 50000 + } + }, + "required": [ + "maxLength" + ] + }, + "refersTo": { + "description": "refersTo is only allowed on identifier properties", + "properties": { + "type": { + "const": "array" + }, + "byteArray": { + "const": true + }, + "contentMediaType": { + "const": "application/x.dash.dpp.identifier" + }, + "minItems": { + "const": 32 + }, + "maxItems": { + "const": 32 + } + }, + "required": [ + "type", + "byteArray", + "contentMediaType", + "minItems", + "maxItems" + ] + }, + "format": { + "description": "prevent slow format validation of large strings", + "properties": { + "maxLength": { + "type": "integer", + "minimum": 0, + "maximum": 50000 + } + }, + "required": [ + "maxLength" + ] + } + }, + "allOf": [ + { + "$comment": "require index for object properties", + "if": { + "properties": { + "type": { + "const": "object" + } + }, + "required": [ + "type" + ] + }, + "then": { + "properties": { + "properties": { + "type": "object", + "additionalProperties": { + "type": "object", + "properties": { + "position": true + }, + "required": [ + "position" + ] + } + } + } + } + }, + { + "$comment": "allow only byte arrays", + "if": { + "properties": { + "type": { + "const": "array" + } + }, + "required": [ + "type" + ] + }, + "then": { + "properties": { + "byteArray": true + }, + "required": [ + "byteArray" + ] + } + }, + { + "$comment": "all object properties must be defined", + "if": { + "properties": { + "type": { + "const": "object" + } + }, + "not": { + "properties": { + "$ref": true + }, + "required": [ + "$ref" + ] + } + }, + "then": { + "properties": { + "properties": { + "$ref": "#/$defs/documentProperties" + }, + "additionalProperties": { + "$ref": "#/$defs/documentSchema/properties/additionalProperties" + } + }, + "required": [ + "properties", + "additionalProperties" + ] + } + } + ] + }, + "documentActionTokenCost": { + "type": "object", + "properties": { + "contractId": { + "type": "array", + "contentMediaType": "application/x.dash.dpp.identifier", + "byteArray": true, + "minItems": 32, + "maxItems": 32 + }, + "tokenPosition": { + "type": "integer", + "minimum": 0, + "maximum": 65535 + }, + "amount": { + "type": "integer", + "minimum": 1, + "maximum": 281474976710655 + }, + "effect": { + "type": "integer", + "enum": [ + 0, + 1 + ], + "description": "0 - TransferTokenToContractOwner (default), 1 - Burn" + }, + "gasFeesPaidBy": { + "type": "integer", + "enum": [ + 0, + 1, + 2 + ], + "description": "0 - DocumentOwner (default), 1 - ContractOwner, 2 - PreferContractOwner" + } + }, + "required": [ + "tokenPosition", + "amount" + ], + "additionalProperties": false + } + }, + "properties": { + "type": { + "type": "string", + "const": "object" + }, + "$schema": { + "type": "string", + "const": "https://github.com/dashpay/platform/blob/master/packages/rs-dpp/schema/meta_schemas/document/v1/document-meta.json" + }, + "$defs": { + "$ref": "#/$defs/documentProperties" + }, + "indices": { + "type": "array", + "items": { + "type": "object", + "properties": { + "name": { + "type": "string", + "minLength": 1, + "maxLength": 32 + }, + "properties": { + "type": "array", + "items": { + "type": "object", + "propertyNames": { + "maxLength": 256 + }, + "additionalProperties": { + "type": "string", + "enum": [ + "asc" + ] + }, + "minProperties": 1, + "maxProperties": 1 + }, + "minItems": 1, + "maxItems": 10 + }, + "unique": { + "type": "boolean" + }, + "nullSearchable": { + "type": "boolean" + }, + "contested": { + "type": "object", + "properties": { + "fieldMatches": { + "type": "array", + "items": { + "type": "object", + "properties": { + "field": { + "type": "string", + "minLength": 1, + "maxLength": 256 + }, + "regexPattern": { + "type": "string", + "minLength": 1, + "maxLength": 256 + } + }, + "additionalProperties": false, + "required": [ + "field", + "regexPattern" + ] + }, + "minItems": 1 + }, + "resolution": { + "type": "integer", + "enum": [ + 0 + ], + "description": "Resolution. 0 - Masternode Vote" + }, + "description": { + "type": "string", + "minLength": 1, + "maxLength": 256 + } + }, + "required": [ + "resolution" + ], + "additionalProperties": false + }, + "countable": { + "oneOf": [ + { + "type": "boolean", + "description": "Legacy form. true == \"countable\", false == \"notCountable\". Kept for back-compat with contracts written before the enum form was introduced." + }, + { + "type": "string", + "enum": ["notCountable", "countable", "countableAllowingOffset"], + "description": "\"countable\" — index uses a CountTree (O(1) totals). \"countableAllowingOffset\" — index uses a ProvableCountTree (totals + future O(log n) range / offset queries). \"notCountable\" — plain NormalTree (no count fast path)." + } + ], + "description": "Whether and how the index supports count fast paths. Adds extra storage cost for non-default values." + }, + "rangeCountable": { + "type": "boolean", + "description": "When true, the property-name level becomes a ProvableCountTree and value trees become CountTrees so range-count queries on the indexed property are O(log n). Requires `countable` to be \"countable\" or \"countableAllowingOffset\"." + }, + "summable": { + "type": "string", + "minLength": 1, + "maxLength": 64, + "description": "Name of an integer document property whose values are aggregated into a sum at the index. When set, the index's value trees become SumTrees and each per-document index reference is a ReferenceWithSumItem contributing the named property's value to ancestor sum-bearing trees. The property must exist on the document type, be in `required`, and have an integer type." + }, + "rangeSummable": { + "type": "boolean", + "description": "When true, the property-name level becomes a ProvableSumTree (or ProvableCountProvableSumTree when paired with `rangeCountable: true`) so range-sum queries on the indexed property are O(log n) via the `AggregateSumOnRange` proof primitive. Requires `summable` to be set." + }, + "averageable": { + "type": "string", + "minLength": 1, + "maxLength": 64, + "description": "Syntactic sugar: `averageable: \"\"` is shorthand for `countable: \"countable\"` + `summable: \"\"`. Enables average queries (which return `(count, sum)` pairs the client divides) without forcing authors to think in terms of count + sum. Same on-disk layout as setting both underlying flags. If you set both `averageable` and `summable`, they must name the same property." + }, + "rangeAverageable": { + "type": "boolean", + "description": "Syntactic sugar: `rangeAverageable: true` is shorthand for `rangeCountable: true` + `rangeSummable: true`. Requires `averageable` to be set." + }, + "rankedCountable": { + "oneOf": [ + { + "type": "boolean", + "description": "When true, the index's terminal property-name tree also carries an ordered secondary tree keyed by each group's document count, so \"top / bottom K groups by count\" queries are O(log n + k) with proofs." + }, + { + "type": "object", + "properties": { + "at": { + "oneOf": [ + { + "type": "string", + "minLength": 1, + "maxLength": 256 + }, + { + "type": "array", + "items": { + "type": "string", + "minLength": 1, + "maxLength": 256 + }, + "minItems": 1, + "maxItems": 10, + "uniqueItems": true + } + ], + "description": "Name of the index property (or array of properties) whose levels carry Count rankings. Each must be one of the index's properties (an index has at most 10, hence maxItems); naming the last property is equivalent to the boolean form. A non-terminal property places a ranking at that prefix level: its values are ranked by whole-subtree document count (e.g. on [hashtag, postId], at: \"hashtag\" ranks hashtags by total count across all their posts), and every level from the shallowest ranked one down to the terminal is laid out count-bearing so each write's delta propagates up through the chain. ANY subset of levels may be named — e.g. [\"hashtag\", \"postId\"] declares both rankings on one index, and a fully ranked index ranks every level." + } + }, + "required": ["at"], + "additionalProperties": false, + "description": "Level-addressed form: places Count rankings at the named properties' levels. Cannot be combined with rankedSummable or rankedAverageable when a non-terminal level is named, and no other index of the document type may share a non-terminal ranked level or any level below it." + } + ], + "description": "Count ranking axis. Requires `rangeCountable: true`. Independent of the Sum and Avg axes (`rankedSummable` / `rankedAverageable`), which stay terminal-level booleans." + }, + "rankedSummable": { + "type": "boolean", + "description": "When true, the index's terminal property-name tree also carries an ordered secondary tree keyed by each group's sum of the `summable` property, so \"top / bottom K groups by sum\" queries are O(log n + k) with proofs. Requires `rangeSummable: true`. Adds the Sum ranking axis only." + }, + "rankedAverageable": { + "type": "boolean", + "description": "When true, the index's terminal property-name tree also carries an ordered secondary tree keyed by each group's average (count + sum pair) of the `averageable` property, so \"top / bottom K groups by average\" queries are O(log n + k) with proofs. Requires `rangeAverageable: true` (which itself implies `rangeCountable` + `rangeSummable`). Adds the Avg ranking axis only — it does NOT imply `rankedCountable` or `rankedSummable`; each ranking axis costs its own secondary tree and is opted into separately." + }, + "timeRange": { + "type": "object", + "properties": { + "on": { + "type": "string", + "minLength": 1, + "maxLength": 256, + "description": "Name of the timestamp index property to bucket. Must be this index's first property and name one of the system timestamps ($createdAt, $updatedAt or $transferredAt). A timeRange index may be unique only when range equals step (non-overlapping windows) and `on` is $createdAt." + }, + "range": { + "type": "integer", + "minimum": 1, + "description": "Length of each time range window, in seconds. Must be an exact multiple of `step`." + }, + "step": { + "type": "integer", + "minimum": 1, + "description": "Interval between successive range starts, in seconds. When `range` > `step` the ranges overlap and a document is indexed under `range / step` bucket-start values, bounded by a protocol-versioned cap (24 at protocol version 14)." + }, + "phase": { + "type": "integer", + "minimum": 0, + "description": "Grid alignment phase, in seconds. Range starts are `phase + k * step`; must be strictly less than `step` (a larger value would be a redundant spelling of `phase % step`) and strictly less than one year (31536000 — a phase further out could sit past current block time on a huge step, leaving valid timestamps before the grid's first bucket). A pure alignment offset — it moves where window boundaries fall (e.g. daily windows cut at 06:00 UTC instead of midnight) and never excludes any real timestamp. Defaults to 0." + }, + "ttl": { + "type": "integer", + "minimum": 1, + "description": "Time to live, in seconds: entries under this index exist for at most this long past their bucket's start, plus a bounded drainage lag — every write into the index continues draining the oldest expired bucket under a per-write operation budget, and expired windows are not queryable (a byStart selection past the horizon is rejected), so every queryable window is complete. Must be at least `range` (a window still able to receive consensus-timestamped writes can never expire) and at most a protocol-versioned cap (604800 — one week — at protocol version 14). Indexes bucketing one field with the same grid share its storage level and must declare the same ttl. Bytes written under a TTL'd index bill to processing at an ephemeral-bytes rate instead of to storage, carry no storage flags, and refund nothing on removal. Omitted means entries live forever. Available from protocol version 14." + } + }, + "required": ["on", "range", "step"], + "additionalProperties": false, + "description": "Buckets the first index property's timestamp into fixed-length, regularly-spaced (possibly overlapping) time ranges. The window parameters (`range`, `step`, `phase`) are declared in seconds, since a bucket is selected from block time and the target block interval is five seconds; the stored key is the range start as a u64 millisecond timestamp, so it stays directly comparable to the source timestamp it buckets. Enables trending/leaderboard queries within the newest/oldest active range. A system-timestamp source must be listed in the document type's required fields. Several indexes may bucket the same timestamp with different grids — each grid gets its own index subtree, keyed by the property name qualified with the grid parameters. Available from protocol version 14." + }, + "terminal": { + "type": "string", + "minLength": 1, + "maxLength": 256, + "description": "Only on indexOnly document types: names the property whose value is this index entry's member key — the docId-analog terminal key under the index's storage marker, stored as an Item instead of a Reference because there is no primary-storage row. Either \"$ownerId\" (the default when omitted) or an identifier property carrying a refersTo declaration (identity, contract, token, or permanentDocument). Must not repeat one of the index's listed properties. Available from protocol version 14." + }, + "preallocated": { + "type": "boolean", + "description": "Only on indexOnly document types whose index path is fully determined by a same-contract permanentDocument refersTo declaration: every index property must be either the referring property itself (its value is the referenced document's $id) or a key of that declaration's propertyAgreement (consensus-equal to a referenced-document property). When true, creating a referenced document also creates this index's dynamic trees for entries referencing it — paid by the referenced document's creator — and deleting the last entry keeps them, so every entry insert costs the same as the first. Available from protocol version 14." + }, + "skipIfAbsent": { + "type": "boolean", + "description": "Only on indexOnly document types: when true, a document that omits this index's first property writes no entry into this index (and a delete recomputes the same skip), so the index holds only documents carrying the property. The first property is the skip trigger: it must be a top-level schema property NOT listed in `required` (making it the only way an indexOnly property may be optional), and every index involving an optional property must be skipIfAbsent with that property first. Every other property that is not a skip trigger must still appear in at least one non-skipIfAbsent index, and at least one $createdAt-free index must remain non-skipIfAbsent (the executed-transition proof index). An absent trigger is distinct from an empty value: absence skips the index, while any present value — empty included — indexes normally. Available from protocol version 14." + } + }, + "required": [ + "properties", + "name" + ], + "dependentRequired": { + "rangeCountable": ["countable"], + "rangeSummable": ["summable"], + "rangeAverageable": ["averageable"] + }, + "$comment": "The ranked prerequisites are value-sensitive, unlike the range* rows above: `dependentRequired` fires on key *presence*, so listing them there would make an explicit `\"rankedCountable\": false` — a written-out opt-out, which the structural parser accepts as such — demand a `rangeCountable` the index does not need. The range* rows keep presence semantics because that is what they shipped with in v2 and changing them would move historical validation results.", + "allOf": [ + { + "if": { + "properties": { + "rankedCountable": { + "anyOf": [{ "const": true }, { "type": "object" }] + } + }, + "required": ["rankedCountable"] + }, + "then": { "required": ["rangeCountable"] } + }, + { + "if": { + "properties": { "rankedSummable": { "const": true } }, + "required": ["rankedSummable"] + }, + "then": { "required": ["rangeSummable"] } + }, + { + "if": { + "properties": { "rankedAverageable": { "const": true } }, + "required": ["rankedAverageable"] + }, + "then": { "required": ["rangeAverageable"] } + } + ], + "additionalProperties": false + }, + "minItems": 1, + "maxItems": 10 + }, + "signatureSecurityLevelRequirement": { + "type": "integer", + "enum": [ + 1, + 2, + 3 + ], + "description": "Public key security level. 1 - Critical, 2 - High, 3 - Medium. If none specified, High level is used" + }, + "documentsKeepHistory": { + "type": "boolean", + "description": "True if the documents keep all their history, default is false" + }, + "keepsTransferHistory": { + "type": "boolean", + "description": "True if transfers of these documents are recorded in the document history system contract, default is false" + }, + "keepsPurchaseHistory": { + "type": "boolean", + "description": "True if purchases of these documents are recorded in the document history system contract, default is false" + }, + "keepsPricingHistory": { + "type": "boolean", + "description": "True if price updates on these documents are recorded in the document history system contract, default is false" + }, + "documentsMutable": { + "type": "boolean", + "description": "True if the documents are mutable, default is true" + }, + "canBeDeleted": { + "type": "boolean", + "description": "True if the documents can be deleted, default is true" + }, + "canBeErased": { + "type": "boolean", + "description": "True if a deleted document may have its retained revisions purged by an erase transition; requires documentsKeepHistory and canBeDeleted, default is false, immutable on contract update" + }, + "transferable": { + "type": "integer", + "enum": [ + 0, + 1 + ], + "description": "Transferable without a marketplace sell. 0 - Never, 1 - Always" + }, + "tradeMode": { + "type": "integer", + "enum": [ + 0, + 1 + ], + "description": "Built in marketplace system. 0 - None, 1 - Direct purchase (The user can buy the item without the need for an approval)" + }, + "creationRestrictionMode": { + "type": "integer", + "enum": [ + 0, + 1, + 2 + ], + "description": "Restrictions of document creation. 0 - No restrictions, 1 - Owner only, 2 - No creation (System Only)" + }, + "requiresIdentityEncryptionBoundedKey": { + "type": "integer", + "enum": [ + 0, + 1, + 2 + ], + "description": "Key requirements. 0 - Unique Non Replaceable, 1 - Multiple, 2 - Multiple with reference to latest key." + }, + "requiresIdentityDecryptionBoundedKey": { + "type": "integer", + "enum": [ + 0, + 1, + 2 + ], + "description": "Key requirements. 0 - Unique Non Replaceable, 1 - Multiple, 2 - Multiple with reference to latest key." + }, + "documentsCountable": { + "type": "boolean", + "description": "When true, the primary key tree uses a CountTree enabling O(1) total document count queries." + }, + "rangeCountable": { + "type": "boolean", + "description": "When true, the primary key tree uses a ProvableCountTree enabling range countable. Implies documentsCountable." + }, + "documentsSummable": { + "type": "string", + "minLength": 1, + "maxLength": 64, + "description": "Name of an integer document property aggregated into the primary-key SumTree (one sum per document type). Stores documents as `ItemWithSumItem` so the primary-key tree's root sum is the total of the named property across all docs of this type. Property must exist on the document type, be in `required`, and have an integer type. Composes with `documentsKeepHistory: true` — keep-history doctypes get a `SumTree` per-document subtree with a `ReferenceWithSumItem` on the `0`-key carrying the current version's value, so the doctype-level root aggregate reflects current versions only (historical versions don't double-count)." + }, + "rangeSummable": { + "type": "boolean", + "description": "When true, the primary key tree uses a ProvableSumTree (or ProvableCountProvableSumTree paired with rangeCountable: true) enabling O(log n) range-sum queries over the primary axis. Requires `documentsSummable` to be set. Rarely useful — range-sum on the primary key with no where-clause filter is unusual; most callers want per-index `rangeSummable` instead. Set this only when you need a provable global range-sum tree." + }, + "documentsAverageable": { + "type": "string", + "minLength": 1, + "maxLength": 64, + "description": "Syntactic sugar: `documentsAverageable: \"\"` is shorthand for `documentsCountable: true` + `documentsSummable: \"\"`. Enables doctype-wide average queries (returns `(count, sum)` the client divides) without authors having to compose the count + sum flags. Same on-disk layout. If you set both `documentsAverageable` and `documentsSummable`, they must name the same property. Composes with `documentsKeepHistory: true` via the per-doc SumTree + ReferenceWithSumItem layout described under `documentsSummable`." + }, + "rangeAverageable": { + "type": "boolean", + "description": "Syntactic sugar: `rangeAverageable: true` is shorthand for `rangeCountable: true` + `rangeSummable: true`. Requires `documentsAverageable` to be set. Same caveat as `rangeSummable` — rarely useful on the primary key; per-index `rangeAverageable` is what most callers want." + }, + "indexOnly": { + "type": "boolean", + "description": "When true, documents of this type are never written to primary storage: the index entries are the rows, each terminating in an Item keyed by the index's `terminal` property instead of a Reference keyed by the document id. Only what is in the indexes exists and is recoverable. Requires: every property required and appearing in at least one index (except a `skipIfAbsent` index's optional first property), $ownerId in at least one index (as a property or terminal), documentsMutable: false, no transfers/trading/history/transient properties, and no doctype-level aggregate keywords (use the index-level count flags). Available from protocol version 14." + }, + "tokenCost": { + "type": "object", + "properties": { + "create": { + "$ref": "#/$defs/documentActionTokenCost" + }, + "replace": { + "$ref": "#/$defs/documentActionTokenCost" + }, + "delete": { + "$ref": "#/$defs/documentActionTokenCost" + }, + "transfer": { + "$ref": "#/$defs/documentActionTokenCost" + }, + "update_price": { + "$ref": "#/$defs/documentActionTokenCost" + }, + "purchase": { + "$ref": "#/$defs/documentActionTokenCost" + } + }, + "additionalProperties": false + }, + "properties": { + "type": "object", + "additionalProperties": { + "type": "object", + "allOf": [ + { + "$ref": "#/$defs/documentSchema" + } + ], + "unevaluatedProperties": false + }, + "properties": { + "$id": true, + "$ownerId": true, + "$revision": true, + "$createdAt": true, + "$updatedAt": true, + "$transferredAt": true, + "$createdAtBlockHeight": true, + "$updatedAtBlockHeight": true, + "$transferredAtBlockHeight": true, + "$createdAtCoreBlockHeight": true, + "$updatedAtCoreBlockHeight": true, + "$transferredAtCoreBlockHeight": true + }, + "propertyNames": { + "oneOf": [ + { + "type": "string", + "pattern": "^[a-zA-Z0-9-_]{1,64}$" + }, + { + "type": "string", + "enum": [ + "$id", + "$ownerId", + "$revision", + "$createdAt", + "$updatedAt", + "$transferredAt", + "$createdAtBlockHeight", + "$updatedAtBlockHeight", + "$transferredAtBlockHeight", + "$createdAtCoreBlockHeight", + "$updatedAtCoreBlockHeight", + "$transferredAtCoreBlockHeight" + ] + } + ] + }, + "minProperties": 1, + "maxProperties": 100 + }, + "transient": { + "type": "array", + "items": { + "type": "string" + } + }, + "additionalProperties": { + "type": "boolean", + "const": false + }, + "required": { + "type": "array", + "items": { + "type": "string" + }, + "uniqueItems": true + }, + "$comment": { + "type": "string" + }, + "description": { + "type": "string" + }, + "minProperties": { + "type": "integer", + "minimum": 0 + }, + "maxProperties": { + "type": "integer", + "minimum": 0 + }, + "dependentRequired": { + "type": "object", + "additionalProperties": { + "type": "array", + "items": { + "type": "string" + }, + "uniqueItems": true + } + } + }, + "required": [ + "$schema", + "type", + "properties", + "additionalProperties" + ], + "dependentRequired": { + "rangeSummable": ["documentsSummable"], + "rangeAverageable": ["documentsAverageable"] + }, + "additionalProperties": false +} diff --git a/packages/rs-dpp/src/data_contract/document_type/accessors/mod.rs b/packages/rs-dpp/src/data_contract/document_type/accessors/mod.rs index 6ea4e0cb2a8..5816ce36801 100644 --- a/packages/rs-dpp/src/data_contract/document_type/accessors/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/accessors/mod.rs @@ -1,6 +1,7 @@ mod v0; mod v1; mod v2; +mod v3; use crate::data_contract::document_type::index::Index; use crate::data_contract::document_type::index_level::IndexLevel; @@ -23,6 +24,7 @@ use std::collections::{BTreeMap, BTreeSet}; pub use v0::*; pub use v1::*; pub use v2::*; +pub use v3::*; impl DocumentTypeV0MutGetters for DocumentType { fn schema_mut(&mut self) -> &mut Value { @@ -30,6 +32,7 @@ impl DocumentTypeV0MutGetters for DocumentType { DocumentType::V0(v0) => v0.schema_mut(), DocumentType::V1(v1) => v1.schema_mut(), DocumentType::V2(v2) => v2.schema_mut(), + DocumentType::V3(v3) => v3.schema_mut(), } } } @@ -40,6 +43,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.name(), DocumentType::V1(v1) => v1.name(), DocumentType::V2(v2) => v2.name(), + DocumentType::V3(v3) => v3.name(), } } @@ -48,6 +52,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.schema(), DocumentType::V1(v1) => v1.schema(), DocumentType::V2(v2) => v2.schema(), + DocumentType::V3(v3) => v3.schema(), } } @@ -56,6 +61,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.schema_owned(), DocumentType::V1(v1) => v1.schema_owned(), DocumentType::V2(v2) => v2.schema_owned(), + DocumentType::V3(v3) => v3.schema_owned(), } } @@ -64,6 +70,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.indexes(), DocumentType::V1(v1) => v1.indexes(), DocumentType::V2(v2) => v2.indexes(), + DocumentType::V3(v3) => v3.indexes(), } } @@ -72,6 +79,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.find_contested_index(), DocumentType::V1(v1) => v1.find_contested_index(), DocumentType::V2(v2) => v2.find_contested_index(), + DocumentType::V3(v3) => v3.find_contested_index(), } } @@ -80,6 +88,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.index_structure(), DocumentType::V1(v1) => v1.index_structure(), DocumentType::V2(v2) => v2.index_structure(), + DocumentType::V3(v3) => v3.index_structure(), } } @@ -88,6 +97,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.flattened_properties(), DocumentType::V1(v1) => v1.flattened_properties(), DocumentType::V2(v2) => v2.flattened_properties(), + DocumentType::V3(v3) => v3.flattened_properties(), } } @@ -96,6 +106,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.properties(), DocumentType::V1(v1) => v1.properties(), DocumentType::V2(v2) => v2.properties(), + DocumentType::V3(v3) => v3.properties(), } } @@ -104,6 +115,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.identifier_paths(), DocumentType::V1(v1) => v1.identifier_paths(), DocumentType::V2(v2) => v2.identifier_paths(), + DocumentType::V3(v3) => v3.identifier_paths(), } } @@ -112,6 +124,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.binary_paths(), DocumentType::V1(v1) => v1.binary_paths(), DocumentType::V2(v2) => v2.binary_paths(), + DocumentType::V3(v3) => v3.binary_paths(), } } @@ -120,6 +133,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.required_fields(), DocumentType::V1(v1) => v1.required_fields(), DocumentType::V2(v2) => v2.required_fields(), + DocumentType::V3(v3) => v3.required_fields(), } } @@ -128,6 +142,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.transient_fields(), DocumentType::V1(v1) => v1.transient_fields(), DocumentType::V2(v2) => v2.transient_fields(), + DocumentType::V3(v3) => v3.transient_fields(), } } @@ -136,6 +151,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.documents_keep_history(), DocumentType::V1(v1) => v1.documents_keep_history(), DocumentType::V2(v2) => v2.documents_keep_history(), + DocumentType::V3(v3) => v3.documents_keep_history(), } } @@ -144,6 +160,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.documents_keep_transfer_history(), DocumentType::V1(v1) => v1.documents_keep_transfer_history(), DocumentType::V2(v2) => v2.documents_keep_transfer_history(), + DocumentType::V3(v3) => v3.documents_keep_transfer_history(), } } @@ -152,6 +169,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.documents_keep_purchase_history(), DocumentType::V1(v1) => v1.documents_keep_purchase_history(), DocumentType::V2(v2) => v2.documents_keep_purchase_history(), + DocumentType::V3(v3) => v3.documents_keep_purchase_history(), } } @@ -160,6 +178,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.documents_keep_pricing_history(), DocumentType::V1(v1) => v1.documents_keep_pricing_history(), DocumentType::V2(v2) => v2.documents_keep_pricing_history(), + DocumentType::V3(v3) => v3.documents_keep_pricing_history(), } } @@ -168,6 +187,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.documents_mutable(), DocumentType::V1(v1) => v1.documents_mutable(), DocumentType::V2(v2) => v2.documents_mutable(), + DocumentType::V3(v3) => v3.documents_mutable(), } } @@ -176,6 +196,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.documents_can_be_deleted(), DocumentType::V1(v1) => v1.documents_can_be_deleted(), DocumentType::V2(v2) => v2.documents_can_be_deleted(), + DocumentType::V3(v3) => v3.documents_can_be_deleted(), } } @@ -184,6 +205,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.documents_transferable(), DocumentType::V1(v1) => v1.documents_transferable(), DocumentType::V2(v2) => v2.documents_transferable(), + DocumentType::V3(v3) => v3.documents_transferable(), } } @@ -192,6 +214,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.trade_mode(), DocumentType::V1(v1) => v1.trade_mode(), DocumentType::V2(v2) => v2.trade_mode(), + DocumentType::V3(v3) => v3.trade_mode(), } } @@ -200,6 +223,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.creation_restriction_mode(), DocumentType::V1(v1) => v1.creation_restriction_mode(), DocumentType::V2(v2) => v2.creation_restriction_mode(), + DocumentType::V3(v3) => v3.creation_restriction_mode(), } } @@ -208,6 +232,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.data_contract_id(), DocumentType::V1(v1) => v1.data_contract_id(), DocumentType::V2(v2) => v2.data_contract_id(), + DocumentType::V3(v3) => v3.data_contract_id(), } } @@ -216,6 +241,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.requires_identity_encryption_bounded_key(), DocumentType::V1(v1) => v1.requires_identity_encryption_bounded_key(), DocumentType::V2(v2) => v2.requires_identity_encryption_bounded_key(), + DocumentType::V3(v3) => v3.requires_identity_encryption_bounded_key(), } } @@ -224,6 +250,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.requires_identity_decryption_bounded_key(), DocumentType::V1(v1) => v1.requires_identity_decryption_bounded_key(), DocumentType::V2(v2) => v2.requires_identity_decryption_bounded_key(), + DocumentType::V3(v3) => v3.requires_identity_decryption_bounded_key(), } } @@ -232,6 +259,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.security_level_requirement(), DocumentType::V1(v1) => v1.security_level_requirement(), DocumentType::V2(v2) => v2.security_level_requirement(), + DocumentType::V3(v3) => v3.security_level_requirement(), } } @@ -241,6 +269,7 @@ impl DocumentTypeV0Getters for DocumentType { DocumentType::V0(v0) => v0.json_schema_validator_ref(), DocumentType::V1(v1) => v1.json_schema_validator_ref(), DocumentType::V2(v2) => v2.json_schema_validator_ref(), + DocumentType::V3(v3) => v3.json_schema_validator_ref(), } } } @@ -251,6 +280,7 @@ impl DocumentTypeV0Setters for DocumentType { DocumentType::V0(v0) => v0.set_data_contract_id(data_contract_id), DocumentType::V1(v1) => v1.set_data_contract_id(data_contract_id), DocumentType::V2(v2) => v2.set_data_contract_id(data_contract_id), + DocumentType::V3(v3) => v3.set_data_contract_id(data_contract_id), } } } @@ -261,6 +291,7 @@ impl DocumentTypeV1Setters for DocumentType { DocumentType::V0(_) => { /* no-op */ } DocumentType::V1(v1) => v1.set_document_creation_token_cost(cost), DocumentType::V2(v2) => v2.set_document_creation_token_cost(cost), + DocumentType::V3(v3) => v3.set_document_creation_token_cost(cost), } } @@ -269,6 +300,7 @@ impl DocumentTypeV1Setters for DocumentType { DocumentType::V0(_) => { /* no-op */ } DocumentType::V1(v1) => v1.set_document_replacement_token_cost(cost), DocumentType::V2(v2) => v2.set_document_replacement_token_cost(cost), + DocumentType::V3(v3) => v3.set_document_replacement_token_cost(cost), } } @@ -277,6 +309,7 @@ impl DocumentTypeV1Setters for DocumentType { DocumentType::V0(_) => { /* no-op */ } DocumentType::V1(v1) => v1.set_document_deletion_token_cost(cost), DocumentType::V2(v2) => v2.set_document_deletion_token_cost(cost), + DocumentType::V3(v3) => v3.set_document_deletion_token_cost(cost), } } @@ -285,6 +318,7 @@ impl DocumentTypeV1Setters for DocumentType { DocumentType::V0(_) => { /* no-op */ } DocumentType::V1(v1) => v1.set_document_transfer_token_cost(cost), DocumentType::V2(v2) => v2.set_document_transfer_token_cost(cost), + DocumentType::V3(v3) => v3.set_document_transfer_token_cost(cost), } } @@ -293,6 +327,7 @@ impl DocumentTypeV1Setters for DocumentType { DocumentType::V0(_) => { /* no-op */ } DocumentType::V1(v1) => v1.set_document_price_update_token_cost(cost), DocumentType::V2(v2) => v2.set_document_price_update_token_cost(cost), + DocumentType::V3(v3) => v3.set_document_price_update_token_cost(cost), } } @@ -301,6 +336,7 @@ impl DocumentTypeV1Setters for DocumentType { DocumentType::V0(_) => { /* no-op */ } DocumentType::V1(v1) => v1.set_document_purchase_token_cost(cost), DocumentType::V2(v2) => v2.set_document_purchase_token_cost(cost), + DocumentType::V3(v3) => v3.set_document_purchase_token_cost(cost), } } } @@ -311,6 +347,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.name(), DocumentTypeRef::V1(v1) => v1.name(), DocumentTypeRef::V2(v2) => v2.name(), + DocumentTypeRef::V3(v3) => v3.name(), } } @@ -319,6 +356,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.schema(), DocumentTypeRef::V1(v1) => v1.schema(), DocumentTypeRef::V2(v2) => v2.schema(), + DocumentTypeRef::V3(v3) => v3.schema(), } } @@ -327,6 +365,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.clone().schema_owned(), DocumentTypeRef::V1(v1) => v1.clone().schema_owned(), DocumentTypeRef::V2(v2) => v2.clone().schema_owned(), + DocumentTypeRef::V3(v3) => v3.clone().schema_owned(), } } @@ -335,6 +374,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.indexes(), DocumentTypeRef::V1(v1) => v1.indexes(), DocumentTypeRef::V2(v2) => v2.indexes(), + DocumentTypeRef::V3(v3) => v3.indexes(), } } @@ -343,6 +383,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.find_contested_index(), DocumentTypeRef::V1(v1) => v1.find_contested_index(), DocumentTypeRef::V2(v2) => v2.find_contested_index(), + DocumentTypeRef::V3(v3) => v3.find_contested_index(), } } @@ -351,6 +392,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.index_structure(), DocumentTypeRef::V1(v1) => v1.index_structure(), DocumentTypeRef::V2(v2) => v2.index_structure(), + DocumentTypeRef::V3(v3) => v3.index_structure(), } } @@ -359,6 +401,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.flattened_properties(), DocumentTypeRef::V1(v1) => v1.flattened_properties(), DocumentTypeRef::V2(v2) => v2.flattened_properties(), + DocumentTypeRef::V3(v3) => v3.flattened_properties(), } } @@ -367,6 +410,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.properties(), DocumentTypeRef::V1(v1) => v1.properties(), DocumentTypeRef::V2(v2) => v2.properties(), + DocumentTypeRef::V3(v3) => v3.properties(), } } @@ -375,6 +419,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.identifier_paths(), DocumentTypeRef::V1(v1) => v1.identifier_paths(), DocumentTypeRef::V2(v2) => v2.identifier_paths(), + DocumentTypeRef::V3(v3) => v3.identifier_paths(), } } @@ -383,6 +428,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.binary_paths(), DocumentTypeRef::V1(v1) => v1.binary_paths(), DocumentTypeRef::V2(v2) => v2.binary_paths(), + DocumentTypeRef::V3(v3) => v3.binary_paths(), } } @@ -391,6 +437,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.required_fields(), DocumentTypeRef::V1(v1) => v1.required_fields(), DocumentTypeRef::V2(v2) => v2.required_fields(), + DocumentTypeRef::V3(v3) => v3.required_fields(), } } @@ -399,6 +446,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.transient_fields(), DocumentTypeRef::V1(v1) => v1.transient_fields(), DocumentTypeRef::V2(v2) => v2.transient_fields(), + DocumentTypeRef::V3(v3) => v3.transient_fields(), } } @@ -407,6 +455,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.documents_keep_history(), DocumentTypeRef::V1(v1) => v1.documents_keep_history(), DocumentTypeRef::V2(v2) => v2.documents_keep_history(), + DocumentTypeRef::V3(v3) => v3.documents_keep_history(), } } @@ -415,6 +464,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.documents_keep_transfer_history(), DocumentTypeRef::V1(v1) => v1.documents_keep_transfer_history(), DocumentTypeRef::V2(v2) => v2.documents_keep_transfer_history(), + DocumentTypeRef::V3(v3) => v3.documents_keep_transfer_history(), } } @@ -423,6 +473,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.documents_keep_purchase_history(), DocumentTypeRef::V1(v1) => v1.documents_keep_purchase_history(), DocumentTypeRef::V2(v2) => v2.documents_keep_purchase_history(), + DocumentTypeRef::V3(v3) => v3.documents_keep_purchase_history(), } } @@ -431,6 +482,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.documents_keep_pricing_history(), DocumentTypeRef::V1(v1) => v1.documents_keep_pricing_history(), DocumentTypeRef::V2(v2) => v2.documents_keep_pricing_history(), + DocumentTypeRef::V3(v3) => v3.documents_keep_pricing_history(), } } @@ -439,6 +491,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.documents_mutable(), DocumentTypeRef::V1(v1) => v1.documents_mutable(), DocumentTypeRef::V2(v2) => v2.documents_mutable(), + DocumentTypeRef::V3(v3) => v3.documents_mutable(), } } @@ -447,6 +500,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.documents_can_be_deleted(), DocumentTypeRef::V1(v1) => v1.documents_can_be_deleted(), DocumentTypeRef::V2(v2) => v2.documents_can_be_deleted(), + DocumentTypeRef::V3(v3) => v3.documents_can_be_deleted(), } } @@ -455,6 +509,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.documents_transferable(), DocumentTypeRef::V1(v1) => v1.documents_transferable(), DocumentTypeRef::V2(v2) => v2.documents_transferable(), + DocumentTypeRef::V3(v3) => v3.documents_transferable(), } } @@ -463,6 +518,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.trade_mode(), DocumentTypeRef::V1(v1) => v1.trade_mode(), DocumentTypeRef::V2(v2) => v2.trade_mode(), + DocumentTypeRef::V3(v3) => v3.trade_mode(), } } @@ -471,6 +527,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.creation_restriction_mode(), DocumentTypeRef::V1(v1) => v1.creation_restriction_mode(), DocumentTypeRef::V2(v2) => v2.creation_restriction_mode(), + DocumentTypeRef::V3(v3) => v3.creation_restriction_mode(), } } @@ -479,6 +536,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.data_contract_id(), DocumentTypeRef::V1(v1) => v1.data_contract_id(), DocumentTypeRef::V2(v2) => v2.data_contract_id(), + DocumentTypeRef::V3(v3) => v3.data_contract_id(), } } @@ -487,6 +545,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.requires_identity_encryption_bounded_key(), DocumentTypeRef::V1(v1) => v1.requires_identity_encryption_bounded_key(), DocumentTypeRef::V2(v2) => v2.requires_identity_encryption_bounded_key(), + DocumentTypeRef::V3(v3) => v3.requires_identity_encryption_bounded_key(), } } @@ -495,6 +554,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.requires_identity_decryption_bounded_key(), DocumentTypeRef::V1(v1) => v1.requires_identity_decryption_bounded_key(), DocumentTypeRef::V2(v2) => v2.requires_identity_decryption_bounded_key(), + DocumentTypeRef::V3(v3) => v3.requires_identity_decryption_bounded_key(), } } @@ -503,6 +563,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.security_level_requirement(), DocumentTypeRef::V1(v1) => v1.security_level_requirement(), DocumentTypeRef::V2(v2) => v2.security_level_requirement(), + DocumentTypeRef::V3(v3) => v3.security_level_requirement(), } } @@ -512,6 +573,7 @@ impl DocumentTypeV0Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => v0.json_schema_validator_ref(), DocumentTypeRef::V1(v1) => v1.json_schema_validator_ref(), DocumentTypeRef::V2(v2) => v2.json_schema_validator_ref(), + DocumentTypeRef::V3(v3) => v3.json_schema_validator_ref(), } } } @@ -521,6 +583,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.name(), DocumentTypeMutRef::V1(v1) => v1.name(), DocumentTypeMutRef::V2(v2) => v2.name(), + DocumentTypeMutRef::V3(v3) => v3.name(), } } @@ -529,6 +592,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.schema(), DocumentTypeMutRef::V1(v1) => v1.schema(), DocumentTypeMutRef::V2(v2) => v2.schema(), + DocumentTypeMutRef::V3(v3) => v3.schema(), } } @@ -537,6 +601,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.clone().schema_owned(), DocumentTypeMutRef::V1(v1) => v1.clone().schema_owned(), DocumentTypeMutRef::V2(v2) => v2.clone().schema_owned(), + DocumentTypeMutRef::V3(v3) => v3.clone().schema_owned(), } } @@ -545,6 +610,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.indexes(), DocumentTypeMutRef::V1(v1) => v1.indexes(), DocumentTypeMutRef::V2(v2) => v2.indexes(), + DocumentTypeMutRef::V3(v3) => v3.indexes(), } } @@ -553,6 +619,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.find_contested_index(), DocumentTypeMutRef::V1(v1) => v1.find_contested_index(), DocumentTypeMutRef::V2(v2) => v2.find_contested_index(), + DocumentTypeMutRef::V3(v3) => v3.find_contested_index(), } } @@ -561,6 +628,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.index_structure(), DocumentTypeMutRef::V1(v1) => v1.index_structure(), DocumentTypeMutRef::V2(v2) => v2.index_structure(), + DocumentTypeMutRef::V3(v3) => v3.index_structure(), } } @@ -569,6 +637,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.flattened_properties(), DocumentTypeMutRef::V1(v1) => v1.flattened_properties(), DocumentTypeMutRef::V2(v2) => v2.flattened_properties(), + DocumentTypeMutRef::V3(v3) => v3.flattened_properties(), } } @@ -577,6 +646,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.properties(), DocumentTypeMutRef::V1(v1) => v1.properties(), DocumentTypeMutRef::V2(v2) => v2.properties(), + DocumentTypeMutRef::V3(v3) => v3.properties(), } } @@ -585,6 +655,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.identifier_paths(), DocumentTypeMutRef::V1(v1) => v1.identifier_paths(), DocumentTypeMutRef::V2(v2) => v2.identifier_paths(), + DocumentTypeMutRef::V3(v3) => v3.identifier_paths(), } } @@ -593,6 +664,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.binary_paths(), DocumentTypeMutRef::V1(v1) => v1.binary_paths(), DocumentTypeMutRef::V2(v2) => v2.binary_paths(), + DocumentTypeMutRef::V3(v3) => v3.binary_paths(), } } @@ -601,6 +673,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.required_fields(), DocumentTypeMutRef::V1(v1) => v1.required_fields(), DocumentTypeMutRef::V2(v2) => v2.required_fields(), + DocumentTypeMutRef::V3(v3) => v3.required_fields(), } } @@ -609,6 +682,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.transient_fields(), DocumentTypeMutRef::V1(v1) => v1.transient_fields(), DocumentTypeMutRef::V2(v2) => v2.transient_fields(), + DocumentTypeMutRef::V3(v3) => v3.transient_fields(), } } @@ -617,6 +691,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.documents_keep_history(), DocumentTypeMutRef::V1(v1) => v1.documents_keep_history(), DocumentTypeMutRef::V2(v2) => v2.documents_keep_history(), + DocumentTypeMutRef::V3(v3) => v3.documents_keep_history(), } } @@ -625,6 +700,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.documents_keep_transfer_history(), DocumentTypeMutRef::V1(v1) => v1.documents_keep_transfer_history(), DocumentTypeMutRef::V2(v2) => v2.documents_keep_transfer_history(), + DocumentTypeMutRef::V3(v3) => v3.documents_keep_transfer_history(), } } @@ -633,6 +709,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.documents_keep_purchase_history(), DocumentTypeMutRef::V1(v1) => v1.documents_keep_purchase_history(), DocumentTypeMutRef::V2(v2) => v2.documents_keep_purchase_history(), + DocumentTypeMutRef::V3(v3) => v3.documents_keep_purchase_history(), } } @@ -641,6 +718,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.documents_keep_pricing_history(), DocumentTypeMutRef::V1(v1) => v1.documents_keep_pricing_history(), DocumentTypeMutRef::V2(v2) => v2.documents_keep_pricing_history(), + DocumentTypeMutRef::V3(v3) => v3.documents_keep_pricing_history(), } } @@ -649,6 +727,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.documents_mutable(), DocumentTypeMutRef::V1(v1) => v1.documents_mutable(), DocumentTypeMutRef::V2(v2) => v2.documents_mutable(), + DocumentTypeMutRef::V3(v3) => v3.documents_mutable(), } } @@ -657,6 +736,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.documents_can_be_deleted(), DocumentTypeMutRef::V1(v1) => v1.documents_can_be_deleted(), DocumentTypeMutRef::V2(v2) => v2.documents_can_be_deleted(), + DocumentTypeMutRef::V3(v3) => v3.documents_can_be_deleted(), } } @@ -665,6 +745,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.documents_transferable(), DocumentTypeMutRef::V1(v1) => v1.documents_transferable(), DocumentTypeMutRef::V2(v2) => v2.documents_transferable(), + DocumentTypeMutRef::V3(v3) => v3.documents_transferable(), } } @@ -673,6 +754,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.trade_mode(), DocumentTypeMutRef::V1(v1) => v1.trade_mode(), DocumentTypeMutRef::V2(v2) => v2.trade_mode(), + DocumentTypeMutRef::V3(v3) => v3.trade_mode(), } } @@ -681,6 +763,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.creation_restriction_mode(), DocumentTypeMutRef::V1(v1) => v1.creation_restriction_mode(), DocumentTypeMutRef::V2(v2) => v2.creation_restriction_mode(), + DocumentTypeMutRef::V3(v3) => v3.creation_restriction_mode(), } } @@ -689,6 +772,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.data_contract_id(), DocumentTypeMutRef::V1(v1) => v1.data_contract_id(), DocumentTypeMutRef::V2(v2) => v2.data_contract_id(), + DocumentTypeMutRef::V3(v3) => v3.data_contract_id(), } } @@ -697,6 +781,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.requires_identity_encryption_bounded_key(), DocumentTypeMutRef::V1(v1) => v1.requires_identity_encryption_bounded_key(), DocumentTypeMutRef::V2(v2) => v2.requires_identity_encryption_bounded_key(), + DocumentTypeMutRef::V3(v3) => v3.requires_identity_encryption_bounded_key(), } } @@ -705,6 +790,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.requires_identity_decryption_bounded_key(), DocumentTypeMutRef::V1(v1) => v1.requires_identity_decryption_bounded_key(), DocumentTypeMutRef::V2(v2) => v2.requires_identity_decryption_bounded_key(), + DocumentTypeMutRef::V3(v3) => v3.requires_identity_decryption_bounded_key(), } } @@ -713,6 +799,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.security_level_requirement(), DocumentTypeMutRef::V1(v1) => v1.security_level_requirement(), DocumentTypeMutRef::V2(v2) => v2.security_level_requirement(), + DocumentTypeMutRef::V3(v3) => v3.security_level_requirement(), } } @@ -722,6 +809,7 @@ impl DocumentTypeV0Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.json_schema_validator_ref(), DocumentTypeMutRef::V1(v1) => v1.json_schema_validator_ref(), DocumentTypeMutRef::V2(v2) => v2.json_schema_validator_ref(), + DocumentTypeMutRef::V3(v3) => v3.json_schema_validator_ref(), } } } @@ -732,6 +820,7 @@ impl DocumentTypeV0Setters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(v0) => v0.set_data_contract_id(data_contract_id), DocumentTypeMutRef::V1(v1) => v1.set_data_contract_id(data_contract_id), DocumentTypeMutRef::V2(v2) => v2.set_data_contract_id(data_contract_id), + DocumentTypeMutRef::V3(v3) => v3.set_data_contract_id(data_contract_id), } } } @@ -742,6 +831,7 @@ impl DocumentTypeV1Getters for DocumentType { DocumentType::V0(_) => None, DocumentType::V1(v1) => v1.document_creation_token_cost(), DocumentType::V2(v2) => v2.document_creation_token_cost(), + DocumentType::V3(v3) => v3.document_creation_token_cost(), } } @@ -750,6 +840,7 @@ impl DocumentTypeV1Getters for DocumentType { DocumentType::V0(_) => None, DocumentType::V1(v1) => v1.document_replacement_token_cost(), DocumentType::V2(v2) => v2.document_replacement_token_cost(), + DocumentType::V3(v3) => v3.document_replacement_token_cost(), } } @@ -758,6 +849,7 @@ impl DocumentTypeV1Getters for DocumentType { DocumentType::V0(_) => None, DocumentType::V1(v1) => v1.document_deletion_token_cost(), DocumentType::V2(v2) => v2.document_deletion_token_cost(), + DocumentType::V3(v3) => v3.document_deletion_token_cost(), } } @@ -766,6 +858,7 @@ impl DocumentTypeV1Getters for DocumentType { DocumentType::V0(_) => None, DocumentType::V1(v1) => v1.document_transfer_token_cost(), DocumentType::V2(v2) => v2.document_transfer_token_cost(), + DocumentType::V3(v3) => v3.document_transfer_token_cost(), } } @@ -774,6 +867,7 @@ impl DocumentTypeV1Getters for DocumentType { DocumentType::V0(_) => None, DocumentType::V1(v1) => v1.document_update_price_token_cost(), DocumentType::V2(v2) => v2.document_update_price_token_cost(), + DocumentType::V3(v3) => v3.document_update_price_token_cost(), } } @@ -782,6 +876,7 @@ impl DocumentTypeV1Getters for DocumentType { DocumentType::V0(_) => None, DocumentType::V1(v1) => v1.document_purchase_token_cost(), DocumentType::V2(v2) => v2.document_purchase_token_cost(), + DocumentType::V3(v3) => v3.document_purchase_token_cost(), } } @@ -790,6 +885,7 @@ impl DocumentTypeV1Getters for DocumentType { DocumentType::V0(_) => vec![], DocumentType::V1(v1) => v1.all_document_token_costs(), DocumentType::V2(v2) => v2.all_document_token_costs(), + DocumentType::V3(v3) => v3.all_document_token_costs(), } } @@ -800,6 +896,7 @@ impl DocumentTypeV1Getters for DocumentType { DocumentType::V0(_) => BTreeMap::new(), DocumentType::V1(v1) => v1.all_external_token_costs_contract_tokens(), DocumentType::V2(v2) => v2.all_external_token_costs_contract_tokens(), + DocumentType::V3(v3) => v3.all_external_token_costs_contract_tokens(), } } } @@ -810,6 +907,7 @@ impl DocumentTypeV1Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(_) => None, DocumentTypeRef::V1(v1) => v1.document_creation_token_cost(), DocumentTypeRef::V2(v2) => v2.document_creation_token_cost(), + DocumentTypeRef::V3(v3) => v3.document_creation_token_cost(), } } @@ -818,6 +916,7 @@ impl DocumentTypeV1Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(_) => None, DocumentTypeRef::V1(v1) => v1.document_replacement_token_cost(), DocumentTypeRef::V2(v2) => v2.document_replacement_token_cost(), + DocumentTypeRef::V3(v3) => v3.document_replacement_token_cost(), } } @@ -826,6 +925,7 @@ impl DocumentTypeV1Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(_) => None, DocumentTypeRef::V1(v1) => v1.document_deletion_token_cost(), DocumentTypeRef::V2(v2) => v2.document_deletion_token_cost(), + DocumentTypeRef::V3(v3) => v3.document_deletion_token_cost(), } } @@ -834,6 +934,7 @@ impl DocumentTypeV1Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(_) => None, DocumentTypeRef::V1(v1) => v1.document_transfer_token_cost(), DocumentTypeRef::V2(v2) => v2.document_transfer_token_cost(), + DocumentTypeRef::V3(v3) => v3.document_transfer_token_cost(), } } @@ -842,6 +943,7 @@ impl DocumentTypeV1Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(_) => None, DocumentTypeRef::V1(v1) => v1.document_update_price_token_cost(), DocumentTypeRef::V2(v2) => v2.document_update_price_token_cost(), + DocumentTypeRef::V3(v3) => v3.document_update_price_token_cost(), } } @@ -850,6 +952,7 @@ impl DocumentTypeV1Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(_) => None, DocumentTypeRef::V1(v1) => v1.document_purchase_token_cost(), DocumentTypeRef::V2(v2) => v2.document_purchase_token_cost(), + DocumentTypeRef::V3(v3) => v3.document_purchase_token_cost(), } } @@ -858,6 +961,7 @@ impl DocumentTypeV1Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(_) => vec![], DocumentTypeRef::V1(v1) => v1.all_document_token_costs(), DocumentTypeRef::V2(v2) => v2.all_document_token_costs(), + DocumentTypeRef::V3(v3) => v3.all_document_token_costs(), } } @@ -868,6 +972,7 @@ impl DocumentTypeV1Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(_) => BTreeMap::new(), DocumentTypeRef::V1(v1) => v1.all_external_token_costs_contract_tokens(), DocumentTypeRef::V2(v2) => v2.all_external_token_costs_contract_tokens(), + DocumentTypeRef::V3(v3) => v3.all_external_token_costs_contract_tokens(), } } } @@ -878,6 +983,7 @@ impl DocumentTypeV1Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(_) => None, DocumentTypeMutRef::V1(v1) => v1.document_creation_token_cost(), DocumentTypeMutRef::V2(v2) => v2.document_creation_token_cost(), + DocumentTypeMutRef::V3(v3) => v3.document_creation_token_cost(), } } @@ -886,6 +992,7 @@ impl DocumentTypeV1Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(_) => None, DocumentTypeMutRef::V1(v1) => v1.document_replacement_token_cost(), DocumentTypeMutRef::V2(v2) => v2.document_replacement_token_cost(), + DocumentTypeMutRef::V3(v3) => v3.document_replacement_token_cost(), } } @@ -894,6 +1001,7 @@ impl DocumentTypeV1Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(_) => None, DocumentTypeMutRef::V1(v1) => v1.document_deletion_token_cost(), DocumentTypeMutRef::V2(v2) => v2.document_deletion_token_cost(), + DocumentTypeMutRef::V3(v3) => v3.document_deletion_token_cost(), } } @@ -902,6 +1010,7 @@ impl DocumentTypeV1Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(_) => None, DocumentTypeMutRef::V1(v1) => v1.document_transfer_token_cost(), DocumentTypeMutRef::V2(v2) => v2.document_transfer_token_cost(), + DocumentTypeMutRef::V3(v3) => v3.document_transfer_token_cost(), } } @@ -910,6 +1019,7 @@ impl DocumentTypeV1Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(_) => None, DocumentTypeMutRef::V1(v1) => v1.document_update_price_token_cost(), DocumentTypeMutRef::V2(v2) => v2.document_update_price_token_cost(), + DocumentTypeMutRef::V3(v3) => v3.document_update_price_token_cost(), } } @@ -918,6 +1028,7 @@ impl DocumentTypeV1Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(_) => None, DocumentTypeMutRef::V1(v1) => v1.document_purchase_token_cost(), DocumentTypeMutRef::V2(v2) => v2.document_purchase_token_cost(), + DocumentTypeMutRef::V3(v3) => v3.document_purchase_token_cost(), } } @@ -926,6 +1037,7 @@ impl DocumentTypeV1Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(_) => vec![], DocumentTypeMutRef::V1(v1) => v1.all_document_token_costs(), DocumentTypeMutRef::V2(v2) => v2.all_document_token_costs(), + DocumentTypeMutRef::V3(v3) => v3.all_document_token_costs(), } } @@ -936,6 +1048,7 @@ impl DocumentTypeV1Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(_) => BTreeMap::new(), DocumentTypeMutRef::V1(v1) => v1.all_external_token_costs_contract_tokens(), DocumentTypeMutRef::V2(v2) => v2.all_external_token_costs_contract_tokens(), + DocumentTypeMutRef::V3(v3) => v3.all_external_token_costs_contract_tokens(), } } } @@ -946,6 +1059,7 @@ impl DocumentTypeV2Getters for DocumentType { DocumentType::V0(_) => false, DocumentType::V1(_) => false, DocumentType::V2(v2) => v2.documents_countable(), + DocumentType::V3(v3) => v3.documents_countable(), } } @@ -954,6 +1068,7 @@ impl DocumentTypeV2Getters for DocumentType { DocumentType::V0(_) => false, DocumentType::V1(_) => false, DocumentType::V2(v2) => v2.range_countable(), + DocumentType::V3(v3) => v3.range_countable(), } } @@ -962,6 +1077,7 @@ impl DocumentTypeV2Getters for DocumentType { DocumentType::V0(_) => None, DocumentType::V1(_) => None, DocumentType::V2(v2) => v2.documents_summable(), + DocumentType::V3(v3) => v3.documents_summable(), } } @@ -970,6 +1086,7 @@ impl DocumentTypeV2Getters for DocumentType { DocumentType::V0(_) => false, DocumentType::V1(_) => false, DocumentType::V2(v2) => v2.range_summable(), + DocumentType::V3(v3) => v3.range_summable(), } } @@ -978,6 +1095,7 @@ impl DocumentTypeV2Getters for DocumentType { DocumentType::V0(_) => false, DocumentType::V1(_) => false, DocumentType::V2(v2) => v2.index_only(), + DocumentType::V3(v3) => v3.index_only(), } } } @@ -988,6 +1106,7 @@ impl DocumentTypeV2Setters for DocumentType { DocumentType::V0(_) => { /* no-op */ } DocumentType::V1(_) => { /* no-op */ } DocumentType::V2(v2) => v2.set_documents_countable(countable), + DocumentType::V3(v3) => v3.set_documents_countable(countable), } } @@ -996,6 +1115,7 @@ impl DocumentTypeV2Setters for DocumentType { DocumentType::V0(_) => { /* no-op */ } DocumentType::V1(_) => { /* no-op */ } DocumentType::V2(v2) => v2.set_range_countable(range_countable), + DocumentType::V3(v3) => v3.set_range_countable(range_countable), } } @@ -1004,6 +1124,7 @@ impl DocumentTypeV2Setters for DocumentType { DocumentType::V0(_) => { /* no-op */ } DocumentType::V1(_) => { /* no-op */ } DocumentType::V2(v2) => v2.set_documents_summable(property), + DocumentType::V3(v3) => v3.set_documents_summable(property), } } @@ -1012,6 +1133,7 @@ impl DocumentTypeV2Setters for DocumentType { DocumentType::V0(_) => { /* no-op */ } DocumentType::V1(_) => { /* no-op */ } DocumentType::V2(v2) => v2.set_range_summable(range_summable), + DocumentType::V3(v3) => v3.set_range_summable(range_summable), } } } @@ -1022,6 +1144,7 @@ impl DocumentTypeV2Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(_) => false, DocumentTypeRef::V1(_) => false, DocumentTypeRef::V2(v2) => v2.documents_countable(), + DocumentTypeRef::V3(v3) => v3.documents_countable(), } } @@ -1030,6 +1153,7 @@ impl DocumentTypeV2Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(_) => false, DocumentTypeRef::V1(_) => false, DocumentTypeRef::V2(v2) => v2.range_countable(), + DocumentTypeRef::V3(v3) => v3.range_countable(), } } @@ -1038,6 +1162,7 @@ impl DocumentTypeV2Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(_) => None, DocumentTypeRef::V1(_) => None, DocumentTypeRef::V2(v2) => v2.documents_summable(), + DocumentTypeRef::V3(v3) => v3.documents_summable(), } } @@ -1046,6 +1171,7 @@ impl DocumentTypeV2Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(_) => false, DocumentTypeRef::V1(_) => false, DocumentTypeRef::V2(v2) => v2.range_summable(), + DocumentTypeRef::V3(v3) => v3.range_summable(), } } @@ -1054,6 +1180,7 @@ impl DocumentTypeV2Getters for DocumentTypeRef<'_> { DocumentTypeRef::V0(_) => false, DocumentTypeRef::V1(_) => false, DocumentTypeRef::V2(v2) => v2.index_only(), + DocumentTypeRef::V3(v3) => v3.index_only(), } } } @@ -1064,6 +1191,7 @@ impl DocumentTypeV2Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(_) => false, DocumentTypeMutRef::V1(_) => false, DocumentTypeMutRef::V2(v2) => v2.documents_countable(), + DocumentTypeMutRef::V3(v3) => v3.documents_countable(), } } @@ -1072,6 +1200,7 @@ impl DocumentTypeV2Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(_) => false, DocumentTypeMutRef::V1(_) => false, DocumentTypeMutRef::V2(v2) => v2.range_countable(), + DocumentTypeMutRef::V3(v3) => v3.range_countable(), } } @@ -1080,6 +1209,7 @@ impl DocumentTypeV2Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(_) => None, DocumentTypeMutRef::V1(_) => None, DocumentTypeMutRef::V2(v2) => v2.documents_summable(), + DocumentTypeMutRef::V3(v3) => v3.documents_summable(), } } @@ -1088,6 +1218,7 @@ impl DocumentTypeV2Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(_) => false, DocumentTypeMutRef::V1(_) => false, DocumentTypeMutRef::V2(v2) => v2.range_summable(), + DocumentTypeMutRef::V3(v3) => v3.range_summable(), } } @@ -1096,6 +1227,40 @@ impl DocumentTypeV2Getters for DocumentTypeMutRef<'_> { DocumentTypeMutRef::V0(_) => false, DocumentTypeMutRef::V1(_) => false, DocumentTypeMutRef::V2(v2) => v2.index_only(), + DocumentTypeMutRef::V3(v3) => v3.index_only(), + } + } +} + +impl DocumentTypeV3Getters for DocumentType { + fn documents_can_be_erased(&self) -> bool { + match self { + DocumentType::V0(_) => false, + DocumentType::V1(_) => false, + DocumentType::V2(_) => false, + DocumentType::V3(v3) => v3.documents_can_be_erased(), + } + } +} + +impl DocumentTypeV3Getters for DocumentTypeRef<'_> { + fn documents_can_be_erased(&self) -> bool { + match self { + DocumentTypeRef::V0(_) => false, + DocumentTypeRef::V1(_) => false, + DocumentTypeRef::V2(_) => false, + DocumentTypeRef::V3(v3) => v3.documents_can_be_erased(), + } + } +} + +impl DocumentTypeV3Getters for DocumentTypeMutRef<'_> { + fn documents_can_be_erased(&self) -> bool { + match self { + DocumentTypeMutRef::V0(_) => false, + DocumentTypeMutRef::V1(_) => false, + DocumentTypeMutRef::V2(_) => false, + DocumentTypeMutRef::V3(v3) => v3.documents_can_be_erased(), } } } diff --git a/packages/rs-dpp/src/data_contract/document_type/accessors/v3/mod.rs b/packages/rs-dpp/src/data_contract/document_type/accessors/v3/mod.rs new file mode 100644 index 00000000000..803e4b1f9c3 --- /dev/null +++ b/packages/rs-dpp/src/data_contract/document_type/accessors/v3/mod.rs @@ -0,0 +1,8 @@ +/// Getters introduced with document type version 3: the keep-history document +/// lifecycle keywords of meta-schema v4. +pub trait DocumentTypeV3Getters { + /// Returns whether a deleted document of this type may have its retained + /// revisions purged by an erase transition. Always false for a document + /// type parsed by an earlier grammar, which has no such keyword. + fn documents_can_be_erased(&self) -> bool; +} diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/common/mod.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/common/mod.rs index 748ce4a3ff2..f6e532c6823 100644 --- a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/common/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/common/mod.rs @@ -26,8 +26,8 @@ use crate::data_contract::document_type::index_level::IndexLevel; use crate::data_contract::document_type::property::DocumentProperty; use crate::data_contract::document_type::property::DocumentPropertyType; use crate::data_contract::document_type::property_names::{ - CAN_BE_DELETED, CREATION_RESTRICTION_MODE, DOCUMENTS_AVERAGEABLE, DOCUMENTS_COUNTABLE, - DOCUMENTS_KEEP_HISTORY, DOCUMENTS_MUTABLE, DOCUMENTS_SUMMABLE, INDEX_ONLY, + CAN_BE_DELETED, CAN_BE_ERASED, CREATION_RESTRICTION_MODE, DOCUMENTS_AVERAGEABLE, + DOCUMENTS_COUNTABLE, DOCUMENTS_KEEP_HISTORY, DOCUMENTS_MUTABLE, DOCUMENTS_SUMMABLE, INDEX_ONLY, KEEPS_PRICING_HISTORY, KEEPS_PURCHASE_HISTORY, KEEPS_TRANSFER_HISTORY, RANGE_AVERAGEABLE, RANGE_COUNTABLE, RANGE_SUMMABLE, TRADE_MODE, TRANSFERABLE, }; @@ -36,6 +36,7 @@ use crate::data_contract::document_type::token_costs::v0::TokenCostsV0; use crate::data_contract::document_type::token_costs::TokenCosts; use crate::data_contract::document_type::v1::DocumentTypeV1; use crate::data_contract::document_type::v2::DocumentTypeV2; +use crate::data_contract::document_type::v3::DocumentTypeV3; use crate::data_contract::document_type::{property_names, DocumentType}; use crate::data_contract::errors::DataContractError; use crate::data_contract::storage_requirements::keys_for_document_type::StorageKeyRequirements; @@ -89,7 +90,7 @@ use crate::data_contract::document_type::validator::StatelessJsonSchemaLazyValid #[cfg(feature = "validation")] use crate::validation::meta_validators::{ DOCUMENT_META_SCHEMA_V0, DOCUMENT_META_SCHEMA_V1, DOCUMENT_META_SCHEMA_V2, - DOCUMENT_META_SCHEMA_V3, + DOCUMENT_META_SCHEMA_V3, DOCUMENT_META_SCHEMA_V4, }; #[cfg(feature = "validation")] use jsonschema::JSONSchema; @@ -205,8 +206,8 @@ pub(super) struct ParserGeneration { /// document types). Forwarded to [`Index::try_from_value_map`] exactly /// like `admit_ranked` and `admit_time_range`. The doc-type-level /// `indexOnly` keyword needs no admission flag of its own: it is read - /// only by the generation-3 driver (`parse_index_only_keyword`), so - /// earlier generations ignore it exactly as they ignore every other + /// only by the generation-3 and later drivers (`parse_index_only_keyword`), + /// so earlier generations ignore it exactly as they ignore every other /// doctype-level keyword they predate. pub admit_index_terminal: bool, /// Whether the index grammar admits the `preallocated` keyword @@ -290,10 +291,11 @@ pub(super) fn select_document_meta_schema( 1 => &*DOCUMENT_META_SCHEMA_V1, 2 => &*DOCUMENT_META_SCHEMA_V2, 3 => &*DOCUMENT_META_SCHEMA_V3, + 4 => &*DOCUMENT_META_SCHEMA_V4, version => { return Err(ProtocolError::UnknownVersionMismatch { method: method_name.to_string(), - known_versions: vec![0, 1, 2, 3], + known_versions: vec![0, 1, 2, 3, 4], received: version, }) } @@ -1942,11 +1944,117 @@ pub(super) fn apply_doctype_aggregates( Ok(()) } +/// Read the doctype-level `canBeErased` flag out of the raw schema. +/// +/// Runs before the core parse for the same reason as +/// [`parse_index_only_keyword`]: the core takes `schema` by value. Only the +/// generation-4 driver calls this; earlier generations have no such keyword +/// and their meta-schemas reject it under `full_validation`. +pub(super) fn parse_can_be_erased_keyword(schema: &Value) -> Result { + let schema_map_opt = schema.to_map().ok(); + + Ok(schema_map_opt + .as_ref() + .and_then(|schema_map| { + Value::inner_optional_bool_value(schema_map, CAN_BE_ERASED) + .map_err(consensus_or_protocol_value_error) + .transpose() + }) + .transpose()? + .unwrap_or(false)) +} + +/// Write the `canBeErased` flag onto the parsed document type after checking +/// the two settings it depends on. +/// +/// Erase purges the retained revisions of a document that has already been +/// deleted, so a type that keeps no history has nothing to purge and a type +/// whose documents can never be deleted can never reach the state erase acts +/// on. Both are rejected rather than silently ignored: a document type whose +/// declared behavior and reachable behavior disagree is a contract-authoring +/// error, not a default. +/// +/// Runs regardless of `full_validation`, like [`apply_index_only`]: the flag +/// governs an irreversible operation, so a stored contract must never come +/// back out of the parser with the flag set on a type that cannot support it. +pub(super) fn apply_can_be_erased( + document_type: &mut DocumentTypeV3, + can_be_erased: bool, + name: &str, +) -> Result<(), ProtocolError> { + if !can_be_erased { + return Ok(()); + } + + // A consensus error, not a bare data-contract error: only the consensus + // variant becomes a paid rejection with a nonce bump when a signed contract + // create or update carries the combination. The bare variant would escape as + // an internal execution error and cost the submitter nothing. + let structure_error = |message: String| { + consensus_or_protocol_data_contract_error(DataContractError::InvalidContractStructure( + message, + )) + }; + + if !document_type.documents_keep_history { + return Err(structure_error(format!( + "document type \"{}\" sets `canBeErased: true` but does not keep history: erase \ + removes retained revisions, and a type without history has none", + name, + ))); + } + if !document_type.documents_can_be_deleted { + return Err(structure_error(format!( + "document type \"{}\" sets `canBeErased: true` but `canBeDeleted: false`: erase \ + applies to deleted documents only, so its documents could never be erased", + name, + ))); + } + + document_type.documents_can_be_erased = true; + Ok(()) +} + +/// Reject a keep-history document type that carries a contested index. +/// +/// A contested resource is awarded by block processing, outside state +/// transition validation, at an id derived from the winner rather than from +/// the contested values. On a keep-history type that award can land on an id +/// whose retained history already exists, which the storage guard turns into a +/// deterministic block failure on every validator. Making the combination safe +/// needs changes to the contested machinery itself, so until then the two are +/// kept apart at contract registration. +pub(super) fn reject_contested_keep_history( + document_type: &DocumentTypeV3, + name: &str, +) -> Result<(), ProtocolError> { + if !document_type.documents_keep_history { + return Ok(()); + } + if let Some((index_name, _)) = document_type + .indices + .iter() + .find(|(_, index)| index.contested_index.is_some()) + { + // Consensus error for the same reason as `apply_can_be_erased`: a + // signed contract carrying this combination must be a paid rejection. + return Err(consensus_or_protocol_data_contract_error( + DataContractError::InvalidContractStructure(format!( + "document type \"{}\" sets `documentsKeepHistory: true` and declares the \ + contested index \"{}\": a contested resource is awarded outside transition \ + validation and cannot be combined with retained history", + name, index_name, + )), + )); + } + Ok(()) +} + /// Read the doctype-level `indexOnly` keyword off the raw schema. /// /// Runs before the core parse because the core takes `schema` by value — /// same shape as [`parse_doctype_aggregate_keywords`]. Only the generation-3 -/// driver calls this; earlier generations ignore the keyword exactly as they +/// and later drivers call this; earlier generations ignore the keyword exactly as they /// ignore every doctype-level keyword they predate (their meta-schemas still /// reject it under `full_validation`). pub(super) fn parse_index_only_keyword(schema: &Value) -> Result { diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs index 5c006d220d9..1f00b199937 100644 --- a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs @@ -22,6 +22,7 @@ mod v0; mod v1; mod v2; mod v3; +mod v4; const NOT_ALLOWED_SYSTEM_PROPERTIES: [&str; 1] = ["$id"]; @@ -104,9 +105,22 @@ impl DocumentType { validation_operations, platform_version, ), + 4 => DocumentType::try_from_schema_v4( + data_contract_id, + data_contract_system_version, + contract_config_version, + name, + schema, + schema_defs, + token_configurations, + data_contact_config, + full_validation, + validation_operations, + platform_version, + ), version => Err(ProtocolError::UnknownVersionMismatch { method: "try_from_schema".to_string(), - known_versions: vec![0, 1, 2, 3], + known_versions: vec![0, 1, 2, 3, 4], received: version, }), } diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/mod.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/mod.rs index 92002b8c6ad..8be6995123a 100644 --- a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/mod.rs @@ -38,7 +38,7 @@ use crate::consensus::basic::data_contract::InvalidIndexedPropertyConstraintErro use super::common; -mod ranked_prefix_overlap; +pub(in crate::data_contract::document_type::class_methods::try_from_schema) mod ranked_prefix_overlap; use ranked_prefix_overlap::validate_no_ranked_prefix_overlap; /// grovedb's ceiling on the key of an entry stored directly under an @@ -215,10 +215,10 @@ fn validate_ranked_index_property_key_length( /// exactly equivalent there — this alias exists only so the call site can pass /// the check unconditionally. #[cfg(feature = "validation")] -const RANKED_INDEX_KEY_LENGTH_CHECK: common::RankedIndexKeyLengthCheck = +pub(in crate::data_contract::document_type::class_methods::try_from_schema) const RANKED_INDEX_KEY_LENGTH_CHECK: common::RankedIndexKeyLengthCheck = validate_ranked_index_property_key_length; #[cfg(not(feature = "validation"))] -const RANKED_INDEX_KEY_LENGTH_CHECK: common::RankedIndexKeyLengthCheck = +pub(in crate::data_contract::document_type::class_methods::try_from_schema) const RANKED_INDEX_KEY_LENGTH_CHECK: common::RankedIndexKeyLengthCheck = common::no_ranked_index_key_length_check; /// Parses a document type schema through the generation-3 grammar: the diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/ranked_prefix_overlap.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/ranked_prefix_overlap.rs index e27fd0c6c7c..23cfa2b77be 100644 --- a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/ranked_prefix_overlap.rs +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v3/ranked_prefix_overlap.rs @@ -65,7 +65,7 @@ fn shares_leading_levels(ranked: &Index, other: &Index, depth: usize) -> bool { }) } -pub(super) fn validate_no_ranked_prefix_overlap( +pub(in crate::data_contract::document_type::class_methods::try_from_schema) fn validate_no_ranked_prefix_overlap( indices: &BTreeMap, ) -> Result<(), ProtocolError> { for ranked in indices.values() { diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v4/keep_history_tests.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v4/keep_history_tests.rs new file mode 100644 index 00000000000..5f637d54b5c --- /dev/null +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v4/keep_history_tests.rs @@ -0,0 +1,500 @@ +//! The keep-history document lifecycle as generation 4 admits it: a +//! keep-history type may allow deletion, may additionally allow erasure, and +//! may not carry a contested index. +use super::*; +use crate::data_contract::document_type::accessors::{ + DocumentTypeV0Getters, DocumentTypeV3Getters, +}; +use crate::data_contract::errors::DataContractError; +use platform_value::platform_value; + +/// Parses through the public dispatcher at the given protocol version so +/// the test exercises the same `try_from_schema` version routing consensus +/// code uses (generation 4 at protocol version 15, 3 at 14, 2 at 12 and 13). +fn parse_at_version( + schema: Value, + protocol_version: u32, + full_validation: bool, +) -> Result { + let platform_version = + PlatformVersion::get(protocol_version).expect("expected platform version"); + let config = DataContractConfig::default_for_version(platform_version) + .expect("default config available"); + DocumentType::try_from_schema( + Identifier::new([1; 32]), + 1, + config.version(), + "test_doc", + schema, + None, + &BTreeMap::new(), + &config, + full_validation, + &mut vec![], + platform_version, + ) +} + +fn parse(schema: Value) -> Result { + parse_at_version(schema, PlatformVersion::latest().protocol_version, true) +} + +fn parse_without_validation(schema: Value) -> Result { + parse_at_version(schema, PlatformVersion::latest().protocol_version, false) +} + +fn keep_history_deletable_schema() -> Value { + platform_value!({ + "type": "object", + "properties": { + "label": { + "type": "string", + "maxLength": 50, + "position": 0, + }, + }, + "additionalProperties": false, + "documentsKeepHistory": true, + "canBeDeleted": true, + }) +} + +/// The two settings describe different things and no longer contradict each +/// other: history decides what is retained, deletion decides what ordinary +/// reads can still see. A delete on such a type removes the document from +/// every ordinary read and leaves its revisions readable. +#[test] +fn should_accept_a_keep_history_type_that_allows_deletion() { + let document_type = parse(keep_history_deletable_schema()) + .expect("a keep-history type may allow deletion at the latest protocol version"); + assert!(document_type.documents_keep_history()); + assert!(document_type.documents_can_be_deleted()); + assert!( + !document_type.documents_can_be_erased(), + "erasure is not implied by deletion; it has to be asked for" + ); +} + +/// The contract config defaults `canBeDeleted` to true, so a keep-history type +/// that says nothing about deletion is deletable — the same value it carried +/// before protocol 14, now with a delete that works. +#[test] +fn should_accept_a_keep_history_type_that_omits_the_delete_flag() { + let schema = platform_value!({ + "type": "object", + "properties": { + "label": {"type": "string", "maxLength": 50, "position": 0}, + }, + "additionalProperties": false, + "documentsKeepHistory": true, + }); + let document_type = parse(schema).expect("the omitted flag defaults to deletable"); + assert!(document_type.documents_keep_history()); + assert!(document_type.documents_can_be_deleted()); +} + +/// A keep-history type that withholds deletion is still valid: its documents +/// are append-only and can never leave ordinary reads. +#[test] +fn should_accept_a_keep_history_type_that_withholds_deletion() { + let schema = platform_value!({ + "type": "object", + "properties": { + "label": {"type": "string", "maxLength": 50, "position": 0}, + }, + "additionalProperties": false, + "documentsKeepHistory": true, + "canBeDeleted": false, + }); + let document_type = parse(schema).expect("an append-only keep-history type is valid"); + assert!(document_type.documents_keep_history()); + assert!(!document_type.documents_can_be_deleted()); +} + +/// The released parsers must keep parsing the same schemas the same way, so a +/// historical block replays identically: generation 2 never saw a rule about +/// the pair, and generation 3 refuses it under full validation. +#[test] +fn should_keep_the_released_parsers_verdict_on_a_keep_history_deletable_type() { + for protocol in [12, 13] { + let document_type = parse_at_version(keep_history_deletable_schema(), protocol, true) + .unwrap_or_else(|error| { + panic!("protocol {protocol} must still accept the schema: {error:?}") + }); + assert!(document_type.documents_keep_history()); + assert!(document_type.documents_can_be_deleted()); + } + assert!( + parse_at_version(keep_history_deletable_schema(), 14, true).is_err(), + "protocol 14 refuses a deletable keep-history type under full validation" + ); +} + +fn erasable_schema(keep_history: bool, can_be_deleted: bool, can_be_erased: bool) -> Value { + platform_value!({ + "type": "object", + "properties": { + "label": {"type": "string", "maxLength": 50, "position": 0}, + }, + "additionalProperties": false, + "documentsKeepHistory": keep_history, + "canBeDeleted": can_be_deleted, + "canBeErased": can_be_erased, + }) +} + +#[test] +fn should_accept_an_erasable_keep_history_deletable_type() { + let document_type = + parse(erasable_schema(true, true, true)).expect("the admitted combination must parse"); + assert!(document_type.documents_can_be_erased()); +} + +/// Erasure purges retained revisions of a document that has already been +/// deleted, so a type with no history has nothing to purge and a type whose +/// documents can never be deleted can never reach the state erasure acts on. +/// Neither is silently ignored. +#[test] +fn should_reject_erasure_without_history_or_without_deletion() { + for (keep_history, can_be_deleted) in [(false, true), (true, false), (false, false)] { + let error = parse(erasable_schema(keep_history, can_be_deleted, true)).unwrap_err(); + let message = format!("{error:?}"); + assert!( + message.contains("canBeErased"), + "the error must name the flag the author has to change; got {message}" + ); + assert_contract_structure_consensus_error(&error); + } +} + +/// A signed contract carrying a refused combination must be a paid rejection +/// with a nonce bump, and only the consensus variant becomes one: the bare +/// data-contract variant escapes the transformation as an internal execution +/// error, which costs the submitter nothing and reports nothing useful. +#[cfg(feature = "validation")] +fn assert_contract_structure_consensus_error(error: &ProtocolError) { + use crate::consensus::basic::BasicError; + use crate::consensus::ConsensusError; + + let ProtocolError::ConsensusError(consensus) = error else { + panic!("expected a consensus error, got {error:?}"); + }; + assert!( + matches!( + consensus.as_ref(), + ConsensusError::BasicError(BasicError::ContractError( + DataContractError::InvalidContractStructure(_) + )) + ), + "expected an invalid-contract-structure basic error, got {consensus:?}" + ); +} + +#[cfg(not(feature = "validation"))] +fn assert_contract_structure_consensus_error(error: &ProtocolError) { + assert!( + matches!( + error, + ProtocolError::DataContractError(DataContractError::InvalidContractStructure(_)) + ), + "without the validation feature the structural variant is all there is, got {error:?}" + ); +} + +/// Erasure defaults to off: a type that says nothing about it cannot have its +/// revisions purged. +#[test] +fn should_default_erasure_to_off() { + let document_type = parse(keep_history_deletable_schema()).expect("parses"); + assert!(!document_type.documents_can_be_erased()); +} + +/// The keyword does not exist in the earlier meta-schemas, so a contract that +/// declares it is refused there rather than silently parsing without it. +#[test] +fn should_reject_the_erasure_keyword_at_released_protocol_versions() { + for protocol in [12, 13, 14] { + assert!( + parse_at_version(erasable_schema(true, true, true), protocol, true).is_err(), + "protocol {protocol} has no canBeErased keyword" + ); + } +} + +/// A contested resource is awarded outside transition validation, at an id +/// derived from the winner rather than from the contested values, so that award +/// can land on an id whose retained history already exists. The two are kept +/// apart at registration until the contested machinery can handle it. +#[test] +fn should_reject_a_keep_history_type_that_carries_a_contested_index() { + let schema = platform_value!({ + "type": "object", + "properties": { + "label": {"type": "string", "maxLength": 50, "position": 0}, + }, + "indices": [ + { + "name": "byLabel", + "properties": [{"label": "asc"}], + "unique": true, + "contested": { + "fieldMatches": [{"field": "label", "regexPattern": "^[a-z]{3,10}$"}], + "resolution": 0, + }, + }, + ], + "required": ["label"], + "additionalProperties": false, + "documentsMutable": false, + "documentsKeepHistory": true, + "canBeDeleted": false, + }); + let error = parse(schema).expect_err("a contested keep-history type must be refused"); + let message = format!("{error:?}"); + assert!( + message.contains("contested"), + "the error must say which index is the problem; got {message}" + ); + assert_contract_structure_consensus_error(&error); +} + +/// The refusal is a registration rule. A contract that carries the combination +/// was registered under a protocol that allowed it, and every stored contract +/// is loaded through the structural parse, so that parse must keep reading it +/// or the activation that migrates its history could not even start. +#[test] +fn should_still_load_a_legacy_contested_keep_history_type_without_validation() { + let schema = platform_value!({ + "type": "object", + "properties": { + "label": {"type": "string", "maxLength": 50, "position": 0}, + }, + "indices": [ + { + "name": "byLabel", + "properties": [{"label": "asc"}], + "unique": true, + "contested": { + "fieldMatches": [{"field": "label", "regexPattern": "^[a-z]{3,10}$"}], + "resolution": 0, + }, + }, + ], + "required": ["label"], + "additionalProperties": false, + "documentsMutable": false, + "documentsKeepHistory": true, + "canBeDeleted": false, + }); + let document_type = parse_without_validation(schema) + .expect("a stored contract with this combination must still load"); + assert!(document_type.documents_keep_history()); +} + +/// The same index without history is unaffected: the refusal is about the +/// combination, not about contested indexes. +#[test] +fn should_accept_a_contested_index_without_history() { + let schema = platform_value!({ + "type": "object", + "properties": { + "label": {"type": "string", "maxLength": 50, "position": 0}, + }, + "indices": [ + { + "name": "byLabel", + "properties": [{"label": "asc"}], + "unique": true, + "contested": { + "fieldMatches": [{"field": "label", "regexPattern": "^[a-z]{3,10}$"}], + "resolution": 0, + }, + }, + ], + "required": ["label"], + "additionalProperties": false, + "documentsMutable": false, + }); + parse(schema).expect("a contested index alone is fine"); +} + +fn repair_schema(keep_history: bool, can_be_deleted: bool) -> Value { + platform_value!({ + "type": "object", + "properties": { + "label": {"type": "string", "maxLength": 50, "position": 0}, + }, + "additionalProperties": false, + "documentsKeepHistory": keep_history, + "canBeDeleted": can_be_deleted, + }) +} + +/// A keep-history type may withdraw deletion, and only in that direction. +#[test] +fn should_allow_a_keep_history_type_to_withdraw_deletion() { + let old = parse_at_version(repair_schema(true, true), 13, true).unwrap(); + let new = parse(repair_schema(true, false)).unwrap(); + let result = old + .as_ref() + .validate_update(new.as_ref(), 2, PlatformVersion::latest()) + .expect("the update must reach a consensus result"); + assert!(result.is_valid(), "rejected: {:?}", result.errors); +} + +#[test] +fn should_preserve_the_immutable_delete_flag_through_protocol_13() { + for protocol in [12, 13] { + let old = parse_at_version(repair_schema(true, true), protocol, true).unwrap(); + let new = parse_at_version(repair_schema(true, false), protocol, true).unwrap(); + let result = old + .as_ref() + .validate_update(new.as_ref(), 2, PlatformVersion::get(protocol).unwrap()) + .unwrap(); + assert!( + !result.is_valid(), + "protocol {protocol} must still refuse the change" + ); + } +} + +#[test] +fn should_reject_other_delete_and_history_flag_changes() { + for (old_flags, new_flags) in [ + ((false, true), (false, false)), + ((false, false), (false, true)), + ((true, false), (true, true)), + ((true, true), (false, false)), + ((false, true), (true, false)), + ] { + let old = parse_at_version(repair_schema(old_flags.0, old_flags.1), 13, true).unwrap(); + // A caller may already have a parsed contract; update validation must + // enforce immutability even without the full-validation parser guard. + let new = parse_without_validation(repair_schema(new_flags.0, new_flags.1)).unwrap(); + let result = old + .as_ref() + .validate_update(new.as_ref(), 2, PlatformVersion::latest()) + .unwrap(); + assert!( + !result.is_valid(), + "unexpectedly accepted {old_flags:?} -> {new_flags:?}" + ); + } +} + +/// Erasability is immutable in both directions. Widening it hands an +/// irreversible operation to a type registered without it; narrowing it after a +/// first chunk has removed revisions strands a partially erased document. +#[test] +fn should_reject_every_change_to_the_erasure_flag() { + for (before, after) in [(false, true), (true, false)] { + let old = parse_without_validation(erasable_schema(true, true, before)).unwrap(); + let new = parse_without_validation(erasable_schema(true, true, after)).unwrap(); + let result = old + .as_ref() + .validate_update(new.as_ref(), 2, PlatformVersion::latest()) + .unwrap(); + let message = result + .errors + .first() + .unwrap_or_else(|| panic!("unexpectedly accepted canBeErased {before} -> {after}")) + .to_string(); + assert!( + message.contains("can not change whether its documents can be erased"), + "canBeErased {before} -> {after} must be refused for being immutable, got {message}" + ); + } +} + +/// An erasable type can never stop being deletable, whichever way the update is +/// spelled: erasure applies to deleted documents only, and since erasability +/// itself cannot be withdrawn, such a type could never reach a state erasure +/// acts on again. Keeping the flag makes the type unparseable; dropping it +/// changes an immutable flag. +#[test] +fn should_refuse_to_withdraw_deletion_from_an_erasable_type() { + assert!( + parse_without_validation(erasable_schema(true, false, true)).is_err(), + "an erasable type that forbids deletion is not a valid type at all" + ); + + let old = parse_without_validation(erasable_schema(true, true, true)).unwrap(); + let new = parse_without_validation(repair_schema(true, false)).unwrap(); + let result = old + .as_ref() + .validate_update(new.as_ref(), 2, PlatformVersion::latest()) + .unwrap(); + assert!(!result.is_valid(), "an erasable type kept its delete flag"); +} + +#[test] +fn should_reject_incompatible_properties_during_a_delete_flag_withdrawal() { + let old = parse_at_version(repair_schema(true, true), 13, true).unwrap(); + let new = parse(platform_value!({ + "type": "object", + "properties": { + "label": {"type": "integer", "position": 0}, + }, + "additionalProperties": false, + "documentsKeepHistory": true, + "canBeDeleted": false, + })) + .unwrap(); + let result = old + .as_ref() + .validate_update(new.as_ref(), 2, PlatformVersion::latest()) + .unwrap(); + assert!( + !result.is_valid(), + "the withdrawal must not bypass schema compatibility" + ); +} + +#[test] +fn should_reject_mutability_change_during_a_delete_flag_withdrawal() { + let old = parse_at_version(repair_schema(true, true), 13, true).unwrap(); + let mut schema = repair_schema(true, false); + schema.set_value("documentsMutable", false.into()).unwrap(); + let new = parse(schema).unwrap(); + let result = old + .as_ref() + .validate_update(new.as_ref(), 2, PlatformVersion::latest()) + .unwrap(); + assert!( + !result.is_valid(), + "the withdrawal must not bypass other configuration checks" + ); +} + +#[test] +fn should_still_validate_a_property_named_can_be_deleted_during_a_withdrawal() { + let mut old_schema = repair_schema(true, true); + old_schema + .set_value( + "properties", + platform_value!({ + "canBeDeleted": {"type": "string", "maxLength": 50, "position": 0}, + }), + ) + .unwrap(); + let mut new_schema = old_schema.clone(); + new_schema.set_value("canBeDeleted", false.into()).unwrap(); + new_schema + .set_value( + "properties", + platform_value!({ + "canBeDeleted": {"type": "integer", "position": 0}, + }), + ) + .unwrap(); + let old = parse_at_version(old_schema, 13, true).unwrap(); + let new = parse(new_schema).unwrap(); + let result = old + .as_ref() + .validate_update(new.as_ref(), 2, PlatformVersion::latest()) + .unwrap(); + assert!( + !result.is_valid(), + "only the top-level config flag may be stripped" + ); +} diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v4/mod.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v4/mod.rs new file mode 100644 index 00000000000..e1748a856db --- /dev/null +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v4/mod.rs @@ -0,0 +1,170 @@ +//! Document-type parser **generation 4** — protocol version 15 and later. +//! +//! Generation 4 is generation 3 plus the keep-history document lifecycle: a +//! keep-history type may allow deletion, may additionally declare +//! `canBeErased`, and may not carry a contested index. Its index grammar, the +//! ranked index-key ceilings and the prefix-overlap rule are generation 3's, +//! reused from that module rather than copied. +//! +//! It exists as its own generation — rather than as a version gate inside the +//! shipped ones — because that is what keeps a historical block from ever +//! picking up grammar that did not exist when it was produced: the dispatcher +//! only routes here from `try_from_schema: 4`, and the grammar this module +//! admits is spelled out below as literals rather than looked up in a version +//! table. +//! +//! The parsing steps themselves are shared with the earlier generations in +//! [`super::common`]. + +use crate::data_contract::config::DataContractConfig; +use crate::data_contract::document_type::index::IndexGrammarAdmissions; +use crate::data_contract::document_type::v2::DocumentTypeV2; +use crate::data_contract::document_type::v3::DocumentTypeV3; +use crate::data_contract::document_type::DocumentType; +use crate::data_contract::{TokenConfiguration, TokenContractPosition}; +use crate::validation::operations::ProtocolValidationOperation; +use crate::version::PlatformVersion; +use crate::ProtocolError; +use platform_value::{Identifier, Value}; +use std::collections::BTreeMap; + +use super::common; +use super::v3::ranked_prefix_overlap::validate_no_ranked_prefix_overlap; +use super::v3::RANKED_INDEX_KEY_LENGTH_CHECK; + +/// Parses a document type schema through the generation-4 grammar: the whole +/// generation-3 grammar plus the keep-history document lifecycle keywords. +/// +/// This parser is only reachable from protocol version 15+ (via +/// CONTRACT_VERSIONS_V7). +/// +/// Generation 4 admits `documentsKeepHistory: true` together with +/// `canBeDeleted: true`: a delete on such a type removes the document from +/// ordinary reads while its retained revisions stay readable. It also admits +/// the `canBeErased` keyword, which additionally allows a deleted document's +/// revisions to be purged, and it refuses a keep-history type that carries a +/// contested index. +#[allow(clippy::too_many_arguments)] +fn try_from_schema_generation_4( + data_contract_id: Identifier, + data_contract_system_version: u16, + contract_config_version: u16, + name: &str, + schema: Value, + schema_defs: Option<&BTreeMap>, + token_configurations: &BTreeMap, + data_contact_config: &DataContractConfig, + full_validation: bool, + validation_operations: &mut impl Extend, + platform_version: &PlatformVersion, +) -> Result { + // Read the aggregate and indexOnly keywords before the core parser + // consumes `schema`. + let aggregates = common::parse_doctype_aggregate_keywords(&schema, name)?; + let index_only = common::parse_index_only_keyword(&schema)?; + let can_be_erased = common::parse_can_be_erased_keyword(&schema)?; + + let v1 = common::parse_document_type_core( + data_contract_id, + data_contract_system_version, + contract_config_version, + name, + schema, + schema_defs, + token_configurations, + data_contact_config, + full_validation, + // Lets the core default omitted index terminals to `$ownerId` before + // it builds the index structure, so the structure's level info is + // born normalized (`apply_index_only` below validates the + // already-normalized set). + index_only, + validation_operations, + &common::ParserGeneration { + // Generation 4 exists if and only if `document_type_schema` is 4: + // CONTRACT_VERSIONS_V7 is the only table that selects this parser, + // and it is the only table naming meta-schema v4. So every constant + // here is a property of the generation, not of a platform version, + // and none of them is read out of a table. + document_type_schema_version: 4, + // Meta-schema v4 carries the `keeps*History` flags forward, so they + // are unconditionally part of this generation's grammar. + admit_history: true, + // Count indexes arrived at PV12; every version selecting this + // generation is far past that boundary. + admit_count_indexes: true, + meta_schema_method_name: "DocumentType::try_from_schema_v4 (document_type_schema)", + // The index grammar is generation 3's, read from the shared + // generation → admission mapping so the registration-cost re-parse + // can never drift from what this parser accepts. The ranked key + // ceilings and the prefix-overlap rule are generation 3's as well. + admit_ranked: IndexGrammarAdmissions::for_schema_generation(4).ranked, + ranked_index_key_length_check: RANKED_INDEX_KEY_LENGTH_CHECK, + ranked_index_structure_check: validate_no_ranked_prefix_overlap, + admit_time_range: IndexGrammarAdmissions::for_schema_generation(4).time_range, + admit_index_terminal: IndexGrammarAdmissions::for_schema_generation(4).terminal, + admit_index_preallocated: IndexGrammarAdmissions::for_schema_generation(4).preallocated, + admit_index_skip_if_absent: IndexGrammarAdmissions::for_schema_generation(4) + .skip_if_absent, + }, + platform_version, + )?; + + let mut v2: DocumentTypeV2 = v1.into(); + common::apply_doctype_aggregates(&mut v2, aggregates, name)?; + // After the aggregates: `apply_index_only` rejects the doctype-level + // aggregate flags (they describe the primary-key tree, which an + // indexOnly type does not have), so it has to see them already applied. + common::apply_index_only(&mut v2, index_only, name)?; + // The lifecycle keyword lives on the document type version this + // generation produces. + let mut v3: DocumentTypeV3 = v2.into(); + // Reads `canBeDeleted` off the parsed result rather than the raw schema, so + // it sees the value resolved against the contract config default (`true` + // when the key is omitted). + common::apply_can_be_erased(&mut v3, can_be_erased, name)?; + // A registration-time rule only: a contract that already carries this + // combination was registered under an earlier protocol, and the structural + // parse that loads stored contracts must keep reading it. + if full_validation { + common::reject_contested_keep_history(&v3, name)?; + } + + Ok(v3) +} + +impl DocumentType { + /// Dispatches to this module's generation-4 parser and wraps the result. + #[allow(clippy::too_many_arguments)] + pub(in crate::data_contract::document_type::class_methods) fn try_from_schema_v4( + data_contract_id: Identifier, + data_contract_system_version: u16, + contract_config_version: u16, + name: &str, + schema: Value, + schema_defs: Option<&BTreeMap>, + token_configurations: &BTreeMap, + data_contact_config: &DataContractConfig, + full_validation: bool, + validation_operations: &mut impl Extend, + platform_version: &PlatformVersion, + ) -> Result { + try_from_schema_generation_4( + data_contract_id, + data_contract_system_version, + contract_config_version, + name, + schema, + schema_defs, + token_configurations, + data_contact_config, + full_validation, + validation_operations, + platform_version, + ) + .map(DocumentType::V3) + } +} + +#[cfg(test)] +mod keep_history_tests; diff --git a/packages/rs-dpp/src/data_contract/document_type/methods/mod.rs b/packages/rs-dpp/src/data_contract/document_type/methods/mod.rs index a9cac78317a..54c30c3ff3c 100644 --- a/packages/rs-dpp/src/data_contract/document_type/methods/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/methods/mod.rs @@ -794,6 +794,9 @@ mod tests { crate::data_contract::document_type::DocumentTypeRef::V2(v2) => { v2.requires_revision() } + crate::data_contract::document_type::DocumentTypeRef::V3(v3) => { + v3.requires_revision() + } } } @@ -808,6 +811,9 @@ mod tests { crate::data_contract::document_type::DocumentTypeRef::V2(v2) => { v2.initial_revision() } + crate::data_contract::document_type::DocumentTypeRef::V3(v3) => { + v3.initial_revision() + } } } @@ -822,6 +828,9 @@ mod tests { crate::data_contract::document_type::DocumentTypeRef::V2(v2) => { v2.top_level_indices() } + crate::data_contract::document_type::DocumentTypeRef::V3(v3) => { + v3.top_level_indices() + } } } @@ -836,6 +845,9 @@ mod tests { crate::data_contract::document_type::DocumentTypeRef::V2(v2) => { v2.top_level_indices_of_contested_unique_indexes() } + crate::data_contract::document_type::DocumentTypeRef::V3(v3) => { + v3.top_level_indices_of_contested_unique_indexes() + } } } @@ -851,6 +863,9 @@ mod tests { crate::data_contract::document_type::DocumentTypeRef::V2(v2) => { v2.index_structure() } + crate::data_contract::document_type::DocumentTypeRef::V3(v3) => { + v3.index_structure() + } } } @@ -869,6 +884,9 @@ mod tests { crate::data_contract::document_type::DocumentTypeRef::V2(v2) => { v2.unique_id_for_document_field(index_level, base_event) } + crate::data_contract::document_type::DocumentTypeRef::V3(v3) => { + v3.unique_id_for_document_field(index_level, base_event) + } } } @@ -883,6 +901,9 @@ mod tests { crate::data_contract::document_type::DocumentTypeRef::V2(v2) => { v2.sanitize_document_properties(properties) } + crate::data_contract::document_type::DocumentTypeRef::V3(v3) => { + v3.sanitize_document_properties(properties) + } } } } diff --git a/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/mod.rs b/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/mod.rs index 6e4418eaeeb..dcf484fe480 100644 --- a/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/mod.rs @@ -6,6 +6,7 @@ use platform_version::version::PlatformVersion; mod common; mod v0; mod v1; +mod v2; impl DocumentTypeRef<'_> { /// Verify that the update to the document type is valid. @@ -27,9 +28,10 @@ impl DocumentTypeRef<'_> { { 0 => self.validate_update_v0(new_document_type, platform_version), 1 => self.validate_update_v1(new_document_type, new_contract_version, platform_version), + 2 => self.validate_update_v2(new_document_type, new_contract_version, platform_version), version => Err(ProtocolError::UnknownVersionMismatch { method: "validate_update".to_string(), - known_versions: vec![0, 1], + known_versions: vec![0, 1, 2], received: version, }), } diff --git a/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/v1/mod.rs b/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/v1/mod.rs index 9f8b980f842..ba7ce64a0b0 100644 --- a/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/v1/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/v1/mod.rs @@ -93,7 +93,7 @@ impl DocumentTypeRef<'_> { /// existing properties stay frozen by the schema compatibility differ; /// this check judges the top-level `required` key, which is stripped /// from the diff exactly like `indices`. - fn validate_required_fields_update( + pub(super) fn validate_required_fields_update( &self, new_document_type: DocumentTypeRef, new_contract_version: u32, @@ -171,7 +171,7 @@ impl DocumentTypeRef<'_> { /// on-disk subtrees. Compare the definitions by index name — the /// comparison must not depend on where a changed index's name sorts /// relative to the document type's other indexes. - fn validate_index_definitions_unchanged( + pub(super) fn validate_index_definitions_unchanged( &self, new_document_type: DocumentTypeRef, ) -> SimpleConsensusValidationResult { diff --git a/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/v2/mod.rs b/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/v2/mod.rs new file mode 100644 index 00000000000..3e196e689d5 --- /dev/null +++ b/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/v2/mod.rs @@ -0,0 +1,69 @@ +//! Protocol v15 generation of document type update validation. +//! +//! v2 is v1 plus the keep-history document lifecycle. Two rules are new: +//! +//! * `canBeErased` is immutable in both directions. Widening it would give an +//! operation over already-stored revisions to a type registered without it; +//! narrowing it after a first erase chunk has irreversibly removed revisions +//! would strand a partially erased, invisible document forever. +//! * A keep-history type may withdraw deletion (`canBeDeleted: true -> false`) +//! while keeping history, and only in that direction, and only while it is +//! not erasable: erase applies to deleted documents only and erasability is +//! itself immutable, so an erasable type that stopped allowing deletion could +//! never again reach a state erase acts on. +//! +//! Every other check is v1's, which v2 delegates to once the two lifecycle +//! rules above have passed. The one case where v1 alone would decide +//! differently, an erasable type withdrawing deletion, is refused here before +//! v1 sees it, so v1's option computation stays what it was. + +use crate::consensus::state::data_contract::document_type_update_error::DocumentTypeUpdateError; +use crate::data_contract::document_type::accessors::{ + DocumentTypeV0Getters, DocumentTypeV3Getters, +}; +use crate::data_contract::document_type::DocumentTypeRef; +use crate::validation::SimpleConsensusValidationResult; +use crate::ProtocolError; +use platform_version::version::PlatformVersion; + +impl DocumentTypeRef<'_> { + #[inline(always)] + pub(super) fn validate_update_v2( + &self, + new_document_type: DocumentTypeRef, + new_contract_version: u32, + platform_version: &PlatformVersion, + ) -> Result { + if new_document_type.documents_can_be_erased() != self.documents_can_be_erased() { + return Ok(SimpleConsensusValidationResult::new_with_error( + DocumentTypeUpdateError::new( + self.data_contract_id(), + self.name(), + format!( + "document type can not change whether its documents can be erased: changing from {} to {}", + self.documents_can_be_erased(), + new_document_type.documents_can_be_erased() + ), + ) + .into(), + )); + } + + if self.documents_can_be_erased() + && self.documents_can_be_deleted() + && !new_document_type.documents_can_be_deleted() + { + return Ok(SimpleConsensusValidationResult::new_with_error( + DocumentTypeUpdateError::new( + self.data_contract_id(), + self.name(), + "document type whose documents can be erased can not stop allowing deletion" + .to_string(), + ) + .into(), + )); + } + + self.validate_update_v1(new_document_type, new_contract_version, platform_version) + } +} diff --git a/packages/rs-dpp/src/data_contract/document_type/methods/versioned_methods.rs b/packages/rs-dpp/src/data_contract/document_type/methods/versioned_methods.rs index c1b95d40cdc..76d59f2b210 100644 --- a/packages/rs-dpp/src/data_contract/document_type/methods/versioned_methods.rs +++ b/packages/rs-dpp/src/data_contract/document_type/methods/versioned_methods.rs @@ -3,6 +3,7 @@ use crate::data_contract::document_type::methods::DocumentTypeBasicMethods; use crate::data_contract::document_type::v0::DocumentTypeV0; use crate::data_contract::document_type::v1::DocumentTypeV1; use crate::data_contract::document_type::v2::DocumentTypeV2; +use crate::data_contract::document_type::v3::DocumentTypeV3; use crate::data_contract::document_type::{ DocumentPropertyType, DocumentType, DocumentTypeRef, Index, CONTRACT_VERSION_STAMP_MAX_SIZE, DEFAULT_HASH_SIZE, MAX_INDEX_SIZE, @@ -780,6 +781,7 @@ impl DocumentTypeV0MethodsVersioned for DocumentTypeV0 {} impl DocumentTypeV0MethodsVersioned for DocumentTypeV1 {} impl DocumentTypeV0MethodsVersioned for DocumentTypeV2 {} +impl DocumentTypeV0MethodsVersioned for DocumentTypeV3 {} impl DocumentTypeV0MethodsVersioned for DocumentType {} impl DocumentTypeV0MethodsVersioned for DocumentTypeRef<'_> {} diff --git a/packages/rs-dpp/src/data_contract/document_type/mod.rs b/packages/rs-dpp/src/data_contract/document_type/mod.rs index 3f9ca299a6c..289324e9ac4 100644 --- a/packages/rs-dpp/src/data_contract/document_type/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/mod.rs @@ -21,6 +21,7 @@ pub(crate) use validate_required_since_within_contract_version::validate_require pub mod v0; pub mod v1; pub mod v2; +pub mod v3; #[cfg(feature = "validation")] pub(crate) mod validator; @@ -30,6 +31,7 @@ use crate::data_contract::document_type::methods::{ use crate::data_contract::document_type::v0::DocumentTypeV0; use crate::data_contract::document_type::v1::DocumentTypeV1; use crate::data_contract::document_type::v2::DocumentTypeV2; +use crate::data_contract::document_type::v3::DocumentTypeV3; use crate::document::Document; use crate::fee::Credits; use crate::version::PlatformVersion; @@ -126,6 +128,12 @@ pub(crate) mod property_names { /// 14). See `apply_index_only` in `try_from_schema::common` for the /// structural constraints the flag imposes. pub const INDEX_ONLY: &str = "indexOnly"; + /// Doctype-level flag declaring that a deleted document of this type may + /// have its retained revisions purged by an erase transition. Requires + /// `documentsKeepHistory: true` and `canBeDeleted: true`, defaults to + /// false, and is immutable across contract updates. Meta-schema v4+ + /// (protocol version 15). + pub const CAN_BE_ERASED: &str = "canBeErased"; } #[derive(Clone, Copy, Debug, PartialEq)] @@ -133,6 +141,7 @@ pub enum DocumentTypeRef<'a> { V0(&'a DocumentTypeV0), V1(&'a DocumentTypeV1), V2(&'a DocumentTypeV2), + V3(&'a DocumentTypeV3), } #[derive(Debug)] @@ -140,6 +149,7 @@ pub enum DocumentTypeMutRef<'a> { V0(&'a mut DocumentTypeV0), V1(&'a mut DocumentTypeV1), V2(&'a mut DocumentTypeV2), + V3(&'a mut DocumentTypeV3), } #[allow(clippy::large_enum_variant)] @@ -148,6 +158,7 @@ pub enum DocumentType { V0(DocumentTypeV0), V1(DocumentTypeV1), V2(DocumentTypeV2), + V3(DocumentTypeV3), } impl DocumentType { @@ -156,6 +167,7 @@ impl DocumentType { DocumentType::V0(v0) => DocumentTypeRef::V0(v0), DocumentType::V1(v1) => DocumentTypeRef::V1(v1), DocumentType::V2(v2) => DocumentTypeRef::V2(v2), + DocumentType::V3(v3) => DocumentTypeRef::V3(v3), } } @@ -164,6 +176,7 @@ impl DocumentType { DocumentType::V0(v0) => DocumentTypeMutRef::V0(v0), DocumentType::V1(v1) => DocumentTypeMutRef::V1(v1), DocumentType::V2(v2) => DocumentTypeMutRef::V2(v2), + DocumentType::V3(v3) => DocumentTypeMutRef::V3(v3), } } @@ -182,6 +195,9 @@ impl DocumentType { DocumentType::V2(v2) => { v2.prefunded_voting_balance_for_document(document, platform_version) } + DocumentType::V3(v3) => { + v3.prefunded_voting_balance_for_document(document, platform_version) + } } } } @@ -192,6 +208,7 @@ impl DocumentTypeRef<'_> { DocumentTypeRef::V0(v0) => DocumentType::V0((*v0).to_owned()), DocumentTypeRef::V1(v1) => DocumentType::V1((*v1).to_owned()), DocumentTypeRef::V2(v2) => DocumentType::V2((*v2).to_owned()), + DocumentTypeRef::V3(v3) => DocumentType::V3((*v3).to_owned()), } } } diff --git a/packages/rs-dpp/src/data_contract/document_type/v1/mod.rs b/packages/rs-dpp/src/data_contract/document_type/v1/mod.rs index 5c4b95d1922..bcd7c487f75 100644 --- a/packages/rs-dpp/src/data_contract/document_type/v1/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/v1/mod.rs @@ -198,7 +198,8 @@ mod tests { match dt { crate::data_contract::document_type::DocumentType::V0(v0) => v0, crate::data_contract::document_type::DocumentType::V1(_) - | crate::data_contract::document_type::DocumentType::V2(_) => { + | crate::data_contract::document_type::DocumentType::V2(_) + | crate::data_contract::document_type::DocumentType::V3(_) => { panic!("expected V0 from first() version routing") } } diff --git a/packages/rs-dpp/src/data_contract/document_type/v3/accessors.rs b/packages/rs-dpp/src/data_contract/document_type/v3/accessors.rs new file mode 100644 index 00000000000..55f07700155 --- /dev/null +++ b/packages/rs-dpp/src/data_contract/document_type/v3/accessors.rs @@ -0,0 +1,274 @@ +use crate::data_contract::document_type::accessors::{ + DocumentTypeV0Getters, DocumentTypeV0MutGetters, DocumentTypeV0Setters, DocumentTypeV1Getters, + DocumentTypeV2Getters, DocumentTypeV2Setters, DocumentTypeV3Getters, +}; +use crate::data_contract::document_type::index::Index; +use crate::data_contract::document_type::index_level::IndexLevel; +use crate::data_contract::document_type::property::DocumentProperty; + +use platform_value::{Identifier, Value}; + +use crate::data_contract::document_type::restricted_creation::CreationRestrictionMode; +use crate::data_contract::document_type::token_costs::accessors::TokenCostGettersV0; +use crate::data_contract::document_type::v3::DocumentTypeV3; +#[cfg(feature = "validation")] +use crate::data_contract::document_type::validator::StatelessJsonSchemaLazyValidator; +use crate::data_contract::storage_requirements::keys_for_document_type::StorageKeyRequirements; +use crate::data_contract::TokenContractPosition; +use crate::document::transfer::Transferable; +use crate::identity::SecurityLevel; +use crate::nft::TradeMode; +use crate::tokens::token_amount_on_contract_token::DocumentActionTokenCost; +use indexmap::IndexMap; +use std::collections::{BTreeMap, BTreeSet}; + +impl DocumentTypeV0MutGetters for DocumentTypeV3 { + fn schema_mut(&mut self) -> &mut Value { + &mut self.schema + } +} + +impl DocumentTypeV0Getters for DocumentTypeV3 { + fn name(&self) -> &String { + &self.name + } + + fn schema(&self) -> &Value { + &self.schema + } + + fn schema_owned(self) -> Value { + self.schema + } + + fn indexes(&self) -> &BTreeMap { + &self.indices + } + + fn find_contested_index(&self) -> Option<&Index> { + self.indices + .iter() + .find(|(_, index)| index.contested_index.is_some()) + .map(|(_, contested_index)| contested_index) + } + + fn index_structure(&self) -> &IndexLevel { + &self.index_structure + } + + fn flattened_properties(&self) -> &IndexMap { + &self.flattened_properties + } + + fn properties(&self) -> &IndexMap { + &self.properties + } + + fn identifier_paths(&self) -> &BTreeSet { + &self.identifier_paths + } + + fn binary_paths(&self) -> &BTreeSet { + &self.binary_paths + } + + fn required_fields(&self) -> &BTreeSet { + &self.required_fields + } + fn transient_fields(&self) -> &BTreeSet { + &self.transient_fields + } + + fn documents_keep_history(&self) -> bool { + self.documents_keep_history + } + + fn documents_keep_transfer_history(&self) -> bool { + self.documents_keep_transfer_history + } + + fn documents_keep_purchase_history(&self) -> bool { + self.documents_keep_purchase_history + } + + fn documents_keep_pricing_history(&self) -> bool { + self.documents_keep_pricing_history + } + + fn documents_mutable(&self) -> bool { + self.documents_mutable + } + + fn documents_can_be_deleted(&self) -> bool { + self.documents_can_be_deleted + } + + fn documents_transferable(&self) -> Transferable { + self.documents_transferable + } + + fn trade_mode(&self) -> TradeMode { + self.trade_mode + } + + fn creation_restriction_mode(&self) -> CreationRestrictionMode { + self.creation_restriction_mode + } + + fn data_contract_id(&self) -> Identifier { + self.data_contract_id + } + + fn requires_identity_encryption_bounded_key(&self) -> Option { + self.requires_identity_encryption_bounded_key + } + + fn requires_identity_decryption_bounded_key(&self) -> Option { + self.requires_identity_decryption_bounded_key + } + + fn security_level_requirement(&self) -> SecurityLevel { + self.security_level_requirement + } + + #[cfg(feature = "validation")] + fn json_schema_validator_ref(&self) -> &StatelessJsonSchemaLazyValidator { + &self.json_schema_validator + } +} + +impl DocumentTypeV0Setters for DocumentTypeV3 { + fn set_data_contract_id(&mut self, data_contract_id: Identifier) { + self.data_contract_id = data_contract_id; + } +} + +impl DocumentTypeV1Getters for DocumentTypeV3 { + fn document_creation_token_cost(&self) -> Option { + self.token_costs.document_creation_token_cost() + } + + fn document_replacement_token_cost(&self) -> Option { + self.token_costs.document_replacement_token_cost() + } + + fn document_deletion_token_cost(&self) -> Option { + self.token_costs.document_deletion_token_cost() + } + + fn document_transfer_token_cost(&self) -> Option { + self.token_costs.document_transfer_token_cost() + } + + fn document_update_price_token_cost(&self) -> Option { + self.token_costs.document_price_update_token_cost() + } + + fn document_purchase_token_cost(&self) -> Option { + self.token_costs.document_purchase_token_cost() + } + + fn all_document_token_costs(&self) -> Vec<&DocumentActionTokenCost> { + let mut result = Vec::new(); + + if let Some(cost) = self.token_costs.document_creation_token_cost_ref() { + result.push(cost); + } + if let Some(cost) = self.token_costs.document_replacement_token_cost_ref() { + result.push(cost); + } + if let Some(cost) = self.token_costs.document_deletion_token_cost_ref() { + result.push(cost); + } + if let Some(cost) = self.token_costs.document_transfer_token_cost_ref() { + result.push(cost); + } + if let Some(cost) = self.token_costs.document_price_update_token_cost_ref() { + result.push(cost); + } + if let Some(cost) = self.token_costs.document_purchase_token_cost_ref() { + result.push(cost); + } + + result + } + + fn all_external_token_costs_contract_tokens( + &self, + ) -> BTreeMap> { + let mut map = BTreeMap::new(); + + for cost in self.all_document_token_costs() { + if let Some(contract_id) = cost.contract_id { + map.entry(contract_id) + .or_insert_with(BTreeSet::new) + .insert(cost.token_contract_position); + } + } + + map + } +} + +impl DocumentTypeV2Getters for DocumentTypeV3 { + fn documents_countable(&self) -> bool { + self.documents_countable || self.range_countable + } + + fn range_countable(&self) -> bool { + self.range_countable + } + + fn documents_summable(&self) -> Option<&str> { + self.documents_summable.as_deref() + } + + fn range_summable(&self) -> bool { + self.range_summable + } + + fn index_only(&self) -> bool { + self.index_only + } +} + +impl DocumentTypeV2Setters for DocumentTypeV3 { + fn set_documents_countable(&mut self, countable: bool) { + self.documents_countable = countable; + if !countable { + // Preserve invariant: range_countable implies documents_countable + self.range_countable = false; + } + } + + fn set_range_countable(&mut self, range_countable: bool) { + self.range_countable = range_countable; + if range_countable { + self.documents_countable = true; + } + } + + fn set_documents_summable(&mut self, property: Option) { + let cleared = property.is_none(); + self.documents_summable = property; + if cleared { + // Preserve invariant: range_summable requires + // documents_summable.is_some() + self.range_summable = false; + } + } + + fn set_range_summable(&mut self, range_summable: bool) { + // Normalize unconditionally: `range_summable` requires a property + // to sum on, so clamp to false when `documents_summable` is unset. + // This way an existing-true-but-inconsistent state can't survive + // a setter call — the invariant always holds after this returns. + self.range_summable = range_summable && self.documents_summable.is_some(); + } +} + +impl DocumentTypeV3Getters for DocumentTypeV3 { + fn documents_can_be_erased(&self) -> bool { + self.documents_can_be_erased + } +} diff --git a/packages/rs-dpp/src/data_contract/document_type/v3/mod.rs b/packages/rs-dpp/src/data_contract/document_type/v3/mod.rs new file mode 100644 index 00000000000..2797e0d5808 --- /dev/null +++ b/packages/rs-dpp/src/data_contract/document_type/v3/mod.rs @@ -0,0 +1,350 @@ +use indexmap::IndexMap; +use std::collections::{BTreeMap, BTreeSet}; + +use crate::data_contract::document_type::index::Index; +use crate::data_contract::document_type::index_level::IndexLevel; +use crate::data_contract::document_type::property::DocumentProperty; +use crate::data_contract::storage_requirements::keys_for_document_type::StorageKeyRequirements; + +use crate::data_contract::document_type::methods::{ + DocumentTypeBasicMethods, DocumentTypeV0Methods, +}; +use crate::data_contract::document_type::restricted_creation::CreationRestrictionMode; +use crate::data_contract::document_type::token_costs::accessors::TokenCostSettersV0; +use crate::data_contract::document_type::token_costs::TokenCosts; +use crate::data_contract::document_type::v0::DocumentTypeV0; +use crate::data_contract::document_type::v1::DocumentTypeV1; +use crate::data_contract::document_type::v2::DocumentTypeV2; +#[cfg(feature = "validation")] +use crate::data_contract::document_type::validator::StatelessJsonSchemaLazyValidator; +use crate::document::transfer::Transferable; +use crate::identity::SecurityLevel; +use crate::nft::TradeMode; +use crate::tokens::token_amount_on_contract_token::DocumentActionTokenCost; +use platform_value::{Identifier, Value}; + +mod accessors; +#[cfg(feature = "random-document-types")] +pub mod random_document_type; + +/// The document type of protocol version 15 and later: [`DocumentTypeV2`] plus +/// the `canBeErased` keyword of meta-schema v4. Parser generation 4 produces it; +/// earlier generations keep producing the version their grammar knows. +#[derive(Debug, PartialEq, Clone)] +pub struct DocumentTypeV3 { + pub(in crate::data_contract) name: String, + pub(in crate::data_contract) schema: Value, + pub(in crate::data_contract) indices: BTreeMap, + pub(in crate::data_contract) index_structure: IndexLevel, + /// Flattened properties flatten all objects for quick lookups for indexes + /// Document field should not contain sub objects. + pub(in crate::data_contract) flattened_properties: IndexMap, + /// Document field can contain sub objects. + pub(in crate::data_contract) properties: IndexMap, + pub(in crate::data_contract) identifier_paths: BTreeSet, + pub(in crate::data_contract) binary_paths: BTreeSet, + /// The required fields on the document type + pub(in crate::data_contract) required_fields: BTreeSet, + /// The transient fields on the document type + pub(in crate::data_contract) transient_fields: BTreeSet, + /// Should documents keep history? + pub(in crate::data_contract) documents_keep_history: bool, + /// Should transfers of documents of this type be recorded in the document + /// history system contract? + pub(in crate::data_contract) documents_keep_transfer_history: bool, + /// Should purchases of documents of this type be recorded in the document + /// history system contract? + pub(in crate::data_contract) documents_keep_purchase_history: bool, + /// Should price updates on documents of this type be recorded in the + /// document history system contract? + pub(in crate::data_contract) documents_keep_pricing_history: bool, + /// Are documents mutable? + pub(in crate::data_contract) documents_mutable: bool, + /// Can documents of this type be deleted? + pub(in crate::data_contract) documents_can_be_deleted: bool, + /// Can documents be transferred without a trade? + pub(in crate::data_contract) documents_transferable: Transferable, + /// How are these documents traded? + pub(in crate::data_contract) trade_mode: TradeMode, + /// Is document creation restricted? + pub(in crate::data_contract) creation_restriction_mode: CreationRestrictionMode, + /// The data contract id + pub(in crate::data_contract) data_contract_id: Identifier, + /// Encryption key storage requirements + pub(in crate::data_contract) requires_identity_encryption_bounded_key: + Option, + /// Decryption key storage requirements + pub(in crate::data_contract) requires_identity_decryption_bounded_key: + Option, + pub(in crate::data_contract) security_level_requirement: SecurityLevel, + #[cfg(feature = "validation")] + pub(in crate::data_contract) json_schema_validator: StatelessJsonSchemaLazyValidator, + /// The token costs associated with state transitions on this document type + pub(in crate::data_contract) token_costs: TokenCosts, + /// When true, the primary key tree uses a CountTree enabling O(1) total document count queries + pub(in crate::data_contract) documents_countable: bool, + /// When true, the primary key tree uses a ProvableCountTree enabling range countable. + /// Implies documents_countable = true. + pub(in crate::data_contract) range_countable: bool, + /// When `Some(property_name)`, the primary key tree is a `SumTree` (or + /// `ProvableSumTree` if [`Self::range_summable`] is also set) summing + /// the named integer property across every document of this type. + /// Enables O(log n) `GetDocumentsSum` queries with no `where` filter. + /// + /// The named property must be `type: integer` and listed in + /// [`Self::required_fields`]; the parser enforces this at contract + /// creation. Composes orthogonally with `documents_countable` — + /// setting both yields a `CountSumTree` (or `ProvableCountSumTree`) + /// that carries both a count and a sum, queryable independently. + pub(in crate::data_contract) documents_summable: Option, + /// When true, the primary key sum tree is a `ProvableSumTree` + /// (committing aggregated sub-sums to every internal merk node), + /// enabling O(log n) `AggregateSumOnRange` queries. Implies + /// [`Self::documents_summable`] is `Some` — enforced by + /// [`crate::data_contract::document_type::accessors::DocumentTypeV2Setters::set_range_summable`]. + pub(in crate::data_contract) range_summable: bool, + /// When true, documents of this type are **indexOnly**: nothing is + /// written to primary storage (there is no `[0]` primary-key tree at + /// all) — the index entries are the rows, each terminating in an `Item` + /// keyed by the index's `terminal` property instead of a `Reference` + /// keyed by the document id. Only what is in the indexes exists and is + /// recoverable. The parser (`apply_index_only`) enforces the structural + /// constraints this layout depends on: every property required and + /// indexed, `$ownerId` recoverable from at least one index, immutable / + /// non-transferable / no history, and per-index terminal typing. + pub(in crate::data_contract) index_only: bool, + /// When true, a deleted document of this type may have its retained + /// revisions purged by an erase transition. Requires + /// [`Self::documents_keep_history`] and [`Self::documents_can_be_deleted`]: + /// erase applies to deleted documents only, so an erase-only type would + /// have nothing to act on. The parser enforces the pairing at contract + /// registration and the update validator keeps the flag immutable, because + /// narrowing it after a first chunk has irreversibly removed revisions + /// would strand a partially erased document forever. + pub(in crate::data_contract) documents_can_be_erased: bool, +} + +impl DocumentTypeBasicMethods for DocumentTypeV3 {} + +impl DocumentTypeV0Methods for DocumentTypeV3 {} + +impl crate::data_contract::document_type::accessors::DocumentTypeV1Setters for DocumentTypeV3 { + fn set_document_creation_token_cost(&mut self, cost: Option) { + self.token_costs.set_document_creation_token_cost(cost) + } + + fn set_document_replacement_token_cost(&mut self, cost: Option) { + self.token_costs.set_document_replacement_token_cost(cost) + } + + fn set_document_deletion_token_cost(&mut self, cost: Option) { + self.token_costs.set_document_deletion_token_cost(cost) + } + + fn set_document_transfer_token_cost(&mut self, cost: Option) { + self.token_costs.set_document_transfer_token_cost(cost) + } + + fn set_document_price_update_token_cost(&mut self, cost: Option) { + self.token_costs.set_document_price_update_token_cost(cost) + } + + fn set_document_purchase_token_cost(&mut self, cost: Option) { + self.token_costs.set_document_purchase_token_cost(cost) + } +} + +impl From for DocumentTypeV3 { + fn from(value: DocumentTypeV0) -> Self { + DocumentTypeV3 { + name: value.name, + schema: value.schema, + indices: value.indices, + index_structure: value.index_structure, + flattened_properties: value.flattened_properties, + properties: value.properties, + identifier_paths: value.identifier_paths, + binary_paths: value.binary_paths, + required_fields: value.required_fields, + transient_fields: value.transient_fields, + documents_keep_history: value.documents_keep_history, + documents_keep_transfer_history: value.documents_keep_transfer_history, + documents_keep_purchase_history: value.documents_keep_purchase_history, + documents_keep_pricing_history: value.documents_keep_pricing_history, + documents_mutable: value.documents_mutable, + documents_can_be_deleted: value.documents_can_be_deleted, + documents_transferable: value.documents_transferable, + trade_mode: value.trade_mode, + creation_restriction_mode: value.creation_restriction_mode, + data_contract_id: value.data_contract_id, + requires_identity_encryption_bounded_key: value + .requires_identity_encryption_bounded_key, + requires_identity_decryption_bounded_key: value + .requires_identity_decryption_bounded_key, + security_level_requirement: value.security_level_requirement, + #[cfg(feature = "validation")] + json_schema_validator: value.json_schema_validator, + token_costs: TokenCosts::V0(Default::default()), + documents_countable: false, + range_countable: false, + documents_summable: None, + range_summable: false, + index_only: false, + documents_can_be_erased: false, + } + } +} + +impl From for DocumentTypeV3 { + fn from(value: DocumentTypeV1) -> Self { + DocumentTypeV3 { + name: value.name, + schema: value.schema, + indices: value.indices, + index_structure: value.index_structure, + flattened_properties: value.flattened_properties, + properties: value.properties, + identifier_paths: value.identifier_paths, + binary_paths: value.binary_paths, + required_fields: value.required_fields, + transient_fields: value.transient_fields, + documents_keep_history: value.documents_keep_history, + documents_keep_transfer_history: value.documents_keep_transfer_history, + documents_keep_purchase_history: value.documents_keep_purchase_history, + documents_keep_pricing_history: value.documents_keep_pricing_history, + documents_mutable: value.documents_mutable, + documents_can_be_deleted: value.documents_can_be_deleted, + documents_transferable: value.documents_transferable, + trade_mode: value.trade_mode, + creation_restriction_mode: value.creation_restriction_mode, + data_contract_id: value.data_contract_id, + requires_identity_encryption_bounded_key: value + .requires_identity_encryption_bounded_key, + requires_identity_decryption_bounded_key: value + .requires_identity_decryption_bounded_key, + security_level_requirement: value.security_level_requirement, + #[cfg(feature = "validation")] + json_schema_validator: value.json_schema_validator, + token_costs: value.token_costs, + documents_countable: false, + range_countable: false, + documents_summable: None, + range_summable: false, + index_only: false, + documents_can_be_erased: false, + } + } +} + +impl From for DocumentTypeV3 { + fn from(value: DocumentTypeV2) -> Self { + DocumentTypeV3 { + name: value.name, + schema: value.schema, + indices: value.indices, + index_structure: value.index_structure, + flattened_properties: value.flattened_properties, + properties: value.properties, + identifier_paths: value.identifier_paths, + binary_paths: value.binary_paths, + required_fields: value.required_fields, + transient_fields: value.transient_fields, + documents_keep_history: value.documents_keep_history, + documents_keep_transfer_history: value.documents_keep_transfer_history, + documents_keep_purchase_history: value.documents_keep_purchase_history, + documents_keep_pricing_history: value.documents_keep_pricing_history, + documents_mutable: value.documents_mutable, + documents_can_be_deleted: value.documents_can_be_deleted, + documents_transferable: value.documents_transferable, + trade_mode: value.trade_mode, + creation_restriction_mode: value.creation_restriction_mode, + data_contract_id: value.data_contract_id, + requires_identity_encryption_bounded_key: value + .requires_identity_encryption_bounded_key, + requires_identity_decryption_bounded_key: value + .requires_identity_decryption_bounded_key, + security_level_requirement: value.security_level_requirement, + #[cfg(feature = "validation")] + json_schema_validator: value.json_schema_validator, + token_costs: value.token_costs, + documents_countable: value.documents_countable, + range_countable: value.range_countable, + documents_summable: value.documents_summable, + range_summable: value.range_summable, + index_only: value.index_only, + documents_can_be_erased: false, + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::data_contract::document_type::accessors::{ + DocumentTypeV0Getters, DocumentTypeV2Getters, DocumentTypeV3Getters, + }; + use crate::data_contract::document_type::v0::DocumentTypeV0; + use crate::data_contract::document_type::DocumentType; + + fn make_v0() -> DocumentTypeV0 { + DocumentTypeV0 { + name: "test".to_string(), + schema: Value::Null, + indices: BTreeMap::new(), + index_structure: IndexLevel::try_from_indices( + Vec::::new(), + "test", + platform_version::version::PlatformVersion::latest(), + ) + .unwrap(), + flattened_properties: IndexMap::new(), + properties: IndexMap::new(), + identifier_paths: BTreeSet::new(), + binary_paths: BTreeSet::new(), + required_fields: BTreeSet::new(), + transient_fields: BTreeSet::new(), + documents_keep_history: true, + documents_keep_transfer_history: false, + documents_keep_purchase_history: false, + documents_keep_pricing_history: false, + documents_mutable: true, + documents_can_be_deleted: true, + documents_transferable: Transferable::Never, + trade_mode: TradeMode::None, + creation_restriction_mode: CreationRestrictionMode::NoRestrictions, + data_contract_id: Identifier::default(), + requires_identity_encryption_bounded_key: None, + requires_identity_decryption_bounded_key: None, + security_level_requirement: SecurityLevel::HIGH, + #[cfg(feature = "validation")] + json_schema_validator: Default::default(), + } + } + + #[test] + fn should_keep_every_earlier_field_and_default_erasability_to_false() { + let mut v2: DocumentTypeV2 = make_v0().into(); + v2.index_only = false; + v2.documents_countable = true; + let v3: DocumentTypeV3 = v2.into(); + + assert_eq!(v3.name(), "test"); + assert!(v3.documents_keep_history()); + assert!(v3.documents_can_be_deleted()); + assert!(v3.documents_countable()); + assert!( + !v3.documents_can_be_erased(), + "a type parsed by an earlier grammar never declared the keyword" + ); + } + + #[test] + fn should_report_erasability_only_from_the_version_that_knows_it() { + let mut v3: DocumentTypeV3 = make_v0().into(); + v3.documents_can_be_erased = true; + + assert!(DocumentType::V3(v3).documents_can_be_erased()); + assert!(!DocumentType::V0(make_v0()).documents_can_be_erased()); + assert!(!DocumentType::V2(make_v0().into()).documents_can_be_erased()); + } +} diff --git a/packages/rs-dpp/src/data_contract/document_type/v3/random_document_type.rs b/packages/rs-dpp/src/data_contract/document_type/v3/random_document_type.rs new file mode 100644 index 00000000000..9a12470f94c --- /dev/null +++ b/packages/rs-dpp/src/data_contract/document_type/v3/random_document_type.rs @@ -0,0 +1,40 @@ +use crate::data_contract::document_type::v0::random_document_type::RandomDocumentTypeParameters; +use crate::data_contract::document_type::v1::DocumentTypeV1; +use crate::data_contract::document_type::v3::DocumentTypeV3; +use crate::version::PlatformVersion; +use crate::ProtocolError; +use platform_value::Identifier; +use rand::rngs::StdRng; + +impl DocumentTypeV3 { + pub fn random_document_type( + parameters: RandomDocumentTypeParameters, + data_contract_id: Identifier, + rng: &mut StdRng, + platform_version: &PlatformVersion, + ) -> Result { + Ok(DocumentTypeV1::random_document_type( + parameters, + data_contract_id, + rng, + platform_version, + )? + .into()) + } + + /// This is used to create an invalid random document type, often for testing + pub fn invalid_random_document_type( + parameters: RandomDocumentTypeParameters, + data_contract_id: Identifier, + rng: &mut StdRng, + platform_version: &PlatformVersion, + ) -> Result { + Ok(DocumentTypeV1::invalid_random_document_type( + parameters, + data_contract_id, + rng, + platform_version, + )? + .into()) + } +} diff --git a/packages/rs-dpp/src/data_contract/methods/validate_update/v0/mod.rs b/packages/rs-dpp/src/data_contract/methods/validate_update/v0/mod.rs index 6cea9ac97b6..1e21df5ba69 100644 --- a/packages/rs-dpp/src/data_contract/methods/validate_update/v0/mod.rs +++ b/packages/rs-dpp/src/data_contract/methods/validate_update/v0/mod.rs @@ -237,6 +237,7 @@ mod tests { DocumentTypeMutRef::V0(dt) => dt.documents_mutable = false, DocumentTypeMutRef::V1(dt) => dt.documents_mutable = false, DocumentTypeMutRef::V2(dt) => dt.documents_mutable = false, + DocumentTypeMutRef::V3(dt) => dt.documents_mutable = false, } let result = old_data_contract diff --git a/packages/rs-dpp/src/document/lifecycle/mod.rs b/packages/rs-dpp/src/document/lifecycle/mod.rs new file mode 100644 index 00000000000..84cccde5177 --- /dev/null +++ b/packages/rs-dpp/src/document/lifecycle/mod.rs @@ -0,0 +1,209 @@ +//! The lifecycle record of a keep-history document. +//! +//! A keep-history document that has been deleted keeps its retained revisions +//! but loses its current pointer and its index references, so nothing in the +//! primary-key tree distinguishes it from a document that never existed. The +//! record supplies that distinction: it exists exactly while the document is +//! deleted or erasing, names the block time it was deleted at, and, once an +//! erasure has been authorized, the block time that erasure started and the +//! revision it started from. Drive stores it beside the document's history and +//! proof verifiers decode it from a proof. + +use crate::ProtocolError; +use bincode::{Decode, DecodeUntrusted, Encode}; +use derive_more::From; +use platform_serialization_derive::{ + PlatformDeserializeTrusted, PlatformDeserializeUntrusted, PlatformSerialize, +}; +use platform_versioning::PlatformVersioned; + +pub mod v0; + +pub use v0::DocumentLifecycleRecordV0; + +/// Largest number of bytes an encoded record can occupy: the format +/// discriminant and six variable-length `u64`s at their widest. Estimates +/// that have to price a record without reading it use this bound. +pub const DOCUMENT_LIFECYCLE_RECORD_MAX_SIZE: u32 = 1 + 6 * 9; + +/// A lifecycle record in whichever layout it was written with. +#[derive( + Debug, + Clone, + Copy, + Encode, + Decode, + DecodeUntrusted, + PlatformDeserializeTrusted, + PlatformDeserializeUntrusted, + PlatformSerialize, + PlatformVersioned, + From, + PartialEq, + Eq, +)] +#[cfg_attr( + feature = "serde-conversion", + derive(serde::Serialize, serde::Deserialize), + serde(tag = "$formatVersion") +)] +#[platform_serialize(unversioned)] +pub enum DocumentLifecycleRecord { + /// The original layout. + #[cfg_attr(feature = "serde-conversion", serde(rename = "0"))] + V0(DocumentLifecycleRecordV0), +} + +impl DocumentLifecycleRecord { + /// Builds the record a delete writes: the deletion time, what the deleted + /// document's revision was and how many revisions its history retained, + /// and no erasure. + pub fn deleted_at(deleted_at_ms: u64, latest_revision: u64, revision_count: u64) -> Self { + DocumentLifecycleRecord::V0(DocumentLifecycleRecordV0 { + deleted_at_ms, + latest_revision, + revision_count, + ..Default::default() + }) + } + + /// Block time the document was deleted at. + pub fn deleted_at_ms(&self) -> u64 { + match self { + DocumentLifecycleRecord::V0(v0) => v0.deleted_at_ms, + } + } + + /// Revision the document carried when it was deleted. + pub fn latest_revision(&self) -> u64 { + match self { + DocumentLifecycleRecord::V0(v0) => v0.latest_revision, + } + } + + /// Number of revisions the history retained when the document was deleted. + pub fn revision_count(&self) -> u64 { + match self { + DocumentLifecycleRecord::V0(v0) => v0.revision_count, + } + } + + /// Whether the revisions retained at deletion numbered one through the + /// deleted revision without a gap, which is what lets a by-revision read + /// map a revision onto a position in the history. + /// + /// A gap can only come from a history written before protocol 15, where + /// two writes in one block overwrote each other's revision; an erase + /// removes the newest revisions first and so never opens one. + pub fn revisions_are_contiguous(&self) -> bool { + self.latest_revision() == self.revision_count() + } + + /// Block time an authorized erasure started at, or zero while none has. + pub fn erasing_started_at_ms(&self) -> u64 { + match self { + DocumentLifecycleRecord::V0(v0) => v0.erasing_started_at_ms, + } + } + + /// Timestamp component of the newest revision retained at erase start. + pub fn erasing_from_time_ms(&self) -> u64 { + match self { + DocumentLifecycleRecord::V0(v0) => v0.erasing_from_time_ms, + } + } + + /// History sequence of the newest revision retained at erase start. + pub fn erasing_from_revision(&self) -> u64 { + match self { + DocumentLifecycleRecord::V0(v0) => v0.erasing_from_revision, + } + } + + /// Whether an authorized erasure has already started, which is what lets a + /// continuation run without any authorization of its own. + /// + /// Read off the revision the erasure started from rather than the time it + /// started at: a history sequence is one or more for every real revision, + /// so zero can only mean the field was never written, while a block time of + /// zero is a value a clock could in principle produce. + pub fn is_erasing(&self) -> bool { + self.erasing_from_revision() != 0 + } + + /// Returns the record an erase start writes over this one, keeping the + /// deletion time and recording the erasure it authorizes. + pub fn starting_erase_at( + &self, + erasing_started_at_ms: u64, + erasing_from_time_ms: u64, + erasing_from_revision: u64, + ) -> Self { + DocumentLifecycleRecord::V0(DocumentLifecycleRecordV0 { + deleted_at_ms: self.deleted_at_ms(), + latest_revision: self.latest_revision(), + revision_count: self.revision_count(), + erasing_started_at_ms, + erasing_from_time_ms, + erasing_from_revision, + }) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::serialization::{PlatformDeserializableUntrusted, PlatformSerializable}; + + #[test] + fn should_round_trip_every_field() { + let record = DocumentLifecycleRecord::deleted_at(7, 12, 11).starting_erase_at(8, 9, 10); + let bytes = record.serialize_to_bytes().expect("serialize"); + let recovered = + DocumentLifecycleRecord::deserialize_from_bytes_untrusted(&bytes).expect("round trip"); + assert_eq!(record, recovered); + assert!(recovered.is_erasing()); + assert_eq!(recovered.latest_revision(), 12); + assert_eq!(recovered.revision_count(), 11); + assert!(!recovered.revisions_are_contiguous()); + assert!(DocumentLifecycleRecord::deleted_at(7, 11, 11).revisions_are_contiguous()); + } + + /// The bound prices a record nobody has read yet, so it must hold for the + /// widest values every field can take. + #[test] + fn should_never_encode_past_the_size_bound() { + let widest = DocumentLifecycleRecord::deleted_at(u64::MAX, u64::MAX, u64::MAX) + .starting_erase_at(u64::MAX, u64::MAX, u64::MAX); + assert_eq!( + widest.serialize_to_bytes().expect("serialize").len(), + DOCUMENT_LIFECYCLE_RECORD_MAX_SIZE as usize + ); + let narrowest = DocumentLifecycleRecord::deleted_at(0, 0, 0); + assert!( + narrowest.serialize_to_bytes().expect("serialize").len() + <= DOCUMENT_LIFECYCLE_RECORD_MAX_SIZE as usize + ); + } + + #[test] + fn should_report_a_record_without_erase_fields_as_deleted_not_erasing() { + let record = DocumentLifecycleRecord::deleted_at(7, 1, 1); + assert!(!record.is_erasing()); + assert_eq!(record.erasing_from_revision(), 0); + } + + /// A block whose time is zero must not make a committed erasure look like + /// one that never started. + #[test] + fn should_report_an_erasure_started_in_a_block_at_time_zero_as_erasing() { + let record = DocumentLifecycleRecord::deleted_at(0, 1, 1).starting_erase_at(0, 0, 1); + assert!(record.is_erasing()); + } + + #[test] + fn should_reject_bytes_that_are_not_a_record() { + assert!(DocumentLifecycleRecord::deserialize_from_bytes_untrusted(&[]).is_err()); + assert!(DocumentLifecycleRecord::deserialize_from_bytes_untrusted(&[9]).is_err()); + } +} diff --git a/packages/rs-dpp/src/document/lifecycle/v0/mod.rs b/packages/rs-dpp/src/document/lifecycle/v0/mod.rs new file mode 100644 index 00000000000..c06cbdc7c26 --- /dev/null +++ b/packages/rs-dpp/src/document/lifecycle/v0/mod.rs @@ -0,0 +1,32 @@ +use bincode::{Decode, DecodeUntrusted, Encode}; +use derive_more::From; + +/// What the committed state says about one keep-history document that is no +/// longer visible to ordinary reads. +#[derive(Debug, Clone, Copy, Encode, Decode, DecodeUntrusted, From, PartialEq, Eq, Default)] +#[cfg_attr( + feature = "serde-conversion", + derive(serde::Serialize, serde::Deserialize), + serde(rename_all = "camelCase") +)] +pub struct DocumentLifecycleRecordV0 { + /// Block time the document was deleted at. + pub deleted_at_ms: u64, + /// Revision the document carried when it was deleted. + pub latest_revision: u64, + /// Number of revisions the history retained when the document was deleted. + /// + /// Together with `latest_revision` this says whether the retained history + /// was contiguous: an erase removes the newest revisions first, so what + /// survives afterwards is a prefix of what was retained here, and a + /// by-revision read stays meaningful exactly when the two are equal. + pub revision_count: u64, + /// Block time an authorized erasure started at, or zero while none has. + pub erasing_started_at_ms: u64, + /// Timestamp component of the newest revision retained when the erasure + /// started, or zero while none has. + pub erasing_from_time_ms: u64, + /// History sequence of the newest revision retained when the erasure + /// started, or zero while none has. + pub erasing_from_revision: u64, +} diff --git a/packages/rs-dpp/src/document/mod.rs b/packages/rs-dpp/src/document/mod.rs index ce4d97fba80..b573937b34b 100644 --- a/packages/rs-dpp/src/document/mod.rs +++ b/packages/rs-dpp/src/document/mod.rs @@ -13,6 +13,7 @@ pub mod errors; pub mod extended_document; mod fields; pub mod generate_document_id; +pub mod lifecycle; pub mod serialization_traits; #[cfg(feature = "factories")] pub mod specialized_document_factory; diff --git a/packages/rs-dpp/src/state_transition/mod.rs b/packages/rs-dpp/src/state_transition/mod.rs index 8ffac4cf8bf..85900ccd24a 100644 --- a/packages/rs-dpp/src/state_transition/mod.rs +++ b/packages/rs-dpp/src/state_transition/mod.rs @@ -990,6 +990,7 @@ impl StateTransition { BatchedTransitionRef::Document(DocumentTransition::IndexOnlyDelete(_)) => { "IndexOnlyDelete" } + BatchedTransitionRef::Document(DocumentTransition::Erase(_)) => "Erase", BatchedTransitionRef::Token(TokenTransition::Transfer(_)) => { "TokenTransfer" } @@ -2561,6 +2562,86 @@ mod tests { })) } + fn sample_format_1_batch_st_with_erase() -> StateTransition { + use crate::state_transition::batch_transition::batched_transition::document_erase_transition::DocumentEraseTransitionV0; + use crate::state_transition::batch_transition::batched_transition::{ + BatchedTransition, DocumentEraseTransition, + }; + use crate::state_transition::batch_transition::BatchTransitionV1; + + let base = DocumentBaseTransition::V0(DocumentBaseTransitionV0 { + id: Identifier::from([1u8; 32]), + identity_contract_nonce: 3, + document_type_name: "note".to_string(), + data_contract_id: Identifier::from([2u8; 32]), + }); + let erase = + DocumentTransition::Erase(DocumentEraseTransition::V0(DocumentEraseTransitionV0 { + base, + })); + StateTransition::Batch(BatchTransition::V1(BatchTransitionV1 { + owner_id: Identifier::from([8u8; 32]), + transitions: vec![BatchedTransition::Document(erase)], + user_fee_increase: 2, + signature_public_key_id: 7, + signature: BinaryData::new(vec![0xEE; 65]), + })) + } + + /// The erase kind rides in the shipped batch formats, so a format 1 batch + /// carrying one keeps format 1's range and decodes under protocol 14 on + /// software that knows the kind. Agreement with software that cannot + /// decode it comes one step later: the basic-structure check refuses the + /// kind wherever the active version publishes no bounds for it. + #[cfg(all(feature = "state-transitions", feature = "validation"))] + #[test] + fn test_format_1_batch_with_erase_decodes_at_protocol_14_and_is_refused_by_structure() { + use crate::consensus::basic::BasicError; + use crate::consensus::ConsensusError; + use crate::serialization::PlatformSerializable; + + let state_transition = sample_format_1_batch_st_with_erase(); + assert_eq!( + state_transition.active_version_range(), + 9..=LATEST_VERSION, + "format 1 arrived with tokens at protocol version 9; the erase kind does not move it" + ); + let bytes = PlatformSerializable::serialize_to_bytes(&state_transition) + .expect("serialize succeeds"); + + let released = PlatformVersion::get(14).expect("protocol version 14 exists"); + let decoded = + StateTransition::deserialize_from_bytes_untrusted_in_version(&bytes, released) + .expect("a format 1 batch with an erase decodes under protocol version 14"); + assert_eq!(decoded.name(), "DocumentsBatch([Erase])"); + let StateTransition::Batch(batch) = decoded else { + panic!("expected a batch, got {decoded:?}"); + }; + + let is_unsupported_version = |error: &ConsensusError| { + matches!( + error, + ConsensusError::BasicError(BasicError::UnsupportedVersionError(_)) + ) + }; + let result = batch + .validate_base_structure(released) + .expect("no protocol err"); + assert!( + result.errors.iter().any(is_unsupported_version), + "protocol version 14 must refuse the erase kind, got {:?}", + result.errors + ); + let result = batch + .validate_base_structure(PlatformVersion::latest()) + .expect("no protocol err"); + assert!( + !result.errors.iter().any(is_unsupported_version), + "the latest protocol version must admit the erase kind, got {:?}", + result.errors + ); + } + fn sample_batch_st_empty() -> StateTransition { StateTransition::Batch(BatchTransition::V0(BatchTransitionV0 { owner_id: Identifier::from([1u8; 32]), diff --git a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_erase_transition/from_document.rs b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_erase_transition/from_document.rs new file mode 100644 index 00000000000..3f6d676ecf3 --- /dev/null +++ b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_erase_transition/from_document.rs @@ -0,0 +1,52 @@ +use crate::data_contract::document_type::DocumentTypeRef; +use crate::document::Document; +use crate::prelude::IdentityNonce; +use crate::ProtocolError; +use platform_version::version::{FeatureVersion, PlatformVersion}; + +use crate::state_transition::batch_transition::batched_transition::document_erase_transition::DocumentEraseTransitionV0; +use crate::state_transition::batch_transition::batched_transition::DocumentEraseTransition; +use crate::tokens::token_payment_info::TokenPaymentInfo; + +impl DocumentEraseTransition { + #[allow(clippy::too_many_arguments)] + pub fn from_document( + document: Document, + document_type: DocumentTypeRef, + token_payment_info: Option, + identity_contract_nonce: IdentityNonce, + feature_version: Option, + base_feature_version: Option, + platform_version: &PlatformVersion, + ) -> Result { + // `None` bounds mean the kind does not exist at this platform version + // (it joined the wire at protocol version 15) — constructing one there + // could only ever produce a transition the network refuses. + let bounds = platform_version + .dpp + .state_transition_serialization_versions + .document_erase_state_transition + .as_ref() + .ok_or_else(|| { + ProtocolError::Generic( + "erase transitions do not exist at this platform version".to_string(), + ) + })?; + match feature_version.unwrap_or(bounds.bounds.default_current_version) { + 0 => Ok(DocumentEraseTransitionV0::from_document( + document, + document_type, + token_payment_info, + identity_contract_nonce, + base_feature_version, + platform_version, + )? + .into()), + version => Err(ProtocolError::UnknownVersionMismatch { + method: "DocumentEraseTransition::from_document".to_string(), + known_versions: vec![0], + received: version, + }), + } + } +} diff --git a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_erase_transition/mod.rs b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_erase_transition/mod.rs new file mode 100644 index 00000000000..ae751b487cf --- /dev/null +++ b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_erase_transition/mod.rs @@ -0,0 +1,97 @@ +mod from_document; +pub mod v0; +pub mod v0_methods; + +use bincode::{Decode, DecodeUntrusted, Encode}; +use derive_more::{Display, From}; +#[cfg(feature = "serde-conversion")] +use serde::{Deserialize, Serialize}; +pub use v0::*; + +#[derive(Debug, Clone, Encode, Decode, PartialEq, Display, From, DecodeUntrusted)] +#[cfg_attr( + feature = "serde-conversion", + derive(Serialize, Deserialize), + serde(tag = "$formatVersion") +)] +pub enum DocumentEraseTransition { + #[display("V0({})", "_0")] + #[cfg_attr(feature = "serde-conversion", serde(rename = "0"))] + V0(DocumentEraseTransitionV0), +} + +#[cfg(all(feature = "json-conversion", feature = "serde-conversion"))] +impl crate::serialization::JsonConvertible for DocumentEraseTransition {} + +#[cfg(all(feature = "value-conversion", feature = "serde-conversion"))] +impl crate::serialization::ValueConvertible for DocumentEraseTransition {} + +#[cfg(all( + test, + feature = "json-conversion", + feature = "value-conversion", + feature = "serde-conversion" +))] +pub(crate) mod json_convertible_tests { + use super::*; + use crate::state_transition::batch_transition::document_base_transition::v0::DocumentBaseTransitionV0; + use crate::state_transition::batch_transition::document_base_transition::DocumentBaseTransition; + use platform_value::{platform_value, Identifier}; + use serde_json::json; + + /// Non-default values per field so the wire-shape assertion catches any + /// silent zero-out / flip on round-trip. + pub(crate) fn fixture() -> DocumentEraseTransition { + DocumentEraseTransition::V0(DocumentEraseTransitionV0 { + base: DocumentBaseTransition::V0(DocumentBaseTransitionV0 { + id: Identifier::new([0xe3; 32]), + identity_contract_nonce: 11, + document_type_name: "note".to_string(), + data_contract_id: Identifier::new([0xf4; 32]), + }), + }) + } + + #[test] + fn json_round_trip_with_full_wire_shape() { + use crate::serialization::JsonConvertible; + let original = fixture(); + let json = original.to_json().expect("to_json"); + // Doubly-tagged externally enum: outer `V0` for + // `DocumentEraseTransition`, inner `V0` for the flattened + // `base: DocumentBaseTransition`. + assert_eq!( + json, + json!({ + "$formatVersion": "0", + "$baseFormatVersion": "0", + "$id": Identifier::new([0xe3; 32]), + "$identityContractNonce": 11, + "$type": "note", + "$dataContractId": Identifier::new([0xf4; 32]), + }) + ); + let recovered = DocumentEraseTransition::from_json(json).expect("from_json"); + assert_eq!(original, recovered); + } + + #[test] + fn value_round_trip_with_full_wire_shape() { + use crate::serialization::ValueConvertible; + let original = fixture(); + let value = original.to_object().expect("to_object"); + assert_eq!( + value, + platform_value!({ + "$formatVersion": "0", + "$baseFormatVersion": "0", + "$id": Identifier::new([0xe3; 32]), + "$identityContractNonce": 11u64, + "$type": "note", + "$dataContractId": Identifier::new([0xf4; 32]), + }) + ); + let recovered = DocumentEraseTransition::from_object(value).expect("from_object"); + assert_eq!(original, recovered); + } +} diff --git a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_erase_transition/v0/from_document.rs b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_erase_transition/v0/from_document.rs new file mode 100644 index 00000000000..b423ad0397c --- /dev/null +++ b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_erase_transition/v0/from_document.rs @@ -0,0 +1,30 @@ +use crate::data_contract::document_type::DocumentTypeRef; +use crate::document::Document; +use crate::prelude::IdentityNonce; +use crate::state_transition::batch_transition::batched_transition::document_erase_transition::DocumentEraseTransitionV0; +use crate::state_transition::batch_transition::document_base_transition::DocumentBaseTransition; +use crate::tokens::token_payment_info::TokenPaymentInfo; +use crate::ProtocolError; +use platform_version::version::{FeatureVersion, PlatformVersion}; + +impl DocumentEraseTransitionV0 { + pub(crate) fn from_document( + document: Document, + document_type: DocumentTypeRef, + token_payment_info: Option, + identity_contract_nonce: IdentityNonce, + base_feature_version: Option, + platform_version: &PlatformVersion, + ) -> Result { + Ok(DocumentEraseTransitionV0 { + base: DocumentBaseTransition::from_document( + &document, + document_type, + token_payment_info, + identity_contract_nonce, + platform_version, + base_feature_version, + )?, + }) + } +} diff --git a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_erase_transition/v0/mod.rs b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_erase_transition/v0/mod.rs new file mode 100644 index 00000000000..c86632c159e --- /dev/null +++ b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_erase_transition/v0/mod.rs @@ -0,0 +1,32 @@ +mod from_document; +pub mod v0_methods; + +use crate::state_transition::batch_transition::document_base_transition::DocumentBaseTransition; + +use bincode::{Decode, DecodeUntrusted, Encode}; +use derive_more::Display; + +#[cfg(feature = "json-conversion")] +use crate::serialization::json_safe_fields; +#[cfg(feature = "serde-conversion")] +use serde::{Deserialize, Serialize}; + +pub use super::super::document_base_transition::IDENTIFIER_FIELDS; + +/// Purges the retained revisions of a document that has already been deleted. +/// +/// The transition carries nothing beyond the base: which chunk it removes, and +/// whether it starts or continues an erasure, are decided from the document's +/// committed lifecycle rather than from anything the submitter signs. +#[cfg_attr(feature = "json-conversion", json_safe_fields)] +#[derive(Debug, Clone, Default, Encode, Decode, PartialEq, Display, DecodeUntrusted)] +#[cfg_attr( + feature = "serde-conversion", + derive(Serialize, Deserialize), + serde(rename_all = "camelCase") +)] +#[display("Base: {}", "base")] +pub struct DocumentEraseTransitionV0 { + #[cfg_attr(feature = "serde-conversion", serde(flatten))] + pub base: DocumentBaseTransition, +} diff --git a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_erase_transition/v0/v0_methods.rs b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_erase_transition/v0/v0_methods.rs new file mode 100644 index 00000000000..6265e81d824 --- /dev/null +++ b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_erase_transition/v0/v0_methods.rs @@ -0,0 +1,17 @@ +use crate::state_transition::batch_transition::batched_transition::document_erase_transition::DocumentEraseTransitionV0; +use crate::state_transition::batch_transition::document_base_transition::document_base_transition_trait::DocumentBaseTransitionAccessors; +use crate::state_transition::batch_transition::document_base_transition::DocumentBaseTransition; + +impl DocumentBaseTransitionAccessors for DocumentEraseTransitionV0 { + fn base(&self) -> &DocumentBaseTransition { + &self.base + } + + fn base_mut(&mut self) -> &mut DocumentBaseTransition { + &mut self.base + } + + fn set_base(&mut self, base: DocumentBaseTransition) { + self.base = base + } +} diff --git a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_erase_transition/v0_methods.rs b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_erase_transition/v0_methods.rs new file mode 100644 index 00000000000..df32a5641fc --- /dev/null +++ b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_erase_transition/v0_methods.rs @@ -0,0 +1,23 @@ +use crate::state_transition::batch_transition::batched_transition::DocumentEraseTransition; +use crate::state_transition::batch_transition::document_base_transition::document_base_transition_trait::DocumentBaseTransitionAccessors; +use crate::state_transition::batch_transition::document_base_transition::DocumentBaseTransition; + +impl DocumentBaseTransitionAccessors for DocumentEraseTransition { + fn base(&self) -> &DocumentBaseTransition { + match self { + DocumentEraseTransition::V0(v0) => &v0.base, + } + } + + fn base_mut(&mut self) -> &mut DocumentBaseTransition { + match self { + DocumentEraseTransition::V0(v0) => &mut v0.base, + } + } + + fn set_base(&mut self, base: DocumentBaseTransition) { + match self { + DocumentEraseTransition::V0(v0) => v0.base = base, + } + } +} diff --git a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_transition.rs b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_transition.rs index 1b9c13ddea9..ca6510dc0ca 100644 --- a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_transition.rs +++ b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_transition.rs @@ -6,7 +6,7 @@ use serde::{Deserialize, Serialize}; use bincode::{Encode, Decode, DecodeUntrusted}; use crate::prelude::{IdentityNonce, Revision}; use crate::state_transition::batch_transition::{DocumentCreateTransition, DocumentDeleteTransition, DocumentReplaceTransition, TokenBurnTransition, TokenConfigUpdateTransition, TokenDestroyFrozenFundsTransition, TokenEmergencyActionTransition, TokenFreezeTransition, TokenMintTransition, TokenClaimTransition, TokenTransferTransition, TokenUnfreezeTransition, TokenDirectPurchaseTransition, TokenSetPriceForDirectPurchaseTransition}; -use crate::state_transition::batch_transition::batched_transition::{DocumentIndexOnlyDeleteTransition, DocumentPurchaseTransition, DocumentTransferTransition, DocumentUpdatePriceTransition}; +use crate::state_transition::batch_transition::batched_transition::{DocumentEraseTransition, DocumentIndexOnlyDeleteTransition, DocumentPurchaseTransition, DocumentTransferTransition, DocumentUpdatePriceTransition}; use crate::state_transition::batch_transition::batched_transition::document_index_only_delete_transition::v0::v0_methods::DocumentIndexOnlyDeleteTransitionV0Methods; use crate::state_transition::batch_transition::batched_transition::document_purchase_transition::v0::v0_methods::DocumentPurchaseTransitionV0Methods; use crate::state_transition::batch_transition::batched_transition::document_transfer_transition::v0::v0_methods::DocumentTransferTransitionV0Methods; @@ -57,6 +57,13 @@ pub enum DocumentTransition { /// PV14+ (see the wire gate in `validate_base_structure_v0`). #[display("IndexOnlyDeleteDocumentTransition({})", "_0")] IndexOnlyDelete(DocumentIndexOnlyDeleteTransition), + + /// The erase kind, which removes the retained revisions of a deleted + /// keep-history document — appended at the end so every existing variant + /// keeps its bincode discriminant. Only exists at protocol version 15 and + /// later (see the wire gate in `validate_base_structure_v0`). + #[display("EraseDocumentTransition({})", "_0")] + Erase(DocumentEraseTransition), } #[cfg(all(feature = "json-conversion", feature = "serde-conversion"))] @@ -74,7 +81,7 @@ impl crate::serialization::ValueConvertible for DocumentTransition {} pub(crate) mod json_convertible_tests { use super::*; use crate::state_transition::batch_transition::batched_transition::{ - document_create_transition, document_delete_transition, + document_create_transition, document_delete_transition, document_erase_transition, document_index_only_delete_transition, document_purchase_transition, document_replace_transition, document_transfer_transition, document_update_price_transition, @@ -183,6 +190,14 @@ pub(crate) mod json_convertible_tests { "indexOnlyDelete", ); } + + #[test] + fn umbrella_erase() { + assert_umbrella_round_trip( + DocumentTransition::Erase(document_erase_transition::json_convertible_tests::fixture()), + "erase", + ); + } } impl BatchTransitionResolversV0 for DocumentTransition { @@ -225,6 +240,14 @@ impl BatchTransitionResolversV0 for DocumentTransition { } } + fn as_transition_erase(&self) -> Option<&DocumentEraseTransition> { + if let Self::Erase(ref t) = self { + Some(t) + } else { + None + } + } + fn as_transition_token_burn(&self) -> Option<&TokenBurnTransition> { None } @@ -325,6 +348,7 @@ impl DocumentTransitionV0Methods for DocumentTransition { DocumentTransition::UpdatePrice(t) => t.base(), DocumentTransition::Purchase(t) => t.base(), DocumentTransition::IndexOnlyDelete(t) => t.base(), + DocumentTransition::Erase(t) => t.base(), } } @@ -337,6 +361,7 @@ impl DocumentTransitionV0Methods for DocumentTransition { DocumentTransition::UpdatePrice(_) => None, DocumentTransition::Purchase(_) => None, DocumentTransition::IndexOnlyDelete(t) => t.data().get(path), + DocumentTransition::Erase(_) => None, } } @@ -357,6 +382,7 @@ impl DocumentTransitionV0Methods for DocumentTransition { DocumentTransition::UpdatePrice(_) => None, DocumentTransition::Purchase(_) => None, DocumentTransition::IndexOnlyDelete(_) => None, + DocumentTransition::Erase(_) => None, } } @@ -373,6 +399,7 @@ impl DocumentTransitionV0Methods for DocumentTransition { DocumentTransition::UpdatePrice(_) => None, DocumentTransition::Purchase(_) => None, DocumentTransition::IndexOnlyDelete(t) => Some(t.data()), + DocumentTransition::Erase(_) => None, } } @@ -391,6 +418,7 @@ impl DocumentTransitionV0Methods for DocumentTransition { DocumentTransition::UpdatePrice(t) => Some(t.revision()), DocumentTransition::Purchase(t) => Some(t.revision()), DocumentTransition::IndexOnlyDelete(_) => None, + DocumentTransition::Erase(_) => None, } } @@ -403,6 +431,7 @@ impl DocumentTransitionV0Methods for DocumentTransition { DocumentTransition::UpdatePrice(t) => t.base().identity_contract_nonce(), DocumentTransition::Purchase(t) => t.base().identity_contract_nonce(), DocumentTransition::IndexOnlyDelete(t) => t.base().identity_contract_nonce(), + DocumentTransition::Erase(t) => t.base().identity_contract_nonce(), } } @@ -426,6 +455,7 @@ impl DocumentTransitionV0Methods for DocumentTransition { DocumentTransition::IndexOnlyDelete(t) => { t.data_mut().insert(property_name, value); } + DocumentTransition::Erase(_) => {} } } @@ -442,6 +472,7 @@ impl DocumentTransitionV0Methods for DocumentTransition { DocumentTransition::UpdatePrice(t) => t.base_mut(), DocumentTransition::Purchase(t) => t.base_mut(), DocumentTransition::IndexOnlyDelete(t) => t.base_mut(), + DocumentTransition::Erase(t) => t.base_mut(), } } @@ -454,6 +485,7 @@ impl DocumentTransitionV0Methods for DocumentTransition { DocumentTransition::UpdatePrice(_) => None, DocumentTransition::Purchase(_) => None, DocumentTransition::IndexOnlyDelete(t) => Some(t.data_mut()), + DocumentTransition::Erase(_) => None, } } @@ -466,6 +498,7 @@ impl DocumentTransitionV0Methods for DocumentTransition { DocumentTransition::UpdatePrice(ref mut t) => t.set_revision(revision), DocumentTransition::Purchase(ref mut t) => t.set_revision(revision), DocumentTransition::IndexOnlyDelete(_) => {} + DocumentTransition::Erase(_) => {} } } @@ -480,6 +513,7 @@ impl DocumentTransitionV0Methods for DocumentTransition { DocumentTransition::IndexOnlyDelete(t) => { t.base_mut().set_identity_contract_nonce(nonce) } + DocumentTransition::Erase(t) => t.base_mut().set_identity_contract_nonce(nonce), } } } diff --git a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_transition_action_type.rs b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_transition_action_type.rs index 02b5ff95f97..b2a946706ee 100644 --- a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_transition_action_type.rs +++ b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_transition_action_type.rs @@ -12,6 +12,7 @@ pub enum DocumentTransitionActionType { UpdatePrice, IgnoreWhileBumpingRevision, IndexOnlyDelete, + Erase, } pub trait DocumentTransitionActionTypeGetter { @@ -28,6 +29,7 @@ impl DocumentTransitionActionTypeGetter for DocumentTransition { DocumentTransition::UpdatePrice(_) => DocumentTransitionActionType::UpdatePrice, DocumentTransition::Purchase(_) => DocumentTransitionActionType::Purchase, DocumentTransition::IndexOnlyDelete(_) => DocumentTransitionActionType::IndexOnlyDelete, + DocumentTransition::Erase(_) => DocumentTransitionActionType::Erase, } } } @@ -46,6 +48,7 @@ impl TryFrom<&str> for DocumentTransitionActionType { "indexOnlyDelete" | "index_only_delete" => { Ok(DocumentTransitionActionType::IndexOnlyDelete) } + "erase" => Ok(DocumentTransitionActionType::Erase), action_type => Err(ProtocolError::Generic(format!( "unknown action type {action_type}" ))), diff --git a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/mod.rs b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/mod.rs index a71a4271563..dda869a3e06 100644 --- a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/mod.rs +++ b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/mod.rs @@ -6,6 +6,7 @@ use serde::{Deserialize, Serialize}; pub mod document_base_transition; pub mod document_create_transition; pub mod document_delete_transition; +pub mod document_erase_transition; pub mod document_index_only_delete_transition; pub mod document_purchase_transition; pub mod document_replace_transition; @@ -36,6 +37,7 @@ use crate::state_transition::batch_transition::batched_transition::token_transit use derive_more::Display; pub use document_create_transition::DocumentCreateTransition; pub use document_delete_transition::DocumentDeleteTransition; +pub use document_erase_transition::DocumentEraseTransition; pub use document_index_only_delete_transition::DocumentIndexOnlyDeleteTransition; pub use document_purchase_transition::DocumentPurchaseTransition; pub use document_replace_transition::DocumentReplaceTransition; @@ -81,7 +83,7 @@ impl crate::serialization::ValueConvertible for BatchedTransition {} pub(crate) mod json_convertible_tests { use super::*; use crate::state_transition::batch_transition::batched_transition::{ - document_create_transition, token_burn_transition, + document_create_transition, document_erase_transition, token_burn_transition, }; use document_transition::DocumentTransition; use token_transition::TokenTransition; @@ -129,6 +131,13 @@ pub(crate) mod json_convertible_tests { assert_umbrella_round_trip(BatchedTransition::Document(inner), "document"); } + #[test] + fn umbrella_document_erase() { + let inner = + DocumentTransition::Erase(document_erase_transition::json_convertible_tests::fixture()); + assert_umbrella_round_trip(BatchedTransition::Document(inner), "document"); + } + #[test] fn umbrella_token() { let inner = TokenTransition::Burn(token_burn_transition::json_convertible_tests::fixture()); diff --git a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/resolvers.rs b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/resolvers.rs index d8dabc14ca0..423bce1c51a 100644 --- a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/resolvers.rs +++ b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/resolvers.rs @@ -1,5 +1,6 @@ use crate::state_transition::batch_transition::batched_transition::{ - BatchedTransition, BatchedTransitionRef, DocumentPurchaseTransition, DocumentTransferTransition, + BatchedTransition, BatchedTransitionRef, DocumentEraseTransition, DocumentPurchaseTransition, + DocumentTransferTransition, }; use crate::state_transition::batch_transition::resolvers::v0::BatchTransitionResolversV0; use crate::state_transition::batch_transition::{ @@ -46,6 +47,13 @@ impl BatchTransitionResolversV0 for BatchedTransition { } } + fn as_transition_erase(&self) -> Option<&DocumentEraseTransition> { + match self { + BatchedTransition::Document(document) => document.as_transition_erase(), + BatchedTransition::Token(_) => None, + } + } + fn as_transition_token_burn(&self) -> Option<&TokenBurnTransition> { match self { BatchedTransition::Document(_) => None, @@ -166,6 +174,13 @@ impl BatchTransitionResolversV0 for BatchedTransitionRef<'_> { } } + fn as_transition_erase(&self) -> Option<&DocumentEraseTransition> { + match self { + BatchedTransitionRef::Document(document) => document.as_transition_erase(), + BatchedTransitionRef::Token(_) => None, + } + } + fn as_transition_token_burn(&self) -> Option<&TokenBurnTransition> { match self { BatchedTransitionRef::Document(_) => None, diff --git a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/token_transition.rs b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/token_transition.rs index 0269ea0b610..4f119c32f42 100644 --- a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/token_transition.rs +++ b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/token_transition.rs @@ -19,7 +19,7 @@ use crate::document::Document; use crate::prelude::IdentityNonce; use crate::ProtocolError; use crate::state_transition::batch_transition::{DocumentCreateTransition, DocumentDeleteTransition, DocumentReplaceTransition, TokenBurnTransition, TokenConfigUpdateTransition, TokenDestroyFrozenFundsTransition, TokenEmergencyActionTransition, TokenFreezeTransition, TokenMintTransition, TokenClaimTransition, TokenTransferTransition, TokenSetPriceForDirectPurchaseTransition}; -use crate::state_transition::batch_transition::batched_transition::{DocumentPurchaseTransition, DocumentTransferTransition}; +use crate::state_transition::batch_transition::batched_transition::{DocumentEraseTransition, DocumentPurchaseTransition, DocumentTransferTransition}; use crate::state_transition::batch_transition::batched_transition::multi_party_action::AllowedAsMultiPartyAction; use crate::state_transition::batch_transition::batched_transition::token_unfreeze_transition::TokenUnfreezeTransition; use crate::state_transition::batch_transition::resolvers::v0::BatchTransitionResolversV0; @@ -264,6 +264,10 @@ impl BatchTransitionResolversV0 for TokenTransition { None } + fn as_transition_erase(&self) -> Option<&DocumentEraseTransition> { + None + } + fn as_transition_token_burn(&self) -> Option<&TokenBurnTransition> { if let Self::Burn(ref t) = self { Some(t) diff --git a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/methods/mod.rs b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/methods/mod.rs index ad8d520cdc5..5e9acff9b05 100644 --- a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/methods/mod.rs +++ b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/methods/mod.rs @@ -205,6 +205,62 @@ impl DocumentsBatchTransitionMethodsV0 for BatchTransition { } #[cfg(feature = "state-transition-signing")] + #[allow(clippy::too_many_arguments)] + async fn new_document_erase_transition_from_document>( + document: Document, + document_type: DocumentTypeRef<'_>, + identity_public_key: &IdentityPublicKey, + identity_contract_nonce: IdentityNonce, + user_fee_increase: UserFeeIncrease, + signer: &S, + platform_version: &PlatformVersion, + options: Option, + ) -> Result { + let resolved_options = options.unwrap_or_default(); + match resolved_options.batch_feature_version.unwrap_or( + platform_version + .dpp + .state_transition_serialization_versions + .batch_state_transition + .default_current_version, + ) { + 0 => Ok( + BatchTransitionV0::new_document_erase_transition_from_document( + document, + document_type, + identity_public_key, + identity_contract_nonce, + user_fee_increase, + signer, + platform_version, + options, + ) + .await?, + ), + 1 => Ok( + BatchTransitionV1::new_document_erase_transition_from_document( + document, + document_type, + identity_public_key, + identity_contract_nonce, + user_fee_increase, + signer, + platform_version, + options, + ) + .await?, + ), + version => Err(ProtocolError::UnknownVersionMismatch { + method: "DocumentsBatchTransition::new_document_erase_transition_from_document" + .to_string(), + known_versions: vec![0, 1], + received: version, + }), + } + } + + #[cfg(feature = "state-transition-signing")] + #[allow(clippy::too_many_arguments)] async fn new_document_transfer_transition_from_document>( document: Document, document_type: DocumentTypeRef<'_>, diff --git a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/methods/v0/mod.rs b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/methods/v0/mod.rs index 01d042c7a8a..60ad2b2a32c 100644 --- a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/methods/v0/mod.rs +++ b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/methods/v0/mod.rs @@ -71,6 +71,25 @@ pub trait DocumentsBatchTransitionMethodsV0: DocumentsBatchTransitionAccessorsV0 options: Option, ) -> Result; + /// Builds a signed erase of one already deleted keep-history document. + /// + /// The transition carries no token payment: the deletion cost was charged + /// when the document was deleted. Whether this erase starts or continues an + /// erasure, and how many revisions it removes, are read from committed + /// state rather than signed here. + #[cfg(feature = "state-transition-signing")] + #[allow(clippy::too_many_arguments)] + async fn new_document_erase_transition_from_document>( + document: Document, + document_type: DocumentTypeRef<'_>, + identity_public_key: &IdentityPublicKey, + identity_contract_nonce: IdentityNonce, + user_fee_increase: UserFeeIncrease, + signer: &S, + platform_version: &PlatformVersion, + options: Option, + ) -> Result; + #[cfg(feature = "state-transition-signing")] #[allow(clippy::too_many_arguments)] async fn new_document_transfer_transition_from_document>( diff --git a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/mod.rs b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/mod.rs index b3abe347bbb..d44ee39e535 100644 --- a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/mod.rs +++ b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/mod.rs @@ -14,7 +14,8 @@ use crate::{identity::SecurityLevel, state_transition::StateTransitionFieldTypes pub use self::batched_transition::{ document_base_transition, document_create_transition, document_create_transition::DocumentCreateTransition, document_delete_transition, - document_delete_transition::DocumentDeleteTransition, document_index_only_delete_transition, + document_delete_transition::DocumentDeleteTransition, document_erase_transition, + document_erase_transition::DocumentEraseTransition, document_index_only_delete_transition, document_index_only_delete_transition::DocumentIndexOnlyDeleteTransition, document_replace_transition, document_replace_transition::DocumentReplaceTransition, token_base_transition, token_burn_transition, token_burn_transition::TokenBurnTransition, diff --git a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/resolvers/v0/mod.rs b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/resolvers/v0/mod.rs index bdad604186b..57b24b9b2c2 100644 --- a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/resolvers/v0/mod.rs +++ b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/resolvers/v0/mod.rs @@ -1,6 +1,6 @@ use crate::state_transition::batch_transition::batched_transition::token_unfreeze_transition::TokenUnfreezeTransition; use crate::state_transition::batch_transition::batched_transition::{ - DocumentPurchaseTransition, DocumentTransferTransition, + DocumentEraseTransition, DocumentPurchaseTransition, DocumentTransferTransition, }; use crate::state_transition::batch_transition::token_direct_purchase_transition::TokenDirectPurchaseTransition; use crate::state_transition::batch_transition::{ @@ -16,6 +16,7 @@ pub trait BatchTransitionResolversV0 { fn as_transition_delete(&self) -> Option<&DocumentDeleteTransition>; fn as_transition_transfer(&self) -> Option<&DocumentTransferTransition>; fn as_transition_purchase(&self) -> Option<&DocumentPurchaseTransition>; + fn as_transition_erase(&self) -> Option<&DocumentEraseTransition>; fn as_transition_token_burn(&self) -> Option<&TokenBurnTransition>; fn as_transition_token_mint(&self) -> Option<&TokenMintTransition>; fn as_transition_token_transfer(&self) -> Option<&TokenTransferTransition>; diff --git a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/tests.rs b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/tests.rs index c8011b7ec59..20dfaaf2e42 100644 --- a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/tests.rs +++ b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/tests.rs @@ -36,7 +36,7 @@ mod batch_transition_tests { use crate::state_transition::batch_transition::methods::v0::DocumentsBatchTransitionMethodsV0; use crate::state_transition::batch_transition::resolvers::v0::BatchTransitionResolversV0; use crate::state_transition::batch_transition::{ - BatchTransitionV0, BatchTransitionV1, + BatchTransition, BatchTransitionV0, BatchTransitionV1, }; use crate::state_transition::StateTransitionLike; use crate::state_transition::StateTransition; @@ -1186,4 +1186,77 @@ mod batch_transition_tests { assert!(matches!(second, BatchedTransitionRef::Token(_))); assert!(iter.next().is_none()); } + + // ----------------------------------------------------------------------- + // The erase kind on the wire + // ----------------------------------------------------------------------- + + fn make_erase_transition(nonce: u64) -> DocumentTransition { + use crate::state_transition::batch_transition::batched_transition::document_erase_transition::DocumentEraseTransitionV0; + use crate::state_transition::batch_transition::batched_transition::DocumentEraseTransition; + + DocumentTransition::Erase(DocumentEraseTransition::V0(DocumentEraseTransitionV0 { + base: make_base_transition(nonce), + })) + } + + /// The erase kind is appended after every shipped kind, so its bincode + /// discriminant is the next one, 7, and every earlier kind keeps its own. + /// Pinning the byte keeps a later variant reordering from silently + /// changing what old software reads. An erase and a delete over the same + /// base encode identically except for that one byte, in both shipped + /// batch formats, and the bytes decode back to an erase. + #[test] + fn should_encode_an_erase_with_the_appended_discriminant_in_both_shipped_formats() { + use crate::serialization::{PlatformDeserializableUntrusted, PlatformSerializable}; + + let delete = make_delete_transition(1); + let erase = make_erase_transition(1); + + let format_0 = + |transition: DocumentTransition| BatchTransition::V0(make_batch_v0(vec![transition])); + let format_1 = |transition: DocumentTransition| { + BatchTransition::V1(make_batch_v1(vec![BatchedTransition::Document(transition)])) + }; + + for (make, format) in [ + ( + &format_0 as &dyn Fn(DocumentTransition) -> BatchTransition, + 0u8, + ), + (&format_1, 1u8), + ] { + let delete_bytes = make(delete.clone()) + .serialize_to_bytes() + .expect("serialize"); + let erase_batch = make(erase.clone()); + let erase_bytes = erase_batch.serialize_to_bytes().expect("serialize"); + + assert_eq!(erase_bytes[0], format, "the first byte is the batch format"); + assert_eq!(erase_bytes.len(), delete_bytes.len()); + let differing: Vec = erase_bytes + .iter() + .zip(&delete_bytes) + .enumerate() + .filter(|(_, (a, b))| a != b) + .map(|(index, _)| index) + .collect(); + assert_eq!( + differing.len(), + 1, + "an erase and a delete differ only in the kind discriminant" + ); + assert_eq!(erase_bytes[differing[0]], 7, "erase is the eighth kind"); + assert_eq!(delete_bytes[differing[0]], 2, "delete is the third kind"); + + let recovered = + BatchTransition::deserialize_from_bytes_untrusted(&erase_bytes).expect("decode"); + assert_eq!(recovered, erase_batch); + assert!(recovered + .first_transition() + .expect("one transition") + .as_transition_erase() + .is_some()); + } + } } diff --git a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/v0/v0_methods.rs b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/v0/v0_methods.rs index 73f5b7bb738..b3dfb97b985 100644 --- a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/v0/v0_methods.rs +++ b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/v0/v0_methods.rs @@ -23,7 +23,8 @@ use crate::state_transition::batch_transition::methods::v0::DocumentsBatchTransi use crate::state_transition::batch_transition::BatchTransitionV0; #[cfg(feature = "state-transition-signing")] use crate::state_transition::batch_transition::{ - BatchTransition, DocumentDeleteTransition, DocumentIndexOnlyDeleteTransition, + BatchTransition, DocumentDeleteTransition, DocumentEraseTransition, + DocumentIndexOnlyDeleteTransition, }; #[cfg(feature = "state-transition-signing")] use crate::data_contract::document_type::accessors::DocumentTypeV2Getters; @@ -182,6 +183,54 @@ impl DocumentsBatchTransitionMethodsV0 for BatchTransitionV0 { } #[cfg(feature = "state-transition-signing")] + #[allow(clippy::too_many_arguments)] + async fn new_document_erase_transition_from_document>( + document: Document, + document_type: DocumentTypeRef<'_>, + identity_public_key: &IdentityPublicKey, + identity_contract_nonce: IdentityNonce, + user_fee_increase: UserFeeIncrease, + signer: &S, + platform_version: &PlatformVersion, + options: Option, + ) -> Result { + let owner_id = document.owner_id(); + let resolved_options = options.unwrap_or_default(); + // Erase never carries a token payment: it acts on a document whose + // deletion cost was already charged. + let erase_transition: DocumentTransition = DocumentEraseTransition::from_document( + document, + document_type, + None, + identity_contract_nonce, + resolved_options.method_feature_version, + resolved_options.base_feature_version, + platform_version, + )? + .into(); + let documents_batch_transition: BatchTransition = BatchTransitionV0 { + owner_id, + transitions: vec![erase_transition], + user_fee_increase, + signature_public_key_id: 0, + signature: Default::default(), + } + .into(); + let mut state_transition: StateTransition = documents_batch_transition.into(); + let required_security_level = document_type.security_level_requirement(); + state_transition + .sign_external_with_options( + identity_public_key, + signer, + Some(|_, _| Ok(required_security_level)), + resolved_options.signing_options, + ) + .await?; + Ok(state_transition) + } + + #[cfg(feature = "state-transition-signing")] + #[allow(clippy::too_many_arguments)] async fn new_document_transfer_transition_from_document>( document: Document, document_type: DocumentTypeRef<'_>, diff --git a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/v1/v0_methods.rs b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/v1/v0_methods.rs index 0c7247ce76c..0a8fc71205b 100644 --- a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/v1/v0_methods.rs +++ b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/v1/v0_methods.rs @@ -28,7 +28,8 @@ use std::slice::Iter; use crate::state_transition::batch_transition::BatchTransitionV1; #[cfg(feature = "state-transition-signing")] use crate::state_transition::batch_transition::{ - BatchTransition, DocumentDeleteTransition, DocumentIndexOnlyDeleteTransition, + BatchTransition, DocumentDeleteTransition, DocumentEraseTransition, + DocumentIndexOnlyDeleteTransition, }; #[cfg(feature = "state-transition-signing")] use crate::data_contract::document_type::accessors::DocumentTypeV2Getters; @@ -257,6 +258,54 @@ impl DocumentsBatchTransitionMethodsV0 for BatchTransitionV1 { } #[cfg(feature = "state-transition-signing")] + #[allow(clippy::too_many_arguments)] + async fn new_document_erase_transition_from_document>( + document: Document, + document_type: DocumentTypeRef<'_>, + identity_public_key: &IdentityPublicKey, + identity_contract_nonce: IdentityNonce, + user_fee_increase: UserFeeIncrease, + signer: &S, + platform_version: &PlatformVersion, + options: Option, + ) -> Result { + let owner_id = document.owner_id(); + let resolved_options = options.unwrap_or_default(); + // Erase never carries a token payment: it acts on a document whose + // deletion cost was already charged. + let erase_transition: DocumentTransition = DocumentEraseTransition::from_document( + document, + document_type, + None, + identity_contract_nonce, + resolved_options.method_feature_version, + resolved_options.base_feature_version, + platform_version, + )? + .into(); + let documents_batch_transition: BatchTransition = BatchTransitionV1 { + owner_id, + transitions: vec![BatchedTransition::Document(erase_transition)], + user_fee_increase, + signature_public_key_id: 0, + signature: Default::default(), + } + .into(); + let mut state_transition: StateTransition = documents_batch_transition.into(); + let required_security_level = document_type.security_level_requirement(); + state_transition + .sign_external_with_options( + identity_public_key, + signer, + Some(|_, _| Ok(required_security_level)), + resolved_options.signing_options, + ) + .await?; + Ok(state_transition) + } + + #[cfg(feature = "state-transition-signing")] + #[allow(clippy::too_many_arguments)] async fn new_document_transfer_transition_from_document>( document: Document, document_type: DocumentTypeRef<'_>, diff --git a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/validation/validate_basic_structure/v0/mod.rs b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/validation/validate_basic_structure/v0/mod.rs index 83887155a60..7bf75a47a24 100644 --- a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/validation/validate_basic_structure/v0/mod.rs +++ b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/validation/validate_basic_structure/v0/mod.rs @@ -4,7 +4,9 @@ use crate::consensus::basic::document::{ }; use crate::consensus::basic::unsupported_version_error::UnsupportedVersionError; use crate::consensus::basic::BasicError; -use crate::state_transition::batch_transition::batched_transition::DocumentIndexOnlyDeleteTransition; +use crate::state_transition::batch_transition::batched_transition::{ + DocumentEraseTransition, DocumentIndexOnlyDeleteTransition, +}; use crate::identity::identity_nonce::MISSING_IDENTITY_REVISIONS_FILTER; use crate::state_transition::batch_transition::accessors::DocumentsBatchTransitionAccessorsV0; @@ -141,6 +143,43 @@ impl BatchTransition { Some(_) => {} } } + + // The erase kind joined the wire at protocol version 15. Old + // software cannot decode it at all, so no historical block can + // contain one; this check exists so that new software agrees + // with old software while an earlier protocol version is still + // active: without it, an erase submitted at protocol version 14 + // would decode fine here while being undecodable on nodes that + // predate the kind. + if let DocumentTransition::Erase(erase) = transition { + let feature_version = match erase { + DocumentEraseTransition::V0(_) => 0, + }; + match &platform_version + .dpp + .state_transition_serialization_versions + .document_erase_state_transition + { + None => { + // The kind does not exist at this protocol + // version; the empty supported range (min 1, + // max 0) states exactly that. + result.add_error(BasicError::UnsupportedVersionError( + UnsupportedVersionError::new(feature_version, 1, 0), + )); + } + Some(bounds) if !bounds.bounds.check_version(feature_version) => { + result.add_error(BasicError::UnsupportedVersionError( + UnsupportedVersionError::new( + feature_version, + bounds.bounds.min_version, + bounds.bounds.max_version, + ), + )); + } + Some(_) => {} + } + } } // Make sure we don't have duplicate transitions @@ -328,6 +367,104 @@ mod tests { }) } + // ----------------------------------------------------------------------- + // erase kind wire gate + // ----------------------------------------------------------------------- + + fn make_erase(nonce: u64, id_byte: u8) -> DocumentTransition { + use crate::state_transition::batch_transition::batched_transition::document_erase_transition::DocumentEraseTransitionV0; + + DocumentTransition::Erase(DocumentEraseTransition::V0(DocumentEraseTransitionV0 { + base: DocumentBaseTransition::V0(DocumentBaseTransitionV0 { + id: Identifier::new([id_byte; 32]), + identity_contract_nonce: nonce, + document_type_name: "test_doc".to_string(), + data_contract_id: Identifier::new([0xAA; 32]), + }), + })) + } + + /// Software that predates the erase kind cannot decode one at all, so no + /// historical block can contain one. The gate exists so that new software + /// agrees with old software while protocol version 14 is still active: + /// refused there, where the kind's table entry is `None`, and admitted at + /// the latest version, whose entry publishes the kind's bounds. + #[test] + fn validate_base_structure_v0_gates_erase_by_protocol_version() { + let batch = make_batch_v0(vec![make_erase(1, 1)]); + + let released = PlatformVersion::get(14).expect("protocol version 14 exists"); + assert!( + released + .dpp + .state_transition_serialization_versions + .document_erase_state_transition + .is_none(), + "the test needs a released version that publishes no erase bounds" + ); + let result = batch + .validate_base_structure_v0(released) + .expect("no protocol err"); + assert!( + result.errors.iter().any(|error| matches!( + error, + ConsensusError::BasicError(BasicError::UnsupportedVersionError(_)) + )), + "protocol version 14 must reject an erase as an unsupported version, got {:?}", + result.errors + ); + + let result = batch + .validate_base_structure_v0(PlatformVersion::latest()) + .expect("no protocol err"); + assert!( + result.is_valid(), + "the latest protocol version must admit an erase, got {:?}", + result.errors + ); + } + + /// A batch may name one document at most once, whatever the kinds: the + /// duplicate finder the structure check runs fingerprints an erase by its + /// document type and id like every other kind, so an erase of a document + /// that another transition in the batch names is reported with it. + #[test] + fn validate_base_structure_v0_treats_an_erase_of_a_named_document_as_a_duplicate() { + let erase = make_erase(1, 7); + let delete = make_delete(2, 7); + let other_document = make_create(3); + + let transitions = vec![&erase, &delete, &other_document]; + let duplicates = find_duplicates_by_id(&transitions, PlatformVersion::latest()) + .expect("no protocol err"); + + assert_eq!( + duplicates.len(), + 2, + "the erase and the delete name the same document, got {duplicates:?}" + ); + assert!(duplicates + .iter() + .any(|t| matches!(t, DocumentTransition::Erase(_)))); + } + + /// An erase in a batch is bounded like every other transition. + #[test] + fn validate_base_structure_v0_bounds_the_nonce_of_an_erase() { + let result = make_batch_v0(vec![make_erase(u64::MAX, 1)]) + .validate_base_structure_v0(PlatformVersion::latest()) + .expect("no protocol err"); + + assert!( + result.errors.iter().any(|error| matches!( + error, + ConsensusError::BasicError(BasicError::NonceOutOfBoundsError(_)) + )), + "expected NonceOutOfBoundsError, got {:?}", + result.errors + ); + } + // ----------------------------------------------------------------------- // indexOnlyDelete (delete-by-values kind) wire gate // ----------------------------------------------------------------------- diff --git a/packages/rs-dpp/src/validation/meta_validators/mod.rs b/packages/rs-dpp/src/validation/meta_validators/mod.rs index 55d2bd4f054..613a42860ed 100644 --- a/packages/rs-dpp/src/validation/meta_validators/mod.rs +++ b/packages/rs-dpp/src/validation/meta_validators/mod.rs @@ -52,6 +52,10 @@ lazy_static! { "../../../schema/meta_schemas/document/v3/document-meta.json" )) .expect("v3 document meta-schema JSON must be valid"); + static ref DOCUMENT_META_JSON_V4: Value = serde_json::from_str::(include_str!( + "../../../schema/meta_schemas/document/v4/document-meta.json" + )) + .expect("v4 document meta-schema JSON must be valid"); pub static ref DRAFT_202012_META_SCHEMA: JSONSchema = JSONSchema::options() .with_draft(Draft::Draft202012) @@ -299,6 +303,61 @@ lazy_static! { .compile(&DOCUMENT_META_JSON_V3) .expect("Invalid data contract schema"); + // Compiled version of document meta schema v4 + // Introduced for protocol version 15 (the keep-history document + // lifecycle). v3 plus the document-type keyword `canBeErased`, which lets + // a deleted keep-history document have its retained revisions purged by + // an erase transition. Hosting it on a schema only v15+ contracts validate + // against leaves v14 validation untouched: under v3 the key still fails + // `additionalProperties: false` on a document type. + pub static ref DOCUMENT_META_SCHEMA_V4: JSONSchema = JSONSchema::options() + .with_keyword( + "byteArray", + |_, _, _| Ok(Box::new(ByteArrayKeyword)), + ) + .with_patterns_regex_engine(RegexEngine::Regex(RegexOptions { + size_limit: Some(5 * (1 << 20)), + ..Default::default() + })) + .should_ignore_unknown_formats(false) + .should_validate_formats(true) + .with_draft(Draft::Draft202012) + .with_document( + "https://json-schema.org/draft/2020-12/meta/applicator".to_string(), + DRAFT202012_APPLICATOR.clone(), + ) + .with_document( + "https://json-schema.org/draft/2020-12/meta/core".to_string(), + DRAFT202012_CORE.clone(), + ) + .with_document( + "https://json-schema.org/draft/2020-12/meta/unevaluated".to_string(), + DRAFT202012_UNEVALUATED.clone(), + ) + .with_document( + "https://json-schema.org/draft/2020-12/meta/validation".to_string(), + DRAFT202012_VALIDATION.clone(), + ) + .with_document( + "https://json-schema.org/draft/2020-12/meta/meta-data".to_string(), + DRAFT202012_META_DATA.clone(), + ) + .with_document( + "https://json-schema.org/draft/2020-12/meta/format-annotation".to_string(), + DRAFT202012_FORMAT_ANNOTATION.clone(), + ) + .with_document( + "https://json-schema.org/draft/2020-12/meta/content".to_string(), + DRAFT202012_CONTENT.clone(), + ) + .with_document( + "https://json-schema.org/draft/2020-12/schema".to_string(), + DRAFT202012.clone(), + ) + .to_owned() + .compile(&DOCUMENT_META_JSON_V4) + .expect("Invalid data contract schema"); + } #[cfg(test)] diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_create_transition_action/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_create_transition_action/mod.rs index 6d67acc456c..5719164e323 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_create_transition_action/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_create_transition_action/mod.rs @@ -11,6 +11,7 @@ use crate::execution::types::state_transition_execution_context::StateTransition use crate::execution::validation::state_transition::batch::action_validation::document::document_create_transition_action::state_v0::DocumentCreateTransitionActionStateValidationV0; use crate::execution::validation::state_transition::batch::action_validation::document::document_create_transition_action::state_v1::DocumentCreateTransitionActionStateValidationV1; use crate::execution::validation::state_transition::batch::action_validation::document::document_create_transition_action::state_v2::DocumentCreateTransitionActionStateValidationV2; +use crate::execution::validation::state_transition::batch::action_validation::document::document_create_transition_action::state_v3::DocumentCreateTransitionActionStateValidationV3; use crate::execution::validation::state_transition::batch::action_validation::document::document_create_transition_action::advanced_structure_v0::DocumentCreateTransitionActionStructureValidationV0; use crate::execution::validation::state_transition::batch::action_validation::document::document_create_transition_action::advanced_structure_v1::DocumentCreateTransitionActionStructureValidationV1; use crate::platform_types::platform::PlatformStateRef; @@ -20,6 +21,7 @@ mod advanced_structure_v1; mod state_v0; mod state_v1; mod state_v2; +mod state_v3; pub trait DocumentCreateTransitionActionValidation { fn validate_structure( @@ -111,9 +113,18 @@ impl DocumentCreateTransitionActionValidation for DocumentCreateTransitionAction transaction, platform_version, ), + // V3 refuses the id of a deleted or erasing keep-history document on top of V2 + 3 => self.validate_state_v3( + platform, + owner_id, + block_info, + execution_context, + transaction, + platform_version, + ), version => Err(Error::Execution(ExecutionError::UnknownVersionMismatch { method: "DocumentCreateTransitionAction::validate_state".to_string(), - known_versions: vec![0, 1, 2], + known_versions: vec![0, 1, 2, 3], received: version, })), } diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_create_transition_action/state_v3/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_create_transition_action/state_v3/mod.rs new file mode 100644 index 00000000000..56abe4492d4 --- /dev/null +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_create_transition_action/state_v3/mod.rs @@ -0,0 +1,99 @@ +use dpp::block::block_info::BlockInfo; +use dpp::consensus::basic::document::InvalidDocumentTypeError; +use dpp::consensus::state::document::document_already_present_error::DocumentAlreadyPresentError; +use dpp::consensus::state::state_error::StateError; +use dpp::consensus::ConsensusError; +use dpp::data_contract::accessors::v0::DataContractV0Getters; +use dpp::data_contract::document_type::accessors::DocumentTypeV0Getters; +use dpp::identifier::Identifier; +use dpp::validation::SimpleConsensusValidationResult; +use dpp::version::PlatformVersion; +use drive::query::TransactionArg; +use drive::state_transition_action::batch::batched_transition::document_transition::document_base_transition_action::DocumentBaseTransitionActionAccessorsV0; +use drive::state_transition_action::batch::batched_transition::document_transition::document_create_transition_action::{ + DocumentCreateTransitionAction, DocumentCreateTransitionActionAccessorsV0, +}; + +use crate::error::Error; +use crate::execution::types::state_transition_execution_context::StateTransitionExecutionContext; +use crate::execution::validation::state_transition::batch::action_validation::document::document_create_transition_action::state_v2::DocumentCreateTransitionActionStateValidationV2; +use crate::execution::validation::state_transition::batch::state::fetch_keep_history_document_lifecycle::fetch_keep_history_document_lifecycle; +use crate::platform_types::platform::PlatformStateRef; + +pub(in crate::execution::validation::state_transition::state_transitions::batch::action_validation) trait DocumentCreateTransitionActionStateValidationV3 +{ + fn validate_state_v3( + &self, + platform: &PlatformStateRef, + owner_id: Identifier, + block_info: &BlockInfo, + execution_context: &mut StateTransitionExecutionContext, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result; +} + +impl DocumentCreateTransitionActionStateValidationV3 for DocumentCreateTransitionAction { + /// V3 is V2 plus the keep-history document lifecycle: a deleted or erasing + /// keep-history document keeps its id, so its id is not free to create. + fn validate_state_v3( + &self, + platform: &PlatformStateRef, + owner_id: Identifier, + block_info: &BlockInfo, + execution_context: &mut StateTransitionExecutionContext, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result { + let validation_result = self.validate_state_v2( + platform, + owner_id, + block_info, + execution_context, + transaction, + platform_version, + )?; + if !validation_result.is_valid() { + return Ok(validation_result); + } + + // A keep-history document that has been deleted keeps its revisions and + // keeps its id: nothing in the primary-key tree marks the id as taken, + // so the by-id read of the earlier generations cannot see the + // reservation. Creating over it would append a new document's revision + // to a deleted one's record. The id becomes free again once an erase + // has removed the last revision. + let contract_fetch_info = self.base().data_contract_fetch_info(); + let contract = &contract_fetch_info.contract; + let document_type_name = self.base().document_type_name(); + let Some(document_type) = contract.document_type_optional_for_name(document_type_name) + else { + return Ok(SimpleConsensusValidationResult::new_with_error( + InvalidDocumentTypeError::new(document_type_name.clone(), contract.id()).into(), + )); + }; + if !document_type.documents_keep_history() { + return Ok(SimpleConsensusValidationResult::new()); + } + + let lifecycle = fetch_keep_history_document_lifecycle( + platform.drive, + contract, + document_type, + self.base().id(), + &block_info.epoch, + execution_context, + transaction, + platform_version, + )?; + if lifecycle.is_present() { + return Ok(SimpleConsensusValidationResult::new_with_error( + ConsensusError::StateError(StateError::DocumentAlreadyPresentError( + DocumentAlreadyPresentError::new(self.base().id()), + )), + )); + } + + Ok(SimpleConsensusValidationResult::new()) + } +} diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_delete_transition_action/advanced_structure_v2/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_delete_transition_action/advanced_structure_v2/mod.rs new file mode 100644 index 00000000000..3e51bf51a1e --- /dev/null +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_delete_transition_action/advanced_structure_v2/mod.rs @@ -0,0 +1,84 @@ +use dpp::consensus::basic::document::{InvalidDocumentTransitionActionError, InvalidDocumentTypeError}; +use dpp::data_contract::accessors::v0::DataContractV0Getters; +use dpp::data_contract::document_type::accessors::{DocumentTypeV0Getters, DocumentTypeV2Getters}; +use dpp::validation::SimpleConsensusValidationResult; +use drive::state_transition_action::batch::batched_transition::document_transition::document_base_transition_action::DocumentBaseTransitionActionAccessorsV0; +use drive::state_transition_action::batch::batched_transition::document_transition::document_delete_transition_action::DocumentDeleteTransitionAction; +use drive::state_transition_action::batch::batched_transition::document_transition::document_delete_transition_action::v0::DocumentDeleteTransitionActionAccessorsV0; + +use crate::error::Error; + +pub(in crate::execution::validation::state_transition::state_transitions::batch::action_validation) trait DocumentDeleteTransitionActionStructureValidationV2 { + fn validate_structure_v2(&self) -> Result; +} +impl DocumentDeleteTransitionActionStructureValidationV2 for DocumentDeleteTransitionAction { + /// From protocol 15 a keep-history type takes part in the document + /// lifecycle: a delete removes the document from ordinary reads and leaves + /// its revisions readable, so v2 no longer refuses a delete because the + /// type keeps history. What it refuses instead is a keep-history type that + /// carries a contested index, which the parser rejects for new contracts + /// but which a contract stored before protocol 15 could still declare. A + /// contested resource is awarded outside transition validation, at an id + /// derived from the winner rather than from the contested values, so that + /// award can land on an id whose retained history already exists. + fn validate_structure_v2(&self) -> Result { + let contract_fetch_info = self.base().data_contract_fetch_info(); + let data_contract = &contract_fetch_info.contract; + let document_type_name = self.base().document_type_name(); + + // Make sure that the document type is defined in the contract + let Some(document_type) = data_contract.document_type_optional_for_name(document_type_name) + else { + return Ok(SimpleConsensusValidationResult::new_with_error( + InvalidDocumentTypeError::new(document_type_name.clone(), data_contract.id()) + .into(), + )); + }; + + if !document_type.documents_can_be_deleted() { + return Ok(SimpleConsensusValidationResult::new_with_error( + InvalidDocumentTransitionActionError::new(format!( + "documents of type {} can not be deleted", + document_type_name + )) + .into(), + )); + } + + if document_type.documents_keep_history() + && document_type + .indexes() + .values() + .any(|index| index.contested_index.is_some()) + { + return Ok(SimpleConsensusValidationResult::new_with_error( + InvalidDocumentTransitionActionError::new(format!( + "documents of keep-history type {} carry a contested index and can not take \ + part in the document lifecycle", + document_type_name + )) + .into(), + )); + } + + // Pair the delete KIND with the doctype's storage mode: an + // indexOnly document has no primary-storage row a by-id delete + // could fetch, so its deletes must come as the indexOnlyDelete + // (delete-by-values) kind — its structure validation enforces the + // mirror rule. `index_only()` can only be true on a PV14+ + // contract, so this branch is unreachable for every historical + // transition. + if document_type.index_only() { + return Ok(SimpleConsensusValidationResult::new_with_error( + InvalidDocumentTransitionActionError::new(format!( + "documents of indexOnly type {} must be deleted with an indexOnlyDelete \ + (delete-by-values) transition carrying the document's values", + document_type_name + )) + .into(), + )); + } + + Ok(SimpleConsensusValidationResult::new()) + } +} diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_delete_transition_action/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_delete_transition_action/mod.rs index 35b5210125a..b97b62eb975 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_delete_transition_action/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_delete_transition_action/mod.rs @@ -8,13 +8,17 @@ use crate::error::Error; use crate::error::execution::ExecutionError; use crate::execution::types::state_transition_execution_context::StateTransitionExecutionContext; use crate::execution::validation::state_transition::batch::action_validation::document::document_delete_transition_action::state_v0::DocumentDeleteTransitionActionStateValidationV0; +use crate::execution::validation::state_transition::batch::action_validation::document::document_delete_transition_action::state_v1::DocumentDeleteTransitionActionStateValidationV1; use crate::execution::validation::state_transition::batch::action_validation::document::document_delete_transition_action::advanced_structure_v0::DocumentDeleteTransitionActionStructureValidationV0; use crate::execution::validation::state_transition::batch::action_validation::document::document_delete_transition_action::advanced_structure_v1::DocumentDeleteTransitionActionStructureValidationV1; +use crate::execution::validation::state_transition::batch::action_validation::document::document_delete_transition_action::advanced_structure_v2::DocumentDeleteTransitionActionStructureValidationV2; use crate::platform_types::platform::PlatformStateRef; mod advanced_structure_v0; mod advanced_structure_v1; +mod advanced_structure_v2; mod state_v0; +mod state_v1; pub trait DocumentDeleteTransitionActionValidation { fn validate_structure( @@ -47,9 +51,10 @@ impl DocumentDeleteTransitionActionValidation for DocumentDeleteTransitionAction { 0 => self.validate_structure_v0(), 1 => self.validate_structure_v1(), + 2 => self.validate_structure_v2(), version => Err(Error::Execution(ExecutionError::UnknownVersionMismatch { method: "DocumentDeleteTransitionAction::validate_structure".to_string(), - known_versions: vec![0, 1], + known_versions: vec![0, 1, 2], received: version, })), } @@ -79,9 +84,17 @@ impl DocumentDeleteTransitionActionValidation for DocumentDeleteTransitionAction transaction, platform_version, ), + 1 => self.validate_state_v1( + platform, + owner_id, + block_info, + execution_context, + transaction, + platform_version, + ), version => Err(Error::Execution(ExecutionError::UnknownVersionMismatch { method: "DocumentDeleteTransitionAction::validate_state".to_string(), - known_versions: vec![0], + known_versions: vec![0, 1], received: version, })), } diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_delete_transition_action/state_v1/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_delete_transition_action/state_v1/mod.rs new file mode 100644 index 00000000000..f3de337bb2d --- /dev/null +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_delete_transition_action/state_v1/mod.rs @@ -0,0 +1,134 @@ +use crate::error::Error; +use crate::execution::types::state_transition_execution_context::StateTransitionExecutionContext; +use crate::execution::validation::state_transition::batch::action_validation::document::document_base_transaction_action::DocumentBaseTransitionActionValidation; +use crate::execution::validation::state_transition::batch::action_validation::document::document_delete_transition_action::state_v0::DocumentDeleteTransitionActionStateValidationV0; +use crate::execution::validation::state_transition::batch::state::fetch_keep_history_document_lifecycle::fetch_keep_history_document_lifecycle; +use crate::platform_types::platform::PlatformStateRef; +use dpp::block::block_info::BlockInfo; +use dpp::consensus::basic::document::InvalidDocumentTypeError; +use dpp::consensus::state::document::document_not_found_error::DocumentNotFoundError; +use dpp::consensus::state::state_error::StateError; +use dpp::consensus::ConsensusError; +use dpp::data_contract::accessors::v0::DataContractV0Getters; +use dpp::data_contract::document_type::accessors::DocumentTypeV0Getters; +use dpp::document::DocumentV0Getters; +use dpp::identifier::Identifier; +use dpp::prelude::ConsensusValidationResult; +use dpp::validation::SimpleConsensusValidationResult; +use dpp::version::PlatformVersion; +use drive::drive::document::lifecycle::DocumentLifecycleState; +use drive::grovedb::TransactionArg; +use drive::state_transition_action::batch::batched_transition::document_transition::document_base_transition_action::DocumentBaseTransitionActionAccessorsV0; +use drive::state_transition_action::batch::batched_transition::document_transition::document_delete_transition_action::v0::DocumentDeleteTransitionActionAccessorsV0; +use drive::state_transition_action::batch::batched_transition::document_transition::document_delete_transition_action::DocumentDeleteTransitionAction; + +use dpp::consensus::state::document::document_owner_id_mismatch_error::DocumentOwnerIdMismatchError; + +pub(in crate::execution::validation::state_transition::state_transitions::batch::action_validation) trait DocumentDeleteTransitionActionStateValidationV1 { + fn validate_state_v1( + &self, + platform: &PlatformStateRef, + owner_id: Identifier, + block_info: &BlockInfo, + execution_context: &mut StateTransitionExecutionContext, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result; +} + +impl DocumentDeleteTransitionActionStateValidationV1 for DocumentDeleteTransitionAction { + /// A keep-history document that has already been deleted is invisible to + /// the ordinary by-id read v0 relies on, so deleting it a second time would + /// otherwise look like deleting a document that never existed. The + /// lifecycle read tells the two apart, and a second delete is refused + /// rather than escalating into anything that removes a revision: only an + /// erase does that, and only after this delete has committed the document + /// to the deleted state. + /// + /// Every other document type is validated exactly as v0 validates it. + fn validate_state_v1( + &self, + platform: &PlatformStateRef, + owner_id: Identifier, + block_info: &BlockInfo, + execution_context: &mut StateTransitionExecutionContext, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result { + let contract_fetch_info = self.base().data_contract_fetch_info(); + let contract = &contract_fetch_info.contract; + let document_type_name = self.base().document_type_name(); + let Some(document_type) = contract.document_type_optional_for_name(document_type_name) + else { + return Ok(SimpleConsensusValidationResult::new_with_error( + InvalidDocumentTypeError::new(document_type_name.clone(), contract.id()).into(), + )); + }; + + if !document_type.documents_keep_history() { + return self.validate_state_v0( + platform, + owner_id, + block_info, + execution_context, + transaction, + platform_version, + ); + } + + let validation_result = self.base().validate_state( + platform, + owner_id, + block_info, + "delete", + execution_context, + transaction, + platform_version, + )?; + if !validation_result.is_valid() { + return Ok(validation_result); + } + + let lifecycle = fetch_keep_history_document_lifecycle( + platform.drive, + contract, + document_type, + self.base().id(), + &block_info.epoch, + execution_context, + transaction, + platform_version, + )?; + + let document = match lifecycle { + DocumentLifecycleState::Active(document) => document, + // A deleted or erasing document is gone from every ordinary read, + // which is exactly what a delete asks for, so asking again is the + // same as asking about an id that holds nothing. + DocumentLifecycleState::Deleted(_) + | DocumentLifecycleState::Erasing + | DocumentLifecycleState::Absent => { + return Ok(ConsensusValidationResult::new_with_error( + ConsensusError::StateError(StateError::DocumentNotFoundError( + DocumentNotFoundError::new(self.base().id()), + )), + )); + } + }; + + // The single ownership call site for a delete. + if document.owner_id() != owner_id { + return Ok(SimpleConsensusValidationResult::new_with_error( + ConsensusError::StateError(StateError::DocumentOwnerIdMismatchError( + DocumentOwnerIdMismatchError::new( + self.base().id(), + owner_id, + document.owner_id(), + ), + )), + )); + } + + Ok(SimpleConsensusValidationResult::new()) + } +} diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_erase_transition_action/advanced_structure_v0/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_erase_transition_action/advanced_structure_v0/mod.rs new file mode 100644 index 00000000000..d514a99a167 --- /dev/null +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_erase_transition_action/advanced_structure_v0/mod.rs @@ -0,0 +1,78 @@ +use crate::error::Error; +use dpp::consensus::basic::document::{ + InvalidDocumentTransitionActionError, InvalidDocumentTypeError, +}; +use dpp::data_contract::accessors::v0::DataContractV0Getters; +use dpp::data_contract::document_type::accessors::{DocumentTypeV0Getters, DocumentTypeV3Getters}; +use dpp::validation::SimpleConsensusValidationResult; +use drive::state_transition_action::batch::batched_transition::document_transition::document_base_transition_action::DocumentBaseTransitionActionAccessorsV0; +use drive::state_transition_action::batch::batched_transition::document_transition::document_erase_transition_action::v0::DocumentEraseTransitionActionAccessorsV0; +use drive::state_transition_action::batch::batched_transition::document_transition::document_erase_transition_action::DocumentEraseTransitionAction; + +pub(in crate::execution::validation::state_transition::state_transitions::batch::action_validation) trait DocumentEraseTransitionActionStructureValidationV0 { + fn validate_structure_v0(&self) -> Result; +} + +impl DocumentEraseTransitionActionStructureValidationV0 for DocumentEraseTransitionAction { + /// Checks what the contract alone decides: whether this type has retained + /// revisions at all, whether it allows them to be purged, and whether it is + /// one of the types the lifecycle keeps out. Whether the particular document + /// is in a state that can be erased is a question about committed state and + /// belongs to state validation. + fn validate_structure_v0(&self) -> Result { + let contract_fetch_info = self.base().data_contract_fetch_info(); + let data_contract = &contract_fetch_info.contract; + let document_type_name = self.base().document_type_name(); + + let Some(document_type) = data_contract.document_type_optional_for_name(document_type_name) + else { + return Ok(SimpleConsensusValidationResult::new_with_error( + InvalidDocumentTypeError::new(document_type_name.clone(), data_contract.id()) + .into(), + )); + }; + + if !document_type.documents_keep_history() { + return Ok(SimpleConsensusValidationResult::new_with_error( + InvalidDocumentTransitionActionError::new(format!( + "documents of type {} do not keep history and can not be erased", + document_type_name + )) + .into(), + )); + } + + if !document_type.documents_can_be_erased() { + return Ok(SimpleConsensusValidationResult::new_with_error( + InvalidDocumentTransitionActionError::new(format!( + "documents of type {} can not be erased", + document_type_name + )) + .into(), + )); + } + + // Defensive, for a contract stored before protocol 15: the parser keeps + // the two apart for every contract registered since. + if document_type + .indexes() + .values() + .any(|index| index.contested_index.is_some()) + { + return Ok(SimpleConsensusValidationResult::new_with_error( + InvalidDocumentTransitionActionError::new(format!( + "documents of keep-history type {} carry a contested index and can not take \ + part in the document lifecycle", + document_type_name + )) + .into(), + )); + } + + // Erase has no token cost of its own: the deletion it follows was paid + // for when the document was deleted. A transition that offers to pay + // one is refused where the offer is still visible, when the transition + // becomes an action. + Ok(SimpleConsensusValidationResult::new()) + } +} diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_erase_transition_action/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_erase_transition_action/mod.rs new file mode 100644 index 00000000000..a98788c0d28 --- /dev/null +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_erase_transition_action/mod.rs @@ -0,0 +1,94 @@ +use crate::error::execution::ExecutionError; +use crate::error::Error; +use crate::execution::types::state_transition_execution_context::StateTransitionExecutionContext; +use crate::execution::validation::state_transition::batch::action_validation::document::document_erase_transition_action::advanced_structure_v0::DocumentEraseTransitionActionStructureValidationV0; +use crate::execution::validation::state_transition::batch::action_validation::document::document_erase_transition_action::state_v0::DocumentEraseTransitionActionStateValidationV0; +use crate::platform_types::platform::PlatformStateRef; +use dpp::block::block_info::BlockInfo; +use dpp::identifier::Identifier; +use dpp::validation::SimpleConsensusValidationResult; +use dpp::version::PlatformVersion; +use drive::grovedb::TransactionArg; +use drive::state_transition_action::batch::batched_transition::document_transition::document_erase_transition_action::DocumentEraseTransitionAction; + +mod advanced_structure_v0; +mod state_v0; + +pub trait DocumentEraseTransitionActionValidation { + fn validate_structure( + &self, + platform_version: &PlatformVersion, + ) -> Result; + + fn validate_state( + &self, + platform: &PlatformStateRef, + owner_id: Identifier, + block_info: &BlockInfo, + execution_context: &mut StateTransitionExecutionContext, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result; +} + +impl DocumentEraseTransitionActionValidation for DocumentEraseTransitionAction { + fn validate_structure( + &self, + platform_version: &PlatformVersion, + ) -> Result { + match platform_version + .drive_abci + .validation_and_processing + .state_transitions + .batch_state_transition + .document_erase_transition_structure_validation + { + Some(0) => self.validate_structure_v0(), + Some(version) => Err(Error::Execution(ExecutionError::UnknownVersionMismatch { + method: "DocumentEraseTransitionAction::validate_structure".to_string(), + known_versions: vec![0], + received: version, + })), + None => Err(Error::Execution(ExecutionError::VersionNotActive { + method: "DocumentEraseTransitionAction::validate_structure".to_string(), + known_versions: vec![0], + })), + } + } + + fn validate_state( + &self, + platform: &PlatformStateRef, + owner_id: Identifier, + block_info: &BlockInfo, + execution_context: &mut StateTransitionExecutionContext, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result { + match platform_version + .drive_abci + .validation_and_processing + .state_transitions + .batch_state_transition + .document_erase_transition_state_validation + { + Some(0) => self.validate_state_v0( + platform, + owner_id, + block_info, + execution_context, + transaction, + platform_version, + ), + Some(version) => Err(Error::Execution(ExecutionError::UnknownVersionMismatch { + method: "DocumentEraseTransitionAction::validate_state".to_string(), + known_versions: vec![0], + received: version, + })), + None => Err(Error::Execution(ExecutionError::VersionNotActive { + method: "DocumentEraseTransitionAction::validate_state".to_string(), + known_versions: vec![0], + })), + } + } +} diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_erase_transition_action/state_v0/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_erase_transition_action/state_v0/mod.rs new file mode 100644 index 00000000000..6285f88b77c --- /dev/null +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/document_erase_transition_action/state_v0/mod.rs @@ -0,0 +1,143 @@ +use crate::error::Error; +use crate::execution::types::execution_operation::ValidationOperation; +use crate::execution::types::state_transition_execution_context::{ + StateTransitionExecutionContext, StateTransitionExecutionContextMethodsV0, +}; +use crate::execution::validation::state_transition::batch::action_validation::document::document_base_transaction_action::DocumentBaseTransitionActionValidation; +use crate::execution::validation::state_transition::batch::state::fetch_keep_history_document_lifecycle::fetch_keep_history_document_lifecycle; +use crate::platform_types::platform::PlatformStateRef; +use dpp::block::block_info::BlockInfo; +use dpp::consensus::basic::document::{ + InvalidDocumentTransitionActionError, InvalidDocumentTypeError, +}; +use dpp::consensus::state::document::document_not_found_error::DocumentNotFoundError; +use dpp::consensus::state::document::document_owner_id_mismatch_error::DocumentOwnerIdMismatchError; +use dpp::consensus::state::state_error::StateError; +use dpp::consensus::ConsensusError; +use dpp::data_contract::accessors::v0::DataContractV0Getters; +use dpp::document::DocumentV0Getters; +use dpp::identifier::Identifier; +use dpp::validation::SimpleConsensusValidationResult; +use dpp::version::PlatformVersion; +use drive::drive::document::lifecycle::DocumentLifecycleState; +use drive::grovedb::TransactionArg; +use drive::state_transition_action::batch::batched_transition::document_transition::document_base_transition_action::DocumentBaseTransitionActionAccessorsV0; +use drive::state_transition_action::batch::batched_transition::document_transition::document_erase_transition_action::v0::DocumentEraseTransitionActionAccessorsV0; +use drive::state_transition_action::batch::batched_transition::document_transition::document_erase_transition_action::DocumentEraseTransitionAction; + +pub(in crate::execution::validation::state_transition::state_transitions::batch::action_validation) trait DocumentEraseTransitionActionStateValidationV0 { + fn validate_state_v0( + &self, + platform: &PlatformStateRef, + owner_id: Identifier, + block_info: &BlockInfo, + execution_context: &mut StateTransitionExecutionContext, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result; +} + +impl DocumentEraseTransitionActionStateValidationV0 for DocumentEraseTransitionAction { + /// An erasure is authorized once, when it starts. + /// + /// A document that is merely deleted may only be committed to erasure by + /// its owner. Once committed, the record left in state is itself the + /// evidence that destruction was authorized, so any identity may submit the + /// remaining chunks and pay for them: an owner who loses their keys, their + /// funds or their permission can no longer strand a half-erased document. + /// A document that is still current cannot be erased at all — deleting it + /// is a separate intent, with its own permission and its own token cost. + fn validate_state_v0( + &self, + platform: &PlatformStateRef, + owner_id: Identifier, + block_info: &BlockInfo, + execution_context: &mut StateTransitionExecutionContext, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result { + let validation_result = self.base().validate_state( + platform, + owner_id, + block_info, + "erase", + execution_context, + transaction, + platform_version, + )?; + if !validation_result.is_valid() { + return Ok(validation_result); + } + + let contract_fetch_info = self.base().data_contract_fetch_info(); + let contract = &contract_fetch_info.contract; + let document_type_name = self.base().document_type_name(); + let Some(document_type) = contract.document_type_optional_for_name(document_type_name) + else { + return Ok(SimpleConsensusValidationResult::new_with_error( + InvalidDocumentTypeError::new(document_type_name.clone(), contract.id()).into(), + )); + }; + + let lifecycle = fetch_keep_history_document_lifecycle( + platform.drive, + contract, + document_type, + self.base().id(), + &block_info.epoch, + execution_context, + transaction, + platform_version, + )?; + + // Removing revisions credits whoever paid for each of them, and those + // balance updates are applied after this transition's fee result is + // formed, against identities that had nothing to do with it. The chunk + // bound limits how many there can be; this is what pays for them. It is + // billed on every path the erase can take, including the refusals, + // because the estimate that admits the transition is formed before the + // lifecycle is known. + execution_context.add_operation(ValidationOperation::PrecalculatedOperation( + platform + .drive + .erase_refund_recipient_cost(&block_info.epoch, platform_version) + .map_err(Error::Drive)?, + )); + + match lifecycle { + DocumentLifecycleState::Active(_) => { + Ok(SimpleConsensusValidationResult::new_with_error( + InvalidDocumentTransitionActionError::new(format!( + "document {} must be deleted before it can be erased", + self.base().id() + )) + .into(), + )) + } + // The single ownership call site for an erase. + DocumentLifecycleState::Deleted(newest) => { + if newest.owner_id() != owner_id { + return Ok(SimpleConsensusValidationResult::new_with_error( + ConsensusError::StateError(StateError::DocumentOwnerIdMismatchError( + DocumentOwnerIdMismatchError::new( + self.base().id(), + owner_id, + newest.owner_id(), + ), + )), + )); + } + Ok(SimpleConsensusValidationResult::new()) + } + // A continuation of an erasure that is already committed: nothing + // to authorize, because the committed record already carries the + // authorization. + DocumentLifecycleState::Erasing => Ok(SimpleConsensusValidationResult::new()), + DocumentLifecycleState::Absent => Ok(SimpleConsensusValidationResult::new_with_error( + ConsensusError::StateError(StateError::DocumentNotFoundError( + DocumentNotFoundError::new(self.base().id()), + )), + )), + } + } +} diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/mod.rs index 3a4130219e7..57f2d3199ad 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/action_validation/document/mod.rs @@ -1,6 +1,7 @@ mod document_base_transaction_action; pub(crate) mod document_create_transition_action; pub(crate) mod document_delete_transition_action; +pub(crate) mod document_erase_transition_action; pub(crate) mod document_index_only_delete_transition_action; pub(crate) mod document_purchase_transition_action; pub(crate) mod document_reference_validation; diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/advanced_structure/v0/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/advanced_structure/v0/mod.rs index 0eafc05f797..0fc65514003 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/advanced_structure/v0/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/advanced_structure/v0/mod.rs @@ -20,11 +20,13 @@ use dpp::version::PlatformVersion; use drive::state_transition_action::batch::BatchTransitionAction; use crate::execution::validation::state_transition::state_transitions::batch::action_validation::document::document_replace_transition_action::DocumentReplaceTransitionActionValidation; use crate::execution::validation::state_transition::state_transitions::batch::action_validation::document::document_delete_transition_action::DocumentDeleteTransitionActionValidation; +use crate::execution::validation::state_transition::state_transitions::batch::action_validation::document::document_erase_transition_action::DocumentEraseTransitionActionValidation; use crate::execution::validation::state_transition::state_transitions::batch::action_validation::document::document_index_only_delete_transition_action::DocumentIndexOnlyDeleteTransitionActionValidation; use crate::execution::validation::state_transition::state_transitions::batch::action_validation::document::document_create_transition_action::DocumentCreateTransitionActionValidation; use dpp::state_transition::batch_transition::document_create_transition::v0::v0_methods::DocumentCreateTransitionV0Methods; use drive::state_transition_action::batch::batched_transition::BatchedTransitionAction; use drive::state_transition_action::batch::batched_transition::document_transition::document_delete_transition_action::v0::DocumentDeleteTransitionActionAccessorsV0; +use drive::state_transition_action::batch::batched_transition::document_transition::document_erase_transition_action::v0::DocumentEraseTransitionActionAccessorsV0; use drive::state_transition_action::batch::batched_transition::document_transition::document_index_only_delete_transition_action::v0::DocumentIndexOnlyDeleteTransitionActionAccessorsV0; use drive::state_transition_action::batch::batched_transition::document_transition::document_purchase_transition_action::DocumentPurchaseTransitionActionAccessorsV0; use drive::state_transition_action::batch::batched_transition::document_transition::document_replace_transition_action::DocumentReplaceTransitionActionAccessorsV0; @@ -217,6 +219,19 @@ impl DocumentsBatchStateTransitionStructureValidationV0 for BatchTransition { )); } } + DocumentTransitionAction::EraseAction(erase_action) => { + let result = erase_action.validate_structure(platform_version)?; + if !result.is_valid() { + let bump_action = StateTransitionAction::BumpIdentityDataContractNonceAction( + BumpIdentityDataContractNonceAction::from_borrowed_document_base_transition_action(erase_action.base(), self.owner_id(), self.user_fee_increase()), + ); + + return Ok(ConsensusValidationResult::new_with_data_and_errors( + bump_action, + result.errors, + )); + } + } DocumentTransitionAction::IndexOnlyDeleteAction(index_only_delete_action) => { let result = index_only_delete_action.validate_structure(platform_version)?; diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/state/fetch_keep_history_document_lifecycle/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/state/fetch_keep_history_document_lifecycle/mod.rs new file mode 100644 index 00000000000..610e51097fc --- /dev/null +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/state/fetch_keep_history_document_lifecycle/mod.rs @@ -0,0 +1,56 @@ +mod v0; + +use crate::error::execution::ExecutionError; +use crate::error::Error; +use crate::execution::types::state_transition_execution_context::StateTransitionExecutionContext; +use dpp::block::epoch::Epoch; +use dpp::data_contract::document_type::DocumentTypeRef; +use dpp::data_contract::DataContract; +use dpp::identifier::Identifier; +use dpp::version::PlatformVersion; +use drive::drive::document::lifecycle::DocumentLifecycleState; +use drive::drive::Drive; +use drive::grovedb::TransactionArg; + +use self::v0::fetch_keep_history_document_lifecycle_v0; + +/// Classifies a keep-history document and bills the reads it performs. +#[allow(clippy::too_many_arguments)] +pub(crate) fn fetch_keep_history_document_lifecycle( + drive: &Drive, + contract: &DataContract, + document_type: DocumentTypeRef, + id: Identifier, + epoch: &Epoch, + execution_context: &mut StateTransitionExecutionContext, + transaction: TransactionArg, + platform_version: &PlatformVersion, +) -> Result { + match platform_version + .drive_abci + .validation_and_processing + .state_transitions + .batch_state_transition + .fetch_keep_history_document_lifecycle + { + Some(0) => fetch_keep_history_document_lifecycle_v0( + drive, + contract, + document_type, + id, + epoch, + execution_context, + transaction, + platform_version, + ), + Some(version) => Err(Error::Execution(ExecutionError::UnknownVersionMismatch { + method: "fetch_keep_history_document_lifecycle".to_string(), + known_versions: vec![0], + received: version, + })), + None => Err(Error::Execution(ExecutionError::VersionNotActive { + method: "fetch_keep_history_document_lifecycle".to_string(), + known_versions: vec![0], + })), + } +} diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/state/fetch_keep_history_document_lifecycle/v0/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/state/fetch_keep_history_document_lifecycle/v0/mod.rs new file mode 100644 index 00000000000..5a22011a383 --- /dev/null +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/state/fetch_keep_history_document_lifecycle/v0/mod.rs @@ -0,0 +1,38 @@ +use crate::error::Error; +use crate::execution::types::execution_operation::ValidationOperation; +use crate::execution::types::state_transition_execution_context::{ + StateTransitionExecutionContext, StateTransitionExecutionContextMethodsV0, +}; +use dpp::block::epoch::Epoch; +use dpp::data_contract::document_type::DocumentTypeRef; +use dpp::data_contract::DataContract; +use dpp::identifier::Identifier; +use dpp::version::PlatformVersion; +use drive::drive::document::lifecycle::DocumentLifecycleState; +use drive::drive::Drive; +use drive::grovedb::TransactionArg; + +#[allow(clippy::too_many_arguments)] +pub(super) fn fetch_keep_history_document_lifecycle_v0( + drive: &Drive, + contract: &DataContract, + document_type: DocumentTypeRef, + id: Identifier, + epoch: &Epoch, + execution_context: &mut StateTransitionExecutionContext, + transaction: TransactionArg, + platform_version: &PlatformVersion, +) -> Result { + let (state, fee_result) = drive + .fetch_document_lifecycle( + contract, + document_type, + id, + Some(epoch), + transaction, + platform_version, + ) + .map_err(Error::Drive)?; + execution_context.add_operation(ValidationOperation::PrecalculatedOperation(fee_result)); + Ok(state) +} diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/state/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/state/mod.rs index 008be12cc67..c464bb94fa5 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/state/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/state/mod.rs @@ -1,2 +1,3 @@ +pub(crate) mod fetch_keep_history_document_lifecycle; pub(crate) mod v0; pub(crate) mod v1; diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/state/v0/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/state/v0/mod.rs index 4fd93181846..c2de31db690 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/state/v0/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/state/v0/mod.rs @@ -17,6 +17,7 @@ use crate::error::execution::ExecutionError; use crate::execution::types::state_transition_execution_context::StateTransitionExecutionContext; use crate::execution::validation::state_transition::batch::action_validation::document::document_create_transition_action::DocumentCreateTransitionActionValidation; use crate::execution::validation::state_transition::batch::action_validation::document::document_delete_transition_action::DocumentDeleteTransitionActionValidation; +use crate::execution::validation::state_transition::batch::action_validation::document::document_erase_transition_action::DocumentEraseTransitionActionValidation; use crate::execution::validation::state_transition::batch::action_validation::document::document_index_only_delete_transition_action::DocumentIndexOnlyDeleteTransitionActionValidation; use crate::execution::validation::state_transition::batch::action_validation::document::document_purchase_transition_action::DocumentPurchaseTransitionActionValidation; use crate::execution::validation::state_transition::batch::action_validation::document::document_replace_transition_action::DocumentReplaceTransitionActionValidation; @@ -144,6 +145,15 @@ impl DocumentsBatchStateTransitionStateValidationV0 for BatchTransition { transaction, platform_version, )?, + DocumentTransitionAction::EraseAction(erase_action) => erase_action + .validate_state( + platform, + owner_id, + block_info, + execution_context, + transaction, + platform_version, + )?, DocumentTransitionAction::IndexOnlyDeleteAction(index_only_delete_action) => { index_only_delete_action.validate_state( platform, diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/deletion.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/deletion.rs index 5d9165a286b..5387104ec18 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/deletion.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/deletion.rs @@ -5,6 +5,9 @@ mod deletion_tests { use crate::execution::validation::state_transition::tests::create_card_game_internal_token_contract_with_owner_identity_burn_tokens; use dpp::tokens::token_payment_info::v0::TokenPaymentInfoV0; use dpp::tokens::token_payment_info::TokenPaymentInfo; + use drive::drive::document::lifecycle::DocumentLifecycleState; + use drive::drive::document::query::QueryDocumentsOutcomeV0Methods; + use drive::query::DriveDocumentQuery; #[tokio::test] async fn test_document_delete_on_document_type_that_is_mutable_and_can_be_deleted() { @@ -388,36 +391,70 @@ mod deletion_tests { assert_eq!(processing_result.aggregated_fees().processing_fee, 445700); } - /// PROTOCOL_VERSION_14 rejects deletes against contradictory keep-history - /// document types as invalid-paid consensus errors. + /// What a delete of a keep-history document does at each protocol version. + #[derive(Clone, Copy, PartialEq, Eq, Debug)] + enum KeepHistoryDeleteOutcome { + /// The released generations before protocol 14 hit the storage guard, + /// an internal error the block loop reports as such. + InternalError, + /// Protocol 14 refuses the delete as a paid consensus error. + PaidRejection, + /// Protocol 15 carries the delete out: the document leaves every + /// ordinary read and its retained revisions stay readable. + Deleted, + } + + #[tokio::test] + async fn test_document_delete_on_document_type_that_keeps_history_succeeds_protocol_version_15() + { + run_document_delete_on_document_type_that_keeps_history_at_protocol_version( + 15, + KeepHistoryDeleteOutcome::Deleted, + ) + .await; + } + + /// PROTOCOL_VERSION_14 rejects deletes against keep-history document types + /// as invalid-paid consensus errors and must keep doing so for replay. #[tokio::test] async fn test_document_delete_on_document_type_that_keeps_history_is_rejected_protocol_version_14( ) { - run_document_delete_on_document_type_that_keeps_history_at_protocol_version(14, true).await; + run_document_delete_on_document_type_that_keeps_history_at_protocol_version( + 14, + KeepHistoryDeleteOutcome::PaidRejection, + ) + .await; } /// PROTOCOL_VERSION_12 preserves the historical InternalError result for - /// replay compatibility. The keep-history structure guard must not run. + /// replay compatibility. The keep-history delete path must not run. #[tokio::test] async fn test_document_delete_on_document_type_that_keeps_history_replays_protocol_version_12() { - run_document_delete_on_document_type_that_keeps_history_at_protocol_version(12, false) - .await; + run_document_delete_on_document_type_that_keeps_history_at_protocol_version( + 12, + KeepHistoryDeleteOutcome::InternalError, + ) + .await; } #[tokio::test] async fn test_document_delete_on_document_type_that_keeps_history_replays_protocol_version_13() { - run_document_delete_on_document_type_that_keeps_history_at_protocol_version(13, false) - .await; + run_document_delete_on_document_type_that_keeps_history_at_protocol_version( + 13, + KeepHistoryDeleteOutcome::InternalError, + ) + .await; } - /// Exercises an already-deployed contradictory contract at both sides of - /// the v14 validation-version boundary. Loading with `full_validation: - /// false` is intentional: reparsing deployed contracts must remain allowed. + /// Exercises an already-deployed keep-history contract on every side of the + /// v14 and v15 validation-version boundaries. Loading with + /// `full_validation: false` is intentional: reparsing deployed contracts + /// must remain allowed. async fn run_document_delete_on_document_type_that_keeps_history_at_protocol_version( protocol_version: dpp::version::ProtocolVersion, - expect_invalid_paid: bool, + expected_outcome: KeepHistoryDeleteOutcome, ) { let platform_version = PlatformVersion::get(protocol_version) .expect("expected platform version for the requested protocol_version"); @@ -428,9 +465,9 @@ mod deletion_tests { let contract_path = "tests/supporting_files/contract/note/note-contract-keep-history-and-can-be-deleted.json"; - // `full_validation: false` bypasses the DPP cross-flag check so the - // intentionally-contradictory fixture loads — mirrors the - // already-deployed-contract scenario this guard is meant to handle. + // `full_validation: false` so the same fixture loads at every protocol + // version under test, including the released ones whose parser + // generation predates the keep-history grammar this contract uses. let note_contract = json_document_to_contract(contract_path, false, platform_version) .expect("expected to get data contract"); platform @@ -461,7 +498,7 @@ mod deletion_tests { ); assert!( note_document_type.documents_can_be_deleted(), - "fixture sanity: doctype must advertise canBeDeleted" + "fixture sanity: doctype must allow deletion" ); let entropy = Bytes32::random_with_rng(&mut rng); @@ -479,9 +516,9 @@ mod deletion_tests { let mut altered_document = document.clone(); altered_document.set_revision(Some(1)); + let deleted_document_id = altered_document.id(); - // Create the document (must succeed — keep-history doctypes accept - // creates, the contradiction only bites at delete time). + // Create the document. let documents_batch_create_transition = BatchTransition::new_document_creation_transition_from_document( document, @@ -526,8 +563,6 @@ mod deletion_tests { .unwrap() .expect("expected to commit transaction"); - // V14 rejects during structure validation; v12 and v13 retain the - // historical InternalError classification for replay. let documents_batch_deletion_transition = BatchTransition::new_document_deletion_transition_from_document( altered_document, @@ -569,13 +604,17 @@ mod deletion_tests { .unwrap() .expect("expected to commit transaction"); + assert_eq!(processing_result.invalid_unpaid_count(), 0); assert_eq!( processing_result.invalid_paid_count(), - usize::from(expect_invalid_paid), + usize::from(expected_outcome == KeepHistoryDeleteOutcome::PaidRejection), "unexpected invalid-paid classification at protocol version {protocol_version}" ); - assert_eq!(processing_result.invalid_unpaid_count(), 0); - assert_eq!(processing_result.valid_count(), 0); + assert_eq!( + processing_result.valid_count(), + usize::from(expected_outcome == KeepHistoryDeleteOutcome::Deleted), + "unexpected success classification at protocol version {protocol_version}" + ); let internal_error_count = processing_result .execution_results() .iter() @@ -583,10 +622,11 @@ mod deletion_tests { .count(); assert_eq!( internal_error_count, - usize::from(!expect_invalid_paid), + usize::from(expected_outcome == KeepHistoryDeleteOutcome::InternalError), "unexpected InternalError classification at protocol version {protocol_version}" ); - if expect_invalid_paid { + + if expected_outcome == KeepHistoryDeleteOutcome::PaidRejection { assert_matches!( processing_result.execution_results().as_slice(), [StateTransitionExecutionResult::PaidConsensusError { @@ -597,6 +637,39 @@ mod deletion_tests { }] if error.action() == "documents of type note can not be deleted" ); } + + if expected_outcome == KeepHistoryDeleteOutcome::Deleted { + // The document is gone from ordinary reads, and its retained + // revisions are not. + let documents = platform + .drive + .query_documents( + DriveDocumentQuery::all_items_query(¬e_contract, note_document_type, None), + None, + false, + None, + Some(protocol_version), + ) + .expect("expected to query documents") + .documents_owned(); + assert!( + documents.is_empty(), + "the deleted document is still visible" + ); + + let (lifecycle, _) = platform + .drive + .fetch_document_lifecycle( + ¬e_contract, + note_document_type, + deleted_document_id, + None, + None, + platform_version, + ) + .expect("expected to read the lifecycle"); + assert_matches!(lifecycle, DocumentLifecycleState::Deleted(_)); + } } #[tokio::test] diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/erase.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/erase.rs new file mode 100644 index 00000000000..7e85f3a4d61 --- /dev/null +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/erase.rs @@ -0,0 +1,1147 @@ +//! Deleting and erasing keep-history documents through signed transitions. +//! +//! Erasure is authorized once. The document's owner commits it, and the record +//! that commitment leaves in state is what lets any identity finish the work, +//! so an owner who loses their keys, their funds or their permission cannot +//! strand a half-erased document. + +use super::*; +use crate::rpc::core::MockCoreRPCLike; +use crate::test::helpers::setup::TempPlatform; +use dpp::consensus::basic::BasicError; +use dpp::data_contract::document_type::accessors::DocumentTypeV3Getters; +use dpp::data_contract::document_type::DocumentTypeRef; +use dpp::data_contract::DataContract; +use dpp::document::Document; +use dpp::identifier::Identifier; +use dpp::identity::{Identity, IdentityPublicKey, SecurityLevel}; +use dpp::prelude::IdentityNonce; +use dpp::state_transition::batch_transition::methods::v0::DocumentsBatchTransitionMethodsV0; +use dpp::state_transition::StateTransition; +use dpp::tokens::token_payment_info::v0::TokenPaymentInfoV0; +use dpp::tokens::token_payment_info::TokenPaymentInfo; +use dpp::version::PlatformVersion; +use dpp::ProtocolError; +use drive::drive::document::lifecycle::DocumentLifecycleState; +use drive::query::document_history_drive_query::{ + DocumentHistoryDriveQuery, DocumentHistoryFilter, DocumentHistoryState, +}; +use drive::util::storage_flags::StorageFlags; +use rand::rngs::StdRng; +use rand::SeedableRng; +use simple_signer::signer::SimpleSigner; + +const ERASABLE_CONTRACT: &str = + "tests/supporting_files/contract/note/note-contract-keep-history-erasable.json"; + +fn process( + platform: &mut TempPlatform, + serialized: Vec, +) -> StateTransitionExecutionResult { + let state = platform.state.load(); + let version = state.current_platform_version().unwrap(); + let transaction = platform.drive.grove.start_transaction(); + let result = platform + .platform + .process_raw_state_transitions( + &[serialized], + &state, + &BlockInfo::default(), + &transaction, + version, + false, + None, + ) + .expect("expected transition processing"); + platform + .drive + .grove + .commit_transaction(transaction) + .unwrap() + .unwrap(); + assert_eq!(result.execution_results().len(), 1); + result.into_execution_results().remove(0) +} + +fn assert_successful(result: &StateTransitionExecutionResult, context: &str) { + assert_matches!( + result, + StateTransitionExecutionResult::SuccessfulExecution { .. }, + "{context}" + ); +} + +/// A platform at protocol 15 with the erasable note contract applied, one +/// identity owning a `note` document, and a second identity that owns nothing. +struct Fixture { + platform: TempPlatform, + contract: DataContract, + owner: Identity, + owner_key: IdentityPublicKey, + owner_signer: SimpleSigner, + stranger: Identity, + stranger_key: IdentityPublicKey, + stranger_signer: SimpleSigner, + document: Document, + entropy: Bytes32, + nonce: IdentityNonce, + stranger_nonce: IdentityNonce, +} + +impl Fixture { + async fn new(document_type_name: &str) -> Self { + let platform_version = PlatformVersion::latest(); + let mut platform = TestPlatformBuilder::new() + .with_initial_protocol_version(platform_version.protocol_version) + .build_with_mock_rpc() + .set_initial_state_structure(); + + let contract = json_document_to_contract(ERASABLE_CONTRACT, true, platform_version) + .expect("the erasable note contract must pass full validation"); + platform + .drive + .apply_contract( + &contract, + BlockInfo::default(), + true, + StorageFlags::optional_default_as_cow(), + None, + platform_version, + ) + .expect("expected to apply the contract"); + + let (owner, owner_signer, owner_key) = + setup_identity(&mut platform, 1001, dash_to_credits!(1.0)); + let (stranger, stranger_signer, stranger_key) = + setup_identity(&mut platform, 1002, dash_to_credits!(1.0)); + + let mut rng = StdRng::seed_from_u64(1291); + let document_type = contract + .document_type_for_name(document_type_name) + .expect("expected the document type"); + let entropy = Bytes32::random_with_rng(&mut rng); + let document = document_type + .random_document_with_identifier_and_entropy( + &mut rng, + owner.id(), + entropy, + DocumentFieldFillType::FillIfNotRequired, + DocumentFieldFillSize::AnyDocumentFillSize, + platform_version, + ) + .expect("expected a random note"); + + let mut fixture = Fixture { + platform, + contract, + owner, + owner_key, + owner_signer, + stranger, + stranger_key, + stranger_signer, + document: document.clone(), + entropy, + nonce: 1, + stranger_nonce: 1, + }; + fixture.create(document, entropy, document_type_name).await; + fixture + } + + fn document_type(&self, name: &str) -> DocumentTypeRef<'_> { + self.contract + .document_type_for_name(name) + .expect("expected the document type") + } + + async fn create(&mut self, document: Document, entropy: Bytes32, document_type_name: &str) { + let platform_version = PlatformVersion::latest(); + let transition = BatchTransition::new_document_creation_transition_from_document( + document, + self.document_type(document_type_name), + entropy.0, + &self.owner_key, + self.nonce, + 0, + None, + &self.owner_signer, + platform_version, + None, + ) + .await + .expect("expected a create transition"); + self.nonce += 1; + let result = process( + &mut self.platform, + transition.serialize_to_bytes().expect("serialized"), + ); + assert_successful(&result, "the create must succeed"); + } + + async fn delete_as_owner( + &mut self, + document_type_name: &str, + ) -> StateTransitionExecutionResult { + let platform_version = PlatformVersion::latest(); + let mut document = self.document.clone(); + document.set_revision(Some(1)); + let transition = BatchTransition::new_document_deletion_transition_from_document( + document, + self.document_type(document_type_name), + &self.owner_key, + self.nonce, + 0, + None, + &self.owner_signer, + platform_version, + None, + ) + .await + .expect("expected a delete transition"); + self.nonce += 1; + process( + &mut self.platform, + transition.serialize_to_bytes().expect("serialized"), + ) + } + + async fn erase( + &mut self, + document_type_name: &str, + as_owner: bool, + token_payment_info: Option, + ) -> StateTransitionExecutionResult { + let platform_version = PlatformVersion::latest(); + let mut document = self.document.clone(); + document.set_revision(Some(1)); + if !as_owner { + document.set_owner_id(self.stranger.id()); + } + let transition = if let Some(token_payment_info) = token_payment_info { + // The erase factory refuses to carry token payment information, so + // a transition that does has to be built as a delete and rewritten + // into an erase, exactly as a hostile client would. + let delete = BatchTransition::new_document_deletion_transition_from_document( + document, + self.document_type(document_type_name), + &self.owner_key, + self.nonce, + 0, + Some(token_payment_info), + &self.owner_signer, + platform_version, + None, + ) + .await + .expect("expected a delete transition to rewrite"); + self.nonce += 1; + rewrite_delete_as_erase( + delete, + &self.owner_key, + &self.owner_signer, + self.document_type(document_type_name) + .security_level_requirement(), + ) + .await + } else if as_owner { + let transition = BatchTransition::new_document_erase_transition_from_document( + document, + self.document_type(document_type_name), + &self.owner_key, + self.nonce, + 0, + &self.owner_signer, + platform_version, + None, + ) + .await + .expect("expected an erase transition"); + self.nonce += 1; + transition + } else { + let transition = BatchTransition::new_document_erase_transition_from_document( + document, + self.document_type(document_type_name), + &self.stranger_key, + self.stranger_nonce, + 0, + &self.stranger_signer, + platform_version, + None, + ) + .await + .expect("expected an erase transition"); + self.stranger_nonce += 1; + transition + }; + process( + &mut self.platform, + transition.serialize_to_bytes().expect("serialized"), + ) + } + + /// Writes revisions straight into storage so the document retains more than + /// one erase chunk can remove. How they got there does not matter to an + /// erase: it reads what the history holds. + fn retain_revisions(&mut self, document_type_name: &str, revisions: u64) { + let platform_version = PlatformVersion::latest(); + let contract = self.contract.clone(); + let document_type = contract + .document_type_for_name(document_type_name) + .expect("expected the document type"); + // The same flags the create wrote, so overwriting the current pointer + // replaces exactly as many bytes as it holds. + let flags = Some(std::borrow::Cow::Owned(StorageFlags::new_single_epoch( + 0, + Some(self.owner.id().to_buffer()), + ))); + let mut document = self.document.clone(); + for revision in 2..=revisions { + document.set_revision(Some(revision)); + self.platform + .drive + .add_document_for_contract( + drive::util::object_size_info::DocumentAndContractInfo { + owned_document_info: drive::util::object_size_info::OwnedDocumentInfo { + document_info: + drive::util::object_size_info::DocumentInfo::DocumentRefInfo(( + &document, + flags.clone(), + )), + owner_id: None, + }, + contract: &contract, + document_type, + }, + true, + BlockInfo::default_with_time(1_000 + revision), + true, + None, + platform_version, + None, + ) + .expect("expected to retain a revision"); + } + } + + /// Builds an erase without processing it, so several can share one block. + async fn erase_transition(&mut self, document_type_name: &str, as_owner: bool) -> Vec { + let platform_version = PlatformVersion::latest(); + let mut document = self.document.clone(); + document.set_revision(Some(1)); + let (key, signer, nonce) = if as_owner { + let nonce = self.nonce; + self.nonce += 1; + (&self.owner_key, &self.owner_signer, nonce) + } else { + document.set_owner_id(self.stranger.id()); + let nonce = self.stranger_nonce; + self.stranger_nonce += 1; + (&self.stranger_key, &self.stranger_signer, nonce) + }; + BatchTransition::new_document_erase_transition_from_document( + document, + self.document_type(document_type_name), + key, + nonce, + 0, + signer, + platform_version, + None, + ) + .await + .expect("expected an erase transition") + .serialize_to_bytes() + .expect("serialized") + } + + fn remaining_revisions(&self, document_type_name: &str) -> u64 { + let query = DocumentHistoryDriveQuery { + contract_id: self.contract.id().to_buffer(), + document_type_name: document_type_name.to_string(), + document_id: self.document.id().to_buffer(), + filter: DocumentHistoryFilter::StartAtTime(0), + limit: Some(1), + }; + self.platform + .drive + .fetch_document_history( + &query, + self.document_type(document_type_name), + None, + PlatformVersion::latest(), + ) + .expect("expected to read the history") + .lifecycle + .expect("history carries lifecycle metadata") + .remaining_revisions + } + + fn history_state(&self, document_type_name: &str) -> DocumentHistoryState { + let query = DocumentHistoryDriveQuery { + contract_id: self.contract.id().to_buffer(), + document_type_name: document_type_name.to_string(), + document_id: self.document.id().to_buffer(), + filter: DocumentHistoryFilter::StartAtTime(0), + limit: Some(1), + }; + self.platform + .drive + .fetch_document_history( + &query, + self.document_type(document_type_name), + None, + PlatformVersion::latest(), + ) + .expect("expected to read the history") + .lifecycle + .expect("history carries lifecycle metadata") + .state + } + + fn lifecycle(&self, document_type_name: &str) -> DocumentLifecycleState { + self.platform + .drive + .fetch_document_lifecycle( + &self.contract, + self.document_type(document_type_name), + self.document.id(), + None, + None, + PlatformVersion::latest(), + ) + .expect("expected to read the lifecycle") + .0 + } +} + +/// Rebuilds a signed batch, swapping its single delete transition for an erase +/// carrying the same base — including the token payment information an erase +/// must refuse. +async fn rewrite_delete_as_erase( + transition: StateTransition, + key: &IdentityPublicKey, + signer: &SimpleSigner, + security_level: SecurityLevel, +) -> StateTransition { + use dpp::state_transition::batch_transition::batched_transition::document_erase_transition::DocumentEraseTransitionV0; + use dpp::state_transition::batch_transition::batched_transition::{ + BatchedTransition, DocumentEraseTransition, DocumentTransition, + }; + use dpp::state_transition::batch_transition::document_base_transition::document_base_transition_trait::DocumentBaseTransitionAccessors; + use dpp::state_transition::batch_transition::BatchTransition; + + let StateTransition::Batch(BatchTransition::V1(mut batch)) = transition else { + panic!("expected a format 1 batch transition"); + }; + let BatchedTransition::Document(DocumentTransition::Delete(delete)) = + batch.transitions.remove(0) + else { + panic!("expected a single document delete"); + }; + batch + .transitions + .push(BatchedTransition::Document(DocumentTransition::Erase( + DocumentEraseTransition::V0(DocumentEraseTransitionV0 { + base: delete.base().clone(), + }), + ))); + let mut rebuilt: StateTransition = BatchTransition::V1(batch).into(); + rebuilt + .sign_external( + key, + signer, + Some(|_: Identifier, _: String| Ok(security_level)), + ) + .await + .expect("expected to re-sign the rewritten batch"); + rebuilt +} + +#[tokio::test] +async fn should_delete_then_erase_a_keep_history_document() { + let mut fixture = Fixture::new("note").await; + assert_matches!(fixture.lifecycle("note"), DocumentLifecycleState::Active(_)); + + assert_successful( + &fixture.delete_as_owner("note").await, + "the delete must succeed", + ); + assert_matches!( + fixture.lifecycle("note"), + DocumentLifecycleState::Deleted(_) + ); + + assert_successful( + &fixture.erase("note", true, None).await, + "the erase must succeed", + ); + assert_matches!(fixture.lifecycle("note"), DocumentLifecycleState::Absent); +} + +/// A delete never escalates into anything that removes a revision, so asking +/// twice is a paid error rather than a second, deeper removal. +#[tokio::test] +async fn should_reject_a_second_delete_of_the_same_document() { + let mut fixture = Fixture::new("note").await; + assert_successful(&fixture.delete_as_owner("note").await, "the first delete"); + + let result = fixture.delete_as_owner("note").await; + assert_matches!( + result, + StateTransitionExecutionResult::PaidConsensusError { + error: ConsensusError::StateError(StateError::DocumentNotFoundError(_)), + .. + } + ); +} + +/// The id stays taken while the revisions are retained, even though nothing in +/// the primary-key tree says so. +#[tokio::test] +async fn should_reject_a_create_over_a_deleted_documents_id() { + let mut fixture = Fixture::new("note").await; + assert_successful(&fixture.delete_as_owner("note").await, "the delete"); + + let platform_version = PlatformVersion::latest(); + let document = fixture.document.clone(); + // The same id the deleted document had: a document id is derived from its + // creator, contract, type and entropy, so re-creating with the same inputs + // targets exactly the reserved id. + let entropy = fixture.entropy; + let transition = BatchTransition::new_document_creation_transition_from_document( + document, + fixture.document_type("note"), + entropy.0, + &fixture.owner_key, + fixture.nonce, + 0, + None, + &fixture.owner_signer, + platform_version, + None, + ) + .await + .expect("expected a create transition"); + let result = process( + &mut fixture.platform, + transition.serialize_to_bytes().expect("serialized"), + ); + assert_matches!( + result, + StateTransitionExecutionResult::PaidConsensusError { + error: ConsensusError::StateError(StateError::DocumentAlreadyPresentError(_)), + .. + } + ); +} + +/// Delete and erase stay two intents. Erasing a document that is still current +/// would let one signature do the work of two. +#[tokio::test] +async fn should_reject_an_erase_of_a_document_that_has_not_been_deleted() { + let mut fixture = Fixture::new("note").await; + + let result = fixture.erase("note", true, None).await; + assert_matches!( + result, + StateTransitionExecutionResult::PaidConsensusError { + error: ConsensusError::BasicError(BasicError::InvalidDocumentTransitionActionError(_)), + .. + } + ); + assert_matches!(fixture.lifecycle("note"), DocumentLifecycleState::Active(_)); +} + +/// A type that keeps history but never opted into erasure keeps its deleted +/// documents' revisions forever. +#[tokio::test] +async fn should_reject_an_erase_of_a_type_that_did_not_ask_for_it() { + let mut fixture = Fixture::new("permanentNote").await; + assert!(!fixture + .document_type("permanentNote") + .documents_can_be_erased()); + assert_successful( + &fixture.delete_as_owner("permanentNote").await, + "the delete must still succeed", + ); + + let result = fixture.erase("permanentNote", true, None).await; + assert_matches!( + result, + StateTransitionExecutionResult::PaidConsensusError { + error: ConsensusError::BasicError(BasicError::InvalidDocumentTransitionActionError(_)), + .. + } + ); + assert_matches!( + fixture.lifecycle("permanentNote"), + DocumentLifecycleState::Deleted(_) + ); +} + +/// Erase has no token cost of its own. Accepting one would let a continuation, +/// which any identity may submit, move somebody else's tokens. +#[tokio::test] +async fn should_reject_an_erase_that_carries_token_payment_information() { + let mut fixture = Fixture::new("note").await; + assert_successful(&fixture.delete_as_owner("note").await, "the delete"); + + let token_payment_info = TokenPaymentInfo::V0(TokenPaymentInfoV0 { + payment_token_contract_id: Some(Identifier::new([5u8; 32])), + token_contract_position: 0, + minimum_token_cost: None, + maximum_token_cost: Some(10), + gas_fees_paid_by: Default::default(), + }); + let result = fixture.erase("note", true, Some(token_payment_info)).await; + assert_matches!( + result, + StateTransitionExecutionResult::PaidConsensusError { + error: ConsensusError::BasicError(BasicError::InvalidDocumentTransitionActionError(_)), + .. + } + ); + assert_matches!( + fixture.lifecycle("note"), + DocumentLifecycleState::Deleted(_) + ); +} + +/// The first erase is the authorized act, so it must come from the owner. +#[tokio::test] +async fn should_reject_an_erase_start_by_an_identity_that_does_not_own_the_document() { + let mut fixture = Fixture::new("note").await; + assert_successful(&fixture.delete_as_owner("note").await, "the delete"); + + let result = fixture.erase("note", false, None).await; + assert_matches!( + result, + StateTransitionExecutionResult::PaidConsensusError { + error: ConsensusError::StateError(StateError::DocumentOwnerIdMismatchError(_)), + .. + } + ); + assert_matches!( + fixture.lifecycle("note"), + DocumentLifecycleState::Deleted(_) + ); +} + +/// A document already invisible to every read is not there to be deleted +/// again, whether or not its erasure has begun. +#[tokio::test] +async fn should_reject_a_delete_of_a_document_that_has_already_been_deleted() { + let mut fixture = Fixture::new("note").await; + assert_successful(&fixture.delete_as_owner("note").await, "the delete"); + assert_successful(&fixture.erase("note", true, None).await, "the erase"); + + let result = fixture.delete_as_owner("note").await; + assert_matches!( + result, + StateTransitionExecutionResult::PaidConsensusError { + error: ConsensusError::StateError(StateError::DocumentNotFoundError(_)), + .. + } + ); +} + +/// An erase of an id that holds nothing is not silently accepted. +#[tokio::test] +async fn should_reject_an_erase_of_an_id_that_holds_nothing() { + let mut fixture = Fixture::new("note").await; + assert_successful(&fixture.delete_as_owner("note").await, "the delete"); + assert_successful(&fixture.erase("note", true, None).await, "the erase"); + + let result = fixture.erase("note", true, None).await; + assert_matches!( + result, + StateTransitionExecutionResult::PaidConsensusError { + error: ConsensusError::StateError(StateError::DocumentNotFoundError(_)), + .. + } + ); +} + +/// Once an erasure is committed, the committed record is the authorization, so +/// an identity that owns nothing can finish the work. An owner who loses their +/// keys, their funds or their permission cannot strand a half-erased document. +#[tokio::test] +async fn should_let_any_identity_finish_an_erasure_its_owner_started() { + let chunk = PlatformVersion::latest() + .system_limits + .max_document_revisions_erased_per_transition + .expect("protocol 15 bounds the chunk") as u64; + + let mut fixture = Fixture::new("note").await; + fixture.retain_revisions("note", chunk + 1); + assert_eq!( + fixture.remaining_revisions("note"), + chunk + 1, + "the fixture must retain more than one chunk can remove" + ); + assert_successful(&fixture.delete_as_owner("note").await, "the delete"); + + // A stranger cannot start one. + assert_matches!( + fixture.erase("note", false, None).await, + StateTransitionExecutionResult::PaidConsensusError { + error: ConsensusError::StateError(StateError::DocumentOwnerIdMismatchError(_)), + .. + } + ); + + assert_successful( + &fixture.erase("note", true, None).await, + "the owner commits the erasure", + ); + assert_eq!(fixture.history_state("note"), DocumentHistoryState::Erasing); + + // And now the same stranger can finish it. + assert_successful( + &fixture.erase("note", false, None).await, + "a continuation needs no authorization of its own", + ); + assert_matches!(fixture.lifecycle("note"), DocumentLifecycleState::Absent); +} + +/// A document whose erasure has begun is not there to be deleted again. +#[tokio::test] +async fn should_reject_a_delete_of_a_document_whose_erasure_has_begun() { + let chunk = PlatformVersion::latest() + .system_limits + .max_document_revisions_erased_per_transition + .expect("protocol 15 bounds the chunk") as u64; + + let mut fixture = Fixture::new("note").await; + fixture.retain_revisions("note", chunk + 1); + assert_successful(&fixture.delete_as_owner("note").await, "the delete"); + assert_successful(&fixture.erase("note", true, None).await, "the erase start"); + assert_eq!(fixture.history_state("note"), DocumentHistoryState::Erasing); + + assert_matches!( + fixture.delete_as_owner("note").await, + StateTransitionExecutionResult::PaidConsensusError { + error: ConsensusError::StateError(StateError::DocumentNotFoundError(_)), + .. + } + ); +} + +/// Two erases in one block each see the previous one's effect, because each +/// transition is applied into the block transaction before the next is +/// validated. +#[tokio::test] +async fn should_finish_an_erasure_across_two_transitions_in_one_block() { + let chunk = PlatformVersion::latest() + .system_limits + .max_document_revisions_erased_per_transition + .expect("protocol 15 bounds the chunk") as u64; + + let mut fixture = Fixture::new("note").await; + fixture.retain_revisions("note", chunk + 1); + assert_successful(&fixture.delete_as_owner("note").await, "the delete"); + + let first = fixture.erase_transition("note", true).await; + let second = fixture.erase_transition("note", false).await; + + let state = fixture.platform.state.load(); + let version = state.current_platform_version().unwrap(); + let transaction = fixture.platform.drive.grove.start_transaction(); + let result = fixture + .platform + .platform + .process_raw_state_transitions( + &[first, second], + &state, + &BlockInfo::default(), + &transaction, + version, + false, + None, + ) + .expect("expected transition processing"); + fixture + .platform + .drive + .grove + .commit_transaction(transaction) + .unwrap() + .unwrap(); + + assert_eq!(result.valid_count(), 2, "both erases must execute"); + assert_matches!(fixture.lifecycle("note"), DocumentLifecycleState::Absent); +} + +/// A delete and the erase that follows it can share a block: the erase reads +/// the lifecycle record the delete has already written into the same +/// transaction. +#[tokio::test] +async fn should_delete_and_erase_in_one_block() { + let mut fixture = Fixture::new("note").await; + + let platform_version = PlatformVersion::latest(); + let mut document = fixture.document.clone(); + document.set_revision(Some(1)); + let delete = BatchTransition::new_document_deletion_transition_from_document( + document.clone(), + fixture.document_type("note"), + &fixture.owner_key, + fixture.nonce, + 0, + None, + &fixture.owner_signer, + platform_version, + None, + ) + .await + .expect("expected a delete transition") + .serialize_to_bytes() + .expect("serialized"); + fixture.nonce += 1; + let erase = fixture.erase_transition("note", true).await; + + let state = fixture.platform.state.load(); + let version = state.current_platform_version().unwrap(); + let transaction = fixture.platform.drive.grove.start_transaction(); + let result = fixture + .platform + .platform + .process_raw_state_transitions( + &[delete, erase], + &state, + &BlockInfo::default(), + &transaction, + version, + false, + None, + ) + .expect("expected transition processing"); + fixture + .platform + .drive + .grove + .commit_transaction(transaction) + .unwrap() + .unwrap(); + + assert_eq!( + result.valid_count(), + 2, + "the erase must see the delete that shares its block" + ); + assert_matches!(fixture.lifecycle("note"), DocumentLifecycleState::Absent); +} + +/// The balance updates an erase's refunds cause land outside its own fee +/// result, against identities that had nothing to do with it. The chunk bound +/// limits how many there can be; this pins that the submitter pays for them, +/// in the estimate that admits the transition and in the fee actually charged. +#[tokio::test] +async fn should_charge_an_erase_for_the_refund_recipients_it_can_credit() { + use dpp::block::epoch::Epoch; + + let mut fixture = Fixture::new("note").await; + assert_successful(&fixture.delete_as_owner("note").await, "the delete"); + + let platform_version = PlatformVersion::latest(); + let recipient_cost = fixture + .platform + .drive + .erase_refund_recipient_cost(&Epoch::new(0).unwrap(), platform_version) + .expect("expected the recipient work to be priced"); + assert!(recipient_cost.processing_fee > 0); + + let result = fixture.erase("note", true, None).await; + let StateTransitionExecutionResult::SuccessfulExecution { + estimated_fees, + fee_result, + .. + } = result + else { + panic!("expected the erase to succeed, got {result:?}"); + }; + let estimated_fees = estimated_fees.expect("an erase is admitted against an estimate"); + + assert!( + fee_result.processing_fee >= recipient_cost.processing_fee, + "the charged fee must include the recipient work: {} is below {}", + fee_result.processing_fee, + recipient_cost.processing_fee + ); + assert!( + estimated_fees.processing_fee >= recipient_cost.processing_fee, + "so must the estimate that admits it: {} is below {}", + estimated_fees.processing_fee, + recipient_cost.processing_fee + ); +} + +/// A paid failure has to leave its nonce bump behind, or the same rejected +/// transition can be replayed for free. The refusals the lifecycle adds are no +/// exception. +#[tokio::test] +async fn should_persist_the_nonce_bump_on_a_refused_erase() { + let mut fixture = Fixture::new("note").await; + + let nonce_after = |fixture: &Fixture| { + fixture + .platform + .drive + .fetch_identity_contract_nonce( + fixture.owner.id().to_buffer(), + fixture.contract.id().to_buffer(), + true, + None, + PlatformVersion::latest(), + ) + .expect("expected to read the identity contract nonce") + .expect("the create already wrote one") + }; + let before = nonce_after(&fixture); + + // An erase of a document that has not been deleted. + assert_matches!( + fixture.erase("note", true, None).await, + StateTransitionExecutionResult::PaidConsensusError { .. } + ); + let after_refusal = nonce_after(&fixture); + assert_eq!( + after_refusal, + before + 1, + "a refused erase must still consume its nonce" + ); + + // And an erase whose token payment is refused in the transformer, which is + // a different refusal path with its own nonce-bump action. + let token_payment_info = TokenPaymentInfo::V0(TokenPaymentInfoV0 { + payment_token_contract_id: Some(Identifier::new([5u8; 32])), + token_contract_position: 0, + minimum_token_cost: None, + maximum_token_cost: Some(10), + gas_fees_paid_by: Default::default(), + }); + assert_matches!( + fixture.erase("note", true, Some(token_payment_info)).await, + StateTransitionExecutionResult::PaidConsensusError { .. } + ); + assert_eq!( + nonce_after(&fixture), + after_refusal + 1, + "the transformer's refusal must consume its nonce too" + ); +} + +/// A delete and a create of the same id in one block: the create sees the +/// delete's reservation through the block transaction and is refused, so an id +/// cannot be recycled while its revisions are retained. +#[tokio::test] +async fn should_reject_a_create_that_follows_a_delete_of_the_same_id_in_one_block() { + let mut fixture = Fixture::new("note").await; + + let platform_version = PlatformVersion::latest(); + let mut document = fixture.document.clone(); + document.set_revision(Some(1)); + let delete = BatchTransition::new_document_deletion_transition_from_document( + document, + fixture.document_type("note"), + &fixture.owner_key, + fixture.nonce, + 0, + None, + &fixture.owner_signer, + platform_version, + None, + ) + .await + .expect("expected a delete transition") + .serialize_to_bytes() + .expect("serialized"); + fixture.nonce += 1; + + let create = BatchTransition::new_document_creation_transition_from_document( + fixture.document.clone(), + fixture.document_type("note"), + fixture.entropy.0, + &fixture.owner_key, + fixture.nonce, + 0, + None, + &fixture.owner_signer, + platform_version, + None, + ) + .await + .expect("expected a create transition") + .serialize_to_bytes() + .expect("serialized"); + fixture.nonce += 1; + + let state = fixture.platform.state.load(); + let version = state.current_platform_version().unwrap(); + let transaction = fixture.platform.drive.grove.start_transaction(); + let result = fixture + .platform + .platform + .process_raw_state_transitions( + &[delete, create], + &state, + &BlockInfo::default(), + &transaction, + version, + false, + None, + ) + .expect("expected transition processing"); + fixture + .platform + .drive + .grove + .commit_transaction(transaction) + .unwrap() + .unwrap(); + + assert_eq!(result.valid_count(), 1, "only the delete may execute"); + assert_eq!(result.invalid_paid_count(), 1, "the create must be refused"); + assert_matches!( + fixture.lifecycle("note"), + DocumentLifecycleState::Deleted(_) + ); +} + +/// Replacing a deleted document is refused for the same reason a second delete +/// is: nothing an ordinary read can see is there any more. This is the existing +/// not-found path, pinned here because the lifecycle is what makes the document +/// invisible while its revisions survive. +#[tokio::test] +async fn should_reject_a_replace_of_a_deleted_document() { + let mut fixture = Fixture::new("note").await; + assert_successful(&fixture.delete_as_owner("note").await, "the delete"); + + let platform_version = PlatformVersion::latest(); + let mut replacement = fixture.document.clone(); + replacement.set_revision(Some(2)); + let transition = BatchTransition::new_document_replacement_transition_from_document( + replacement, + fixture.document_type("note"), + &fixture.owner_key, + fixture.nonce, + 0, + None, + &fixture.owner_signer, + platform_version, + None, + ) + .await + .expect("expected a replace transition"); + fixture.nonce += 1; + + let result = process( + &mut fixture.platform, + transition.serialize_to_bytes().expect("serialized"), + ); + assert_matches!( + result, + StateTransitionExecutionResult::PaidConsensusError { + error: ConsensusError::StateError(StateError::DocumentNotFoundError(_)), + .. + } + ); + assert_matches!( + fixture.lifecycle("note"), + DocumentLifecycleState::Deleted(_) + ); +} + +/// The erase kind is appended to the shipped batch, so a node that knows it +/// decodes a format 1 batch carrying one at protocol 14 as well as at 15. +/// What keeps such a node agreeing with a node that cannot decode the kind is +/// the basic-structure gate: at protocol 14, where no bounds are published for +/// the kind, the batch is refused as an unsupported version before any state +/// is read or any fee is charged. +#[tokio::test] +async fn should_refuse_an_erase_at_protocol_14_as_an_unsupported_version() { + let released = PlatformVersion::get(14).expect("protocol version 14 exists"); + assert!( + released + .dpp + .state_transition_serialization_versions + .document_erase_state_transition + .is_none(), + "the test needs a released version that publishes no erase bounds" + ); + let mut platform = TestPlatformBuilder::new() + .with_initial_protocol_version(released.protocol_version) + .build_with_mock_rpc() + .set_initial_state_structure(); + + // A deployed keep-history contract, loaded the way protocol 14 loads it. + let contract = json_document_to_contract( + "tests/supporting_files/contract/note/note-contract-keep-history-and-can-be-deleted.json", + false, + released, + ) + .expect("expected the keep-history note contract"); + platform + .drive + .apply_contract( + &contract, + BlockInfo::default(), + true, + StorageFlags::optional_default_as_cow(), + None, + released, + ) + .expect("expected to apply the contract"); + let (owner, signer, key) = setup_identity(&mut platform, 1003, dash_to_credits!(1.0)); + + // The transition itself can only be built by software that knows the + // kind; a protocol 14 node receives the bytes from such software. + let latest = PlatformVersion::latest(); + let document_type = contract + .document_type_for_name("note") + .expect("expected the note document type"); + let mut rng = StdRng::seed_from_u64(1303); + let entropy = Bytes32::random_with_rng(&mut rng); + let document = document_type + .random_document_with_identifier_and_entropy( + &mut rng, + owner.id(), + entropy, + DocumentFieldFillType::FillIfNotRequired, + DocumentFieldFillSize::AnyDocumentFillSize, + latest, + ) + .expect("expected a random note"); + let erase = BatchTransition::new_document_erase_transition_from_document( + document, + document_type, + &key, + 1, + 0, + &signer, + latest, + None, + ) + .await + .expect("expected an erase transition"); + assert_matches!( + erase, + StateTransition::Batch(dpp::state_transition::batch_transition::BatchTransition::V1(_)), + "the erase rides in the shipped batch format" + ); + + let result = process( + &mut platform, + erase.serialize_to_bytes().expect("serialized"), + ); + + assert_matches!( + result, + StateTransitionExecutionResult::UnpaidConsensusError( + dpp::consensus::ConsensusError::BasicError(BasicError::UnsupportedVersionError(_)) + ), + "protocol 14 must refuse the erase kind by its version bounds" + ); +} diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/lifecycle_contracts.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/lifecycle_contracts.rs new file mode 100644 index 00000000000..22636c59201 --- /dev/null +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/lifecycle_contracts.rs @@ -0,0 +1,303 @@ +//! Signed contracts carrying a combination the lifecycle refuses. +//! +//! The parser cannot produce a `DataContract` holding one, so these build a +//! valid contract, rewrite the schema the transition carries on the wire, and +//! re-sign — which is all a hostile client has to do. What the tests pin is the +//! error category: only a consensus error becomes a paid rejection with a nonce +//! bump, while a bare data-contract error escapes as an internal execution +//! error that costs the submitter nothing. + +use super::*; +use crate::rpc::core::MockCoreRPCLike; +use crate::test::helpers::setup::TempPlatform; +use dpp::data_contract::accessors::v0::DataContractV0Setters; +use dpp::data_contract::DataContractFactory; +use dpp::identifier::Identifier; +use dpp::identity::identity_public_key::accessors::v0::IdentityPublicKeyGettersV0; +use dpp::identity::SecurityLevel; +use dpp::platform_value::platform_value; +use dpp::prelude::IdentityNonce; +use dpp::state_transition::data_contract_create_transition::methods::DataContractCreateTransitionMethodsV0; +use dpp::state_transition::data_contract_create_transition::DataContractCreateTransition; +use dpp::state_transition::data_contract_update_transition::methods::DataContractUpdateTransitionMethodsV0; +use dpp::state_transition::data_contract_update_transition::DataContractUpdateTransition; +use dpp::state_transition::StateTransition; +use dpp::version::PlatformVersion; +use dpp::ProtocolError; +use drive::util::storage_flags::StorageFlags; + +/// A keep-history type the parser does admit, which every test below starts +/// from before rewriting the schema on the wire. +fn admissible_schema() -> Value { + platform_value!({ + "type": "object", + "documentsKeepHistory": true, + "documentsMutable": true, + "canBeDeleted": true, + "properties": { + "message": {"type": "string", "maxLength": 64, "position": 0}, + }, + "required": ["message"], + "additionalProperties": false, + }) +} + +/// Erasure asked for by a type with no history to erase. +fn erasure_without_history() -> Value { + platform_value!({ + "type": "object", + "documentsKeepHistory": false, + "documentsMutable": true, + "canBeDeleted": true, + "canBeErased": true, + "properties": { + "message": {"type": "string", "maxLength": 64, "position": 0}, + }, + "required": ["message"], + "additionalProperties": false, + }) +} + +/// A contested resource whose type also retains history. +fn contested_keep_history() -> Value { + platform_value!({ + "type": "object", + "documentsKeepHistory": true, + "documentsMutable": false, + "canBeDeleted": true, + "indices": [ + { + "name": "byMessage", + "properties": [{"message": "asc"}], + "unique": true, + "contested": { + "fieldMatches": [{"field": "message", "regexPattern": "^[a-z]{3,10}$"}], + "resolution": 0, + }, + }, + ], + "properties": { + "message": {"type": "string", "maxLength": 64, "position": 0}, + }, + "required": ["message"], + "additionalProperties": false, + }) +} + +async fn refused_create( + schema: Value, +) -> (TempPlatform, Identifier, IdentityNonce) { + let platform_version = PlatformVersion::latest(); + let mut platform = TestPlatformBuilder::new() + .with_initial_protocol_version(platform_version.protocol_version) + .build_with_mock_rpc() + .set_genesis_state(); + let platform_state = platform.state.load(); + let (identity, signer, key) = setup_identity(&mut platform, 4001, dash_to_credits!(10.0)); + + let contract = DataContractFactory::new(platform_version.protocol_version) + .expect("expected a contract factory") + .create_with_value_config( + identity.id(), + 1, + platform_value!({ "note": admissible_schema() }), + None, + None, + ) + .expect("the admissible type must parse") + .data_contract_owned(); + + let mut transition = DataContractCreateTransition::new_from_data_contract( + contract, + 1, + &identity.clone().into_partial_identity_info(), + key.id(), + &signer, + platform_version, + None, + ) + .await + .expect("expected a contract create transition"); + + // Rewrite the schema the wire carries; the parser refuses to build one. + let StateTransition::DataContractCreate(DataContractCreateTransition::V0(create)) = + &mut transition + else { + panic!("expected a v0 contract create"); + }; + create + .data_contract + .document_schemas_mut() + .insert("note".to_string(), schema); + transition + .sign_external( + &key, + &signer, + None:: Result>, + ) + .await + .expect("expected to re-sign the rewritten contract"); + + let transaction = platform.drive.grove.start_transaction(); + let processing_result = platform + .platform + .process_raw_state_transitions( + &[transition.serialize_to_bytes().expect("serialized")], + &platform_state, + &BlockInfo::default(), + &transaction, + platform_version, + false, + None, + ) + .expect("expected transition processing"); + platform + .drive + .grove + .commit_transaction(transaction) + .unwrap() + .unwrap(); + + assert_eq!( + processing_result.invalid_paid_count(), + 1, + "a refused contract must cost its submitter, not vanish into an internal error" + ); + assert_matches!( + processing_result.execution_results().as_slice(), + [StateTransitionExecutionResult::PaidConsensusError { .. }] + ); + + let nonce = platform + .drive + .fetch_identity_nonce(identity.id().to_buffer(), true, None, platform_version) + .expect("expected to read the identity nonce") + .expect("the nonce must be persisted"); + + (platform, identity.id(), nonce) +} + +#[tokio::test] +async fn should_charge_for_a_signed_contract_asking_for_erasure_without_history() { + let (_platform, _identity, nonce) = refused_create(erasure_without_history()).await; + assert_eq!( + nonce, 1, + "the refusal must persist the nonce bump, so the same transition cannot be replayed" + ); +} + +#[tokio::test] +async fn should_charge_for_a_signed_contested_keep_history_contract() { + let (_platform, _identity, nonce) = refused_create(contested_keep_history()).await; + assert_eq!(nonce, 1, "the refusal must persist the nonce bump"); +} + +/// The same refusal on the update path, which has its own error-category split. +#[tokio::test] +async fn should_charge_for_a_signed_contract_update_asking_for_erasure_without_history() { + let platform_version = PlatformVersion::latest(); + let mut platform = TestPlatformBuilder::new() + .with_initial_protocol_version(platform_version.protocol_version) + .build_with_mock_rpc() + .set_genesis_state(); + let platform_state = platform.state.load(); + let (identity, signer, key) = setup_identity(&mut platform, 4002, dash_to_credits!(1.0)); + + let mut contract = DataContractFactory::new(platform_version.protocol_version) + .expect("expected a contract factory") + .create_with_value_config( + identity.id(), + 1, + platform_value!({ "note": admissible_schema() }), + None, + None, + ) + .expect("the admissible type must parse") + .data_contract_owned(); + platform + .drive + .apply_contract( + &contract, + BlockInfo::default(), + true, + StorageFlags::optional_default_as_cow(), + None, + platform_version, + ) + .expect("expected to apply the contract"); + + let contract_id = contract.id(); + contract.set_version(2); + let mut transition = DataContractUpdateTransition::new_from_data_contract( + contract, + &identity.clone().into_partial_identity_info(), + key.id(), + 1, + 0, + &signer, + platform_version, + None, + ) + .await + .expect("expected a contract update transition"); + + let StateTransition::DataContractUpdate(DataContractUpdateTransition::V0(update)) = + &mut transition + else { + panic!("expected a v0 contract update"); + }; + update + .data_contract + .document_schemas_mut() + .insert("note".to_string(), erasure_without_history()); + transition + .sign_external( + &key, + &signer, + None:: Result>, + ) + .await + .expect("expected to re-sign the rewritten update"); + + let transaction = platform.drive.grove.start_transaction(); + let processing_result = platform + .platform + .process_raw_state_transitions( + &[transition.serialize_to_bytes().expect("serialized")], + &platform_state, + &BlockInfo::default(), + &transaction, + platform_version, + false, + None, + ) + .expect("expected transition processing"); + platform + .drive + .grove + .commit_transaction(transaction) + .unwrap() + .unwrap(); + + assert_eq!(processing_result.invalid_paid_count(), 1); + assert_matches!( + processing_result.execution_results().as_slice(), + [StateTransitionExecutionResult::PaidConsensusError { .. }] + ); + + let nonce = platform + .drive + .fetch_identity_contract_nonce( + identity.id().to_buffer(), + contract_id.to_buffer(), + true, + None, + platform_version, + ) + .expect("expected to read the identity contract nonce"); + assert_eq!( + nonce, + Some(1), + "the refusal must persist the contract nonce bump" + ); +} diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/mod.rs index a1a23bbcbe9..2ffb2fb5513 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/mod.rs @@ -1,8 +1,10 @@ mod creation; mod deletion; mod dpns; +mod erase; mod index_only; mod keep_history; +mod lifecycle_contracts; mod nft; mod ranked_group_drain; mod replacement; diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/transformer/v0/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/transformer/v0/mod.rs index 01480cab7fe..1634c94baa6 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/transformer/v0/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/transformer/v0/mod.rs @@ -56,6 +56,7 @@ use dpp::state_transition::batch_transition::batched_transition::document_purcha use dpp::state_transition::{StateTransitionHasUserFeeIncrease, StateTransitionOwned}; use drive::state_transition_action::batch::batched_transition::document_transition::document_create_transition_action::DocumentCreateTransitionAction; use drive::state_transition_action::batch::batched_transition::document_transition::document_delete_transition_action::DocumentDeleteTransitionAction; +use drive::state_transition_action::batch::batched_transition::document_transition::document_erase_transition_action::DocumentEraseTransitionAction; use drive::state_transition_action::batch::batched_transition::document_transition::document_index_only_delete_transition_action::DocumentIndexOnlyDeleteTransitionAction; use drive::state_transition_action::batch::batched_transition::document_transition::document_replace_transition_action::DocumentReplaceTransitionAction; use drive::state_transition_action::batch::BatchTransitionAction; @@ -856,6 +857,16 @@ impl BatchTransitionInternalTransformerV0 for BatchTransition { Ok(batched_action) } + DocumentTransition::Erase(document_erase_transition) => { + let (batched_action, fee_result) = DocumentEraseTransitionAction::try_from_document_borrowed_erase_transition_with_contract_lookup(document_erase_transition, owner_id, user_fee_increase, |_identifier| { + Ok(data_contract_fetch_info.clone()) + })?; + + execution_context + .add_operation(ValidationOperation::PrecalculatedOperation(fee_result)); + + Ok(batched_action) + } DocumentTransition::IndexOnlyDelete(document_index_only_delete_transition) => { let (batched_action, fee_result) = DocumentIndexOnlyDeleteTransitionAction::try_from_document_borrowed_index_only_delete_transition_with_contract_lookup(document_index_only_delete_transition, owner_id, user_fee_increase, |_identifier| { Ok(data_contract_fetch_info.clone()) diff --git a/packages/rs-drive-abci/src/query/document_history/v0/mod.rs b/packages/rs-drive-abci/src/query/document_history/v0/mod.rs index 94e1505bda1..fb5ec9dd51d 100644 --- a/packages/rs-drive-abci/src/query/document_history/v0/mod.rs +++ b/packages/rs-drive-abci/src/query/document_history/v0/mod.rs @@ -138,9 +138,15 @@ impl Platform { lifecycle: history.lifecycle.map(|lifecycle| Lifecycle { state: match lifecycle.state { DocumentHistoryState::Active => State::Active, + DocumentHistoryState::Deleted => State::Deleted, + DocumentHistoryState::Erasing => State::Erasing, DocumentHistoryState::Absent => State::Absent, } as i32, remaining_revisions: lifecycle.remaining_revisions, + deleted_at_ms: lifecycle.times.deleted_at_ms, + erasing_started_at_ms: lifecycle.times.erasing_started_at_ms, + erasing_from_time_ms: lifecycle.times.erasing_from_time_ms, + erasing_from_revision: lifecycle.times.erasing_from_revision, }), }) }; diff --git a/packages/rs-drive-abci/src/query/document_history/v0/tests.rs b/packages/rs-drive-abci/src/query/document_history/v0/tests.rs index f723b97c80b..453c93b8f0c 100644 --- a/packages/rs-drive-abci/src/query/document_history/v0/tests.rs +++ b/packages/rs-drive-abci/src/query/document_history/v0/tests.rs @@ -415,6 +415,8 @@ fn history_api_proof_round_trip(gapped: bool) { match expected_lifecycle.state { DocumentHistoryState::Active => State::Active, DocumentHistoryState::Absent => State::Absent, + DocumentHistoryState::Deleted => State::Deleted, + DocumentHistoryState::Erasing => State::Erasing, } as i32 ); let with_proofs = |mut response: GetDocumentHistoryResponseV0, @@ -577,3 +579,198 @@ fn should_reject_missing_selectors_before_reading_state() { .unwrap() .is_valid()); } + +#[test] +fn should_derive_deleted_and_erasing_lifecycle_from_the_proof() { + use dpp::block::extended_block_info::v0::ExtendedBlockInfoV0Setters; + + let (platform, state, version) = setup_platform(None, Network::Testnet, None); + let mut state = state.as_ref().clone(); + let contract = json_document_to_contract( + concat!( + env!("CARGO_MANIFEST_DIR"), + "/../rs-drive/tests/supporting_files/contract/dashpay/dashpay-contract-with-profile-history.json" + ), + false, + version, + ) + .unwrap(); + platform + .drive + .apply_contract(&contract, BlockInfo::default(), true, None, None, version) + .unwrap(); + let document_type = contract.document_type_for_name("profile").unwrap(); + let owner = Identifier::from([8; 32]); + let mut document = json_document_to_document( + concat!( + env!("CARGO_MANIFEST_DIR"), + "/../rs-drive/tests/supporting_files/contract/dashpay/profile0.json" + ), + Some(owner), + document_type, + version, + ) + .unwrap(); + let chunk = version + .system_limits + .max_document_revisions_erased_per_transition + .expect("protocol 15 bounds the erase chunk") as u64; + for revision in 1..=chunk + 2 { + document.set_revision(Some(revision)); + platform + .drive + .add_document_for_contract( + DocumentAndContractInfo { + owned_document_info: OwnedDocumentInfo { + document_info: DocumentInfo::DocumentRefInfo((&document, None)), + owner_id: None, + }, + contract: &contract, + document_type, + }, + revision > 1, + BlockInfo::default_with_time(1_000 + revision), + true, + None, + version, + None, + ) + .unwrap(); + } + + let key = SecretKey::::from_hash(b"document-history-quorum"); + let provider = Provider { + contract: Arc::new(contract.clone()), + key: key.public_key().to_bytes().try_into().unwrap(), + }; + state.last_committed_block_info = Some( + dpp::block::extended_block_info::v0::ExtendedBlockInfoV0 { + basic_info: BlockInfo { + height: 42, + core_height: 12, + time_ms: 3_000, + epoch: Default::default(), + }, + app_hash: [0; 32], + quorum_hash: [9; 32], + block_id_hash: [7; 32], + proposer_pro_tx_hash: [0; 32], + signature: [0; 96], + round: 0, + } + .into(), + ); + let wire_request = GetDocumentHistoryRequestV0 { + data_contract_id: contract.id().to_vec(), + document_type_name: "profile".into(), + document_id: document.id().to_vec(), + limit: None, + prove: true, + filter: Some(Filter::StartAtMs(0)), + }; + let request: GetDocumentHistoryRequest = wire_request.clone().into(); + + for (stage, expected_state) in [ + ("deleted", DocumentHistoryState::Deleted), + ("erasing", DocumentHistoryState::Erasing), + ] { + let operations = if stage == "deleted" { + platform + .drive + .delete_document_for_contract_operations( + document.id(), + &contract, + document_type, + &BlockInfo::default_with_time(5_000), + Some(owner), + None, + &mut None, + None, + version, + ) + .unwrap() + } else { + platform + .drive + .erase_document_for_contract_operations( + document.id(), + &contract, + document_type, + &BlockInfo::default_with_time(6_000), + &mut None, + None, + version, + ) + .unwrap() + }; + platform + .drive + .apply_batch_low_level_drive_operations( + None, + None, + operations, + &mut vec![], + &version.drive, + ) + .unwrap(); + + let root = platform + .drive + .grove + .root_hash(None, &version.drive.grove_version) + .value + .unwrap(); + let metadata = platform.response_metadata_v0(&state, CheckpointUsed::Current); + let signed = signed_proof( + vec![], + root, + &metadata, + &key, + platform.config.validator_set.quorum_type as u32, + ); + let committed = state.last_committed_block_info.as_mut().unwrap(); + committed.set_app_hash(root); + committed.set_signature(signed.signature.try_into().unwrap()); + + let response = platform + .query_document_history_v0(wire_request.clone(), &state, version) + .unwrap() + .into_data() + .unwrap(); + // A proved response carries the proof and nothing else: there are no + // lifecycle fields on the wire for a client to trust, so every value + // the verifier returns below is derived from the proof itself. + assert!( + matches!(response.result, Some(ResponseResult::Proof(_))), + "{stage}: a proved response must not carry lifecycle claims" + ); + let verify = |response: GetDocumentHistoryResponseV0| { + DocumentHistory::maybe_from_proof( + request.clone(), + GetDocumentHistoryResponse::from(response), + Network::Testnet, + version, + &provider, + ) + }; + let history = verify(response.clone()) + .unwrap_or_else(|error| panic!("{stage}: honest response must verify: {error}")) + .expect("the proof authenticates a history"); + let lifecycle = history + .lifecycle + .expect("the proof authenticates the lifecycle"); + assert_eq!(lifecycle.state, expected_state, "{stage}"); + assert_eq!(lifecycle.times.deleted_at_ms, 5_000, "{stage}"); + + let mut tampered = response; + let proof = proof_mut(&mut tampered); + let mut envelope = DocumentHistoryProof::from_bytes(&proof.grovedb_proof).unwrap(); + let middle = envelope.metadata_proof.len() / 2; + envelope.metadata_proof[middle] ^= 1; + proof.grovedb_proof = envelope.to_bytes().unwrap(); + assert!( + verify(tampered).is_err(), + "{stage}: changing the proof that authenticates lifecycle must be rejected" + ); + } +} diff --git a/packages/rs-drive-abci/tests/strategy_tests/verify_state_transitions.rs b/packages/rs-drive-abci/tests/strategy_tests/verify_state_transitions.rs index c45020d0a2b..fb51b793486 100644 --- a/packages/rs-drive-abci/tests/strategy_tests/verify_state_transitions.rs +++ b/packages/rs-drive-abci/tests/strategy_tests/verify_state_transitions.rs @@ -613,6 +613,14 @@ pub(crate) fn verify_state_transitions_were_or_were_not_executed( // we expect no document assert!(document.is_none()); } + DocumentTransitionAction::EraseAction(_) => { + // An erase acts on a document that was + // already deleted, so the by-id query this + // harness proves against returns nothing + // whether or not the erase executed; + // strategies do not generate erases today. + assert!(document.is_none()); + } DocumentTransitionAction::IndexOnlyDeleteAction(_) => { // indexOnly documents never have a // primary-storage row, so the by-id query diff --git a/packages/rs-drive-abci/tests/supporting_files/contract/note/note-contract-keep-history-erasable.json b/packages/rs-drive-abci/tests/supporting_files/contract/note/note-contract-keep-history-erasable.json new file mode 100644 index 00000000000..eb9cc08e987 --- /dev/null +++ b/packages/rs-drive-abci/tests/supporting_files/contract/note/note-contract-keep-history-erasable.json @@ -0,0 +1,36 @@ +{ + "$formatVersion": "1", + "id": "5Bqs6itzfoDXzmgQibYZQABbqYsXmawVf7SKe3mKDQVf", + "ownerId": "2b994p95akyNFKtkDnDvBRUotDbkH54MHwGbhQLr5gcU", + "version": 1, + "keywords": [], + "documentSchemas": { + "note": { + "type": "object", + "documentsKeepHistory": true, + "documentsMutable": true, + "canBeDeleted": true, + "canBeErased": true, + "properties": { + "message": { "type": "string", "maxLength": 256, "position": 0 } + }, + "required": ["message"], + "additionalProperties": false, + "$comment": "A keep-history type whose deleted documents may also have their retained revisions purged. Deleting a note removes it from ordinary reads and leaves its revisions readable; erasing removes those revisions a bounded chunk at a time." + }, + "permanentNote": { + "type": "object", + "documentsKeepHistory": true, + "documentsMutable": true, + "canBeDeleted": true, + "properties": { + "message": { "type": "string", "maxLength": 256, "position": 0 } + }, + "required": ["message"], + "additionalProperties": false, + "$comment": "The same type without canBeErased: its deleted documents keep their revisions forever, which is what the erase refusal is checked against." + } + }, + "groups": {}, + "tokens": {} +} diff --git a/packages/rs-drive-proof-verifier/src/types.rs b/packages/rs-drive-proof-verifier/src/types.rs index d8839998c15..f9633cf60e0 100644 --- a/packages/rs-drive-proof-verifier/src/types.rs +++ b/packages/rs-drive-proof-verifier/src/types.rs @@ -115,7 +115,8 @@ pub type RetrievedValues = IndexMap; /// Contains a map of data contract revisions to data contracts. pub type DataContractHistory = RetrievedValues; pub use drive::query::document_history_drive_query::{ - DocumentHistoryEntry, DocumentHistoryLifecycle, DocumentHistoryState, + DocumentHistoryEntry, DocumentHistoryLifecycle, DocumentHistoryLifecycleTimes, + DocumentHistoryState, }; /// Ordered document revisions with lifecycle metadata when the storage layout supports it. diff --git a/packages/rs-drive/src/drive/contract/insert/insert_contract/v0/tests/batched_group_drain.rs b/packages/rs-drive/src/drive/contract/insert/insert_contract/v0/tests/batched_group_drain.rs index 531272d06aa..d374b60b257 100644 --- a/packages/rs-drive/src/drive/contract/insert/insert_contract/v0/tests/batched_group_drain.rs +++ b/packages/rs-drive/src/drive/contract/insert/insert_contract/v0/tests/batched_group_drain.rs @@ -177,6 +177,7 @@ fn delete_op<'a>( ) -> DriveOperation<'a> { DriveOperation::DocumentOperation(DocumentOperationType::DeleteDocument { document_id, + deleter_id: None, contract_info: DataContractInfo::BorrowedDataContract(contract), document_type_info: DocumentTypeInfo::DocumentTypeNameAsStr(doctype), }) diff --git a/packages/rs-drive/src/drive/contract/update/update_keywords/v0/mod.rs b/packages/rs-drive/src/drive/contract/update/update_keywords/v0/mod.rs index 08f3168acd6..d6b840b4d37 100644 --- a/packages/rs-drive/src/drive/contract/update/update_keywords/v0/mod.rs +++ b/packages/rs-drive/src/drive/contract/update/update_keywords/v0/mod.rs @@ -146,9 +146,10 @@ impl Drive { *doc_id, &contract, document_type, + block_info, + None, None, estimated_costs_only_with_layer_info, - block_info.time_ms, transaction, platform_version, )?); diff --git a/packages/rs-drive/src/drive/document/delete/delete_document_for_contract/v0/mod.rs b/packages/rs-drive/src/drive/document/delete/delete_document_for_contract/v0/mod.rs index b82d4b25c51..18e27f9a5dd 100644 --- a/packages/rs-drive/src/drive/document/delete/delete_document_for_contract/v0/mod.rs +++ b/packages/rs-drive/src/drive/document/delete/delete_document_for_contract/v0/mod.rs @@ -37,8 +37,9 @@ impl Drive { document_id, contract, document_type_name, + &block_info, + None, estimated_costs_only_with_layer_info, - block_info.time_ms, transaction, &mut drive_operations, platform_version, diff --git a/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_apply_and_add_to_operations/mod.rs b/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_apply_and_add_to_operations/mod.rs index e1a451d52a1..22e37125306 100644 --- a/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_apply_and_add_to_operations/mod.rs +++ b/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_apply_and_add_to_operations/mod.rs @@ -4,6 +4,7 @@ use crate::drive::Drive; use crate::error::drive::DriveError; use crate::error::Error; use crate::fees::op::LowLevelDriveOperation; +use dpp::block::block_info::BlockInfo; use dpp::data_contract::DataContract; use dpp::identifier::Identifier; @@ -13,17 +14,19 @@ use grovedb::{EstimatedLayerInformation, TransactionArg}; use std::collections::HashMap; impl Drive { - /// Deletes a document. + /// Deletes a document and adds the operations to the given list. /// /// # Parameters /// * `document_id`: The ID of the document to delete. /// * `contract`: The contract that contains the document. /// * `document_type_name`: The name of the document type. - /// * `owner_id`: The owner ID of the document. + /// * `block_info`: The block this delete belongs to. + /// * `deleter_id`: The identity credited with the lifecycle record a + /// keep-history delete writes; `None` credits nobody. /// * `estimated_costs_only_with_layer_info`: An optional hashmap with layer information for estimated costs. /// * `transaction`: The transaction argument. /// * `drive_operations`: A mutable vector of low level drive operations. - /// * `drive_version`: The drive version to select the correct function version to run. + /// * `platform_version`: The platform version to select the correct function version to run. /// /// # Returns /// * `Ok(())` if the operation was successful. @@ -34,10 +37,11 @@ impl Drive { document_id: Identifier, contract: &DataContract, document_type_name: &str, + block_info: &BlockInfo, + deleter_id: Option, estimated_costs_only_with_layer_info: Option< HashMap, >, - block_time_ms: u64, transaction: TransactionArg, drive_operations: &mut Vec, platform_version: &PlatformVersion, @@ -53,8 +57,9 @@ impl Drive { document_id, contract, document_type_name, + block_info, + deleter_id, estimated_costs_only_with_layer_info, - block_time_ms, transaction, drive_operations, platform_version, diff --git a/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_apply_and_add_to_operations/v0/mod.rs b/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_apply_and_add_to_operations/v0/mod.rs index 2a65843e041..4d84f634ee9 100644 --- a/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_apply_and_add_to_operations/v0/mod.rs +++ b/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_apply_and_add_to_operations/v0/mod.rs @@ -1,6 +1,7 @@ use crate::drive::Drive; use crate::error::Error; use crate::fees::op::LowLevelDriveOperation; +use dpp::block::block_info::BlockInfo; use dpp::data_contract::DataContract; use dpp::identifier::Identifier; @@ -18,10 +19,11 @@ impl Drive { document_id: Identifier, contract: &DataContract, document_type_name: &str, + block_info: &BlockInfo, + deleter_id: Option, mut estimated_costs_only_with_layer_info: Option< HashMap, >, - block_time_ms: u64, transaction: TransactionArg, drive_operations: &mut Vec, platform_version: &PlatformVersion, @@ -37,9 +39,10 @@ impl Drive { document_id, contract, document_type_name, + block_info, + deleter_id, None, &mut estimated_costs_only_with_layer_info, - block_time_ms, transaction, platform_version, )?; diff --git a/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_id/v0/mod.rs b/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_id/v0/mod.rs index 50e415bbb06..934fa62531f 100644 --- a/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_id/v0/mod.rs +++ b/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_id/v0/mod.rs @@ -56,8 +56,9 @@ impl Drive { document_id, contract, document_type_name, + &block_info, + None, estimated_costs_only_with_layer_info, - block_info.time_ms, transaction, &mut drive_operations, platform_version, diff --git a/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_id_with_named_type_operations/mod.rs b/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_id_with_named_type_operations/mod.rs index 10d1297bdf0..6e54b7ff430 100644 --- a/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_id_with_named_type_operations/mod.rs +++ b/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_id_with_named_type_operations/mod.rs @@ -4,7 +4,7 @@ use crate::drive::Drive; use crate::error::drive::DriveError; use crate::error::Error; use crate::fees::op::LowLevelDriveOperation; -use dpp::block::epoch::Epoch; +use dpp::block::block_info::BlockInfo; use dpp::identifier::Identifier; use dpp::version::PlatformVersion; @@ -20,7 +20,9 @@ impl Drive { /// * `contract_id`: The ID of the contract that contains the document. /// * `document_type_name`: The name of the document type. /// * `owner_id`: The owner ID of the document. - /// * `epoch`: The epoch of the block. + /// * `block_info`: The block this delete belongs to. + /// * `deleter_id`: The identity credited with the lifecycle record a + /// keep-history delete writes; `None` credits nobody. /// * `previous_batch_operations`: Previous batch operations to include. /// * `estimated_costs_only_with_layer_info`: Estimated costs with layer info. /// * `transaction`: The transaction argument. @@ -35,12 +37,12 @@ impl Drive { document_id: Identifier, contract_id: Identifier, document_type_name: &str, - epoch: &Epoch, + block_info: &BlockInfo, + deleter_id: Option, previous_batch_operations: Option<&mut Vec>, estimated_costs_only_with_layer_info: &mut Option< HashMap, >, - block_time_ms: u64, transaction: TransactionArg, platform_version: &PlatformVersion, ) -> Result, Error> { @@ -55,10 +57,10 @@ impl Drive { document_id, contract_id, document_type_name, - epoch, + block_info, + deleter_id, previous_batch_operations, estimated_costs_only_with_layer_info, - block_time_ms, transaction, platform_version, ), diff --git a/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_id_with_named_type_operations/v0/mod.rs b/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_id_with_named_type_operations/v0/mod.rs index 1349e7e07a6..fbb84eaa6f1 100644 --- a/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_id_with_named_type_operations/v0/mod.rs +++ b/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_id_with_named_type_operations/v0/mod.rs @@ -10,7 +10,7 @@ use crate::error::document::DocumentError; use crate::error::Error; use crate::fees::op::LowLevelDriveOperation; -use dpp::block::epoch::Epoch; +use dpp::block::block_info::BlockInfo; use dpp::data_contract::accessors::v0::DataContractV0Getters; use dpp::identifier::Identifier; @@ -25,19 +25,19 @@ impl Drive { document_id: Identifier, contract_id: Identifier, document_type_name: &str, - epoch: &Epoch, + block_info: &BlockInfo, + deleter_id: Option, previous_batch_operations: Option<&mut Vec>, estimated_costs_only_with_layer_info: &mut Option< HashMap, >, - block_time_ms: u64, transaction: TransactionArg, platform_version: &PlatformVersion, ) -> Result, Error> { let mut operations = vec![]; let Some(contract_fetch_info) = self.get_contract_with_fetch_info_and_add_to_operations( contract_id.to_buffer(), - Some(epoch), + Some(&block_info.epoch), true, transaction, &mut operations, @@ -53,9 +53,10 @@ impl Drive { document_id, contract, document_type, + block_info, + deleter_id, previous_batch_operations, estimated_costs_only_with_layer_info, - block_time_ms, transaction, platform_version, ) diff --git a/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_operations/mod.rs b/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_operations/mod.rs index c4884500950..a4004b2e1bf 100644 --- a/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_operations/mod.rs +++ b/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_operations/mod.rs @@ -1,10 +1,12 @@ mod v0; +mod v1; use crate::drive::Drive; use crate::error::drive::DriveError; use crate::error::Error; use crate::fees::op::LowLevelDriveOperation; +use dpp::block::block_info::BlockInfo; use dpp::data_contract::document_type::DocumentTypeRef; use dpp::data_contract::DataContract; @@ -21,25 +23,29 @@ impl Drive { /// * `document_id`: The ID of the document to delete. /// * `contract`: The contract that contains the document. /// * `document_type`: The type of the document. + /// * `block_info`: The block this delete belongs to. + /// * `deleter_id`: The identity credited with the lifecycle record a + /// keep-history delete writes; `None` credits nobody. /// * `previous_batch_operations`: Previous batch operations to include. /// * `estimated_costs_only_with_layer_info`: Estimated costs with layer info. /// * `transaction`: The transaction argument. - /// * `drive_version`: The drive version to select the correct function version to run. + /// * `platform_version`: The platform version to select the correct function version to run. /// /// # Returns /// * `Ok(Vec)` if the operation was successful. /// * `Err(DriveError::UnknownVersionMismatch)` if the drive version does not match known versions. #[allow(clippy::too_many_arguments)] - pub(crate) fn delete_document_for_contract_operations( + pub fn delete_document_for_contract_operations( &self, document_id: Identifier, contract: &DataContract, document_type: DocumentTypeRef, + block_info: &BlockInfo, + deleter_id: Option, previous_batch_operations: Option<&mut Vec>, estimated_costs_only_with_layer_info: &mut Option< HashMap, >, - block_time_ms: u64, transaction: TransactionArg, platform_version: &PlatformVersion, ) -> Result, Error> { @@ -47,7 +53,7 @@ impl Drive { self.prepare_document_time_range_ttl( contract, document_type, - block_time_ms, + block_info.time_ms, transaction, platform_version, )?; @@ -56,9 +62,10 @@ impl Drive { document_id, contract, document_type, + block_info, + deleter_id, previous_batch_operations, estimated_costs_only_with_layer_info, - block_time_ms, transaction, platform_version, ) @@ -72,11 +79,12 @@ impl Drive { document_id: Identifier, contract: &DataContract, document_type: DocumentTypeRef, + block_info: &BlockInfo, + deleter_id: Option, previous_batch_operations: Option<&mut Vec>, estimated_costs_only_with_layer_info: &mut Option< HashMap, >, - block_time_ms: u64, transaction: TransactionArg, platform_version: &PlatformVersion, ) -> Result, Error> { @@ -93,13 +101,24 @@ impl Drive { document_type, previous_batch_operations, estimated_costs_only_with_layer_info, - block_time_ms, + block_info.time_ms, + transaction, + platform_version, + ), + 1 => self.delete_document_for_contract_operations_v1( + document_id, + contract, + document_type, + block_info, + deleter_id, + previous_batch_operations, + estimated_costs_only_with_layer_info, transaction, platform_version, ), version => Err(Error::Drive(DriveError::UnknownVersionMismatch { method: "delete_document_for_contract_operations".to_string(), - known_versions: vec![0], + known_versions: vec![0, 1], received: version, })), } @@ -112,10 +131,13 @@ impl Drive { /// * `document_id`: The ID of the document to delete. /// * `contract`: The contract that contains the document. /// * `document_type`: The type of the document. + /// * `block_info`: The block this delete belongs to. + /// * `deleter_id`: The identity credited with the lifecycle record a + /// keep-history delete writes; `None` credits nobody. /// * `previous_batch_operations`: Previous batch operations to include. /// * `estimated_costs_only_with_layer_info`: Estimated costs with layer info. /// * `transaction`: The transaction argument. - /// * `drive_version`: The drive version to select the correct function version to run. + /// * `platform_version`: The platform version to select the correct function version to run. /// /// # Returns /// * `Ok(Vec)` if the operation was successful. @@ -126,11 +148,12 @@ impl Drive { document_id: Identifier, contract: &DataContract, document_type: DocumentTypeRef, + block_info: &BlockInfo, + deleter_id: Option, previous_batch_operations: Option<&mut Vec>, estimated_costs_only_with_layer_info: &mut Option< HashMap, >, - block_time_ms: u64, transaction: TransactionArg, platform_version: &PlatformVersion, ) -> Result, Error> { @@ -147,13 +170,24 @@ impl Drive { document_type, previous_batch_operations, estimated_costs_only_with_layer_info, - block_time_ms, + block_info.time_ms, + transaction, + platform_version, + ), + 1 => self.force_delete_document_for_contract_operations_v1( + document_id, + contract, + document_type, + block_info, + deleter_id, + previous_batch_operations, + estimated_costs_only_with_layer_info, transaction, platform_version, ), version => Err(Error::Drive(DriveError::UnknownVersionMismatch { method: "force_delete_document_for_contract_operations".to_string(), - known_versions: vec![0], + known_versions: vec![0, 1], received: version, })), } diff --git a/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_operations/v1/mod.rs b/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_operations/v1/mod.rs new file mode 100644 index 00000000000..4a025fa4a83 --- /dev/null +++ b/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_operations/v1/mod.rs @@ -0,0 +1,365 @@ +use grovedb::batch::KeyInfoPath; +use grovedb::{Element, EstimatedLayerInformation, TransactionArg, TreeType}; + +use dpp::data_contract::document_type::DocumentTypeRef; + +use std::collections::HashMap; + +use crate::drive::constants::{DOCUMENT_HISTORY_CURRENT_REFERENCE_PATH_SIZE, STORAGE_FLAGS_SIZE}; +use crate::drive::document::paths::{ + contract_documents_primary_key_path, document_history_path, document_lifecycle_path, + DOCUMENT_LIFECYCLE_TREE_KEY, +}; +use crate::drive::document::primary_key_tree_type::DocumentTypePrimaryKeyTreeType; +use crate::util::object_size_info::DocumentInfo::{ + DocumentEstimatedAverageSize, DocumentOwnedInfo, +}; +use crate::util::storage_flags::StorageFlags; +use dpp::document::lifecycle::DocumentLifecycleRecord; +use dpp::serialization::PlatformSerializable; + +use dpp::block::block_info::BlockInfo; +use dpp::data_contract::DataContract; +use dpp::document::{Document, DocumentV0Getters}; + +use crate::drive::Drive; +use crate::util::grove_operations::QueryTarget::{QueryTargetTree, QueryTargetValue}; +use crate::util::grove_operations::{BatchInsertTreeApplyType, DirectQueryType, QueryType}; +use crate::util::object_size_info::PathKeyElementInfo::PathKeyElement; +use crate::util::object_size_info::PathKeyInfo::PathKey; +use crate::util::object_size_info::{DocumentAndContractInfo, OwnedDocumentInfo}; + +use crate::error::drive::DriveError; + +use crate::error::Error; +use crate::fees::op::LowLevelDriveOperation; + +use dpp::data_contract::accessors::v0::DataContractV0Getters; +use dpp::data_contract::document_type::accessors::DocumentTypeV0Getters; +use dpp::data_contract::document_type::methods::DocumentTypeV0Methods; +use dpp::document::serialization_traits::DocumentPlatformConversionMethodsV0; +use dpp::identifier::Identifier; + +use dpp::version::PlatformVersion; + +impl Drive { + /// Prepares the operations for deleting a document. + #[inline(always)] + #[allow(clippy::too_many_arguments)] + pub(super) fn delete_document_for_contract_operations_v1( + &self, + document_id: Identifier, + contract: &DataContract, + document_type: DocumentTypeRef, + block_info: &BlockInfo, + deleter_id: Option, + previous_batch_operations: Option<&mut Vec>, + estimated_costs_only_with_layer_info: &mut Option< + HashMap, + >, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result, Error> { + if !document_type.documents_can_be_deleted() { + return Err(Error::Drive(DriveError::UpdatingReadOnlyImmutableDocument( + "this document type is not mutable and can not be deleted", + ))); + } + + self.force_delete_document_for_contract_operations_v1( + document_id, + contract, + document_type, + block_info, + deleter_id, + previous_batch_operations, + estimated_costs_only_with_layer_info, + transaction, + platform_version, + ) + } + + /// Prepares the operations for deleting a document. + /// + /// Deleting a keep-history document removes it from every ordinary read + /// without touching a single retained revision: the current pointer and the + /// index references that lead to it go, and a lifecycle record recording the + /// deletion time takes their place. The revisions stay where they are, and + /// only an erase can remove them. + /// + /// Every other document type is deleted exactly as v0 deletes it. + #[inline(always)] + #[allow(clippy::too_many_arguments)] + pub(super) fn force_delete_document_for_contract_operations_v1( + &self, + document_id: Identifier, + contract: &DataContract, + document_type: DocumentTypeRef, + block_info: &BlockInfo, + deleter_id: Option, + previous_batch_operations: Option<&mut Vec>, + estimated_costs_only_with_layer_info: &mut Option< + HashMap, + >, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result, Error> { + if !document_type.documents_keep_history() { + return self.force_delete_document_for_contract_operations_v0( + document_id, + contract, + document_type, + previous_batch_operations, + estimated_costs_only_with_layer_info, + block_info.time_ms, + transaction, + platform_version, + ); + } + + let mut batch_operations: Vec = vec![]; + + let contract_documents_primary_key_path = contract_documents_primary_key_path( + contract.id_ref().as_bytes(), + document_type.name().as_str(), + ); + + let query_type = if let Some(estimated_costs_only_with_layer_info) = + estimated_costs_only_with_layer_info + { + Self::add_estimation_costs_for_levels_up_to_contract_document_type_excluded( + contract, + estimated_costs_only_with_layer_info, + &platform_version.drive, + )?; + // The read follows the current pointer into the type's history, so + // the dry run has to pay for the reference hop as well as the + // revision it lands on. `DirectQueryType` cannot express that: its + // conversion hardcodes an empty reference-size list. + QueryType::StatelessQuery { + in_tree_type: document_type.primary_key_tree_type(platform_version)?, + query_target: QueryTargetValue( + document_type.estimated_size(platform_version)? as u32 + ), + estimated_reference_sizes: vec![DOCUMENT_HISTORY_CURRENT_REFERENCE_PATH_SIZE], + } + } else { + QueryType::StatefulQuery + }; + + // Resolves the pointer, so this is the current revision's item, whose + // flags are the ones the index references were written with. + let document_element: Option = self.grove_get( + (&contract_documents_primary_key_path).into(), + document_id.as_slice(), + query_type.clone(), + transaction, + &mut batch_operations, + &platform_version.drive, + )?; + + let (document_info, latest_revision) = + if let QueryType::StatelessQuery { query_target, .. } = query_type { + (DocumentEstimatedAverageSize(query_target.len()), 0) + } else if let Some(document_element) = &document_element { + let Element::Item(data, element_flags) = document_element else { + return Err(Error::Drive(DriveError::CorruptedDocumentNotItem( + "the current pointer of a keep-history document did not resolve to an item", + ))); + }; + let document = + Document::from_bytes(data.as_slice(), document_type, platform_version)?; + let storage_flags = StorageFlags::map_cow_some_element_flags_ref(element_flags)?; + let latest_revision = document.revision().unwrap_or(1); + ( + DocumentOwnedInfo((document, storage_flags)), + latest_revision, + ) + } else { + return Err(Error::Drive(DriveError::DeletingDocumentThatDoesNotExist( + "document being deleted does not exist", + ))); + }; + + // The record remembers how many revisions the history retained next to + // the revision being deleted, so a later by-revision read can tell a + // contiguous history from one a pre-protocol-15 overwrite left gapped. + // That count is the history tree's own aggregate, read off its element + // in the type's history tree. + let mut history_type_path = document_history_path( + contract.id_ref().as_bytes(), + document_type.name().as_str(), + document_id.as_slice(), + ); + history_type_path.pop(); + let history_query_type = if estimated_costs_only_with_layer_info.is_some() { + DirectQueryType::StatelessDirectQuery { + in_tree_type: TreeType::NormalTree, + query_target: QueryTargetTree(STORAGE_FLAGS_SIZE, TreeType::ProvableCountTree), + } + } else { + DirectQueryType::StatefulDirectQuery + }; + let history_element = self.grove_get_raw( + history_type_path.as_slice().into(), + document_id.as_slice(), + history_query_type, + transaction, + &mut batch_operations, + &platform_version.drive, + )?; + let revision_count = match ( + estimated_costs_only_with_layer_info.is_some(), + history_element, + ) { + (true, _) => 0, + (false, Some(Element::ProvableCountTree(_, count, _))) => count, + (false, _) => { + return Err(Error::Drive(DriveError::CorruptedDriveState( + "a keep-history document being deleted has no history tree".to_string(), + ))) + } + }; + + // The pointer goes; the revisions it named stay in the history tree. + self.remove_document_from_primary_storage( + document_id, + document_type, + contract_documents_primary_key_path, + estimated_costs_only_with_layer_info, + transaction, + &mut batch_operations, + platform_version, + )?; + + let document_and_contract_info = DocumentAndContractInfo { + owned_document_info: OwnedDocumentInfo { + document_info, + owner_id: None, + }, + contract, + document_type, + }; + + self.remove_indices_for_top_index_level_for_contract_operations( + &document_and_contract_info, + &previous_batch_operations, + estimated_costs_only_with_layer_info, + block_info.time_ms, + transaction, + &mut batch_operations, + platform_version, + )?; + + // The record is the deleter's byte, refunded to them when the terminal + // erase chunk removes it. A caller with no signer writes it unflagged, + // and it then refunds nobody, exactly as the unflagged structural bytes + // of a non-deletable contract do. + let record_flags = deleter_id + .map(|id| StorageFlags::new_single_epoch(block_info.epoch.index, Some(id.to_buffer()))); + self.add_lifecycle_record_operations( + document_id, + contract, + document_type, + DocumentLifecycleRecord::deleted_at( + block_info.time_ms, + latest_revision, + revision_count, + ), + record_flags.as_ref(), + estimated_costs_only_with_layer_info, + transaction, + &mut batch_operations, + platform_version, + )?; + + Ok(batch_operations) + } + + /// Creates the document type's lifecycle tree if it does not exist yet and + /// writes one record into it. + /// + /// The tree is created on demand so that a document type whose documents are + /// never deleted never pays for one, and the record is flagged to the + /// deleter, who is refunded when the terminal erase chunk removes it. + #[allow(clippy::too_many_arguments)] + pub(crate) fn add_lifecycle_record_operations( + &self, + document_id: Identifier, + contract: &DataContract, + document_type: DocumentTypeRef, + record: DocumentLifecycleRecord, + storage_flags: Option<&StorageFlags>, + estimated_costs_only_with_layer_info: &mut Option< + HashMap, + >, + transaction: TransactionArg, + batch_operations: &mut Vec, + platform_version: &PlatformVersion, + ) -> Result<(), Error> { + let lifecycle_path = + document_lifecycle_path(contract.id_ref().as_bytes(), document_type.name().as_str()); + let element_flags = StorageFlags::map_to_some_element_flags(storage_flags); + if let Some(layers) = estimated_costs_only_with_layer_info { + Self::add_estimation_costs_for_lifecycle_record( + contract, + document_type, + layers, + platform_version, + )?; + } + + let tree_apply_type = if estimated_costs_only_with_layer_info.is_some() { + BatchInsertTreeApplyType::StatelessBatchInsertTree { + in_tree_type: TreeType::NormalTree, + tree_type: TreeType::NormalTree, + // The container is unflagged, so its element carries no + // beneficiary payload to size. + flags_len: 0, + } + } else { + BatchInsertTreeApplyType::StatefulBatchInsertTree + }; + + let mut document_type_path = crate::drive::document::paths::contract_document_type_path_vec( + contract.id_ref().as_bytes(), + document_type.name().as_str(), + ); + let lifecycle_tree_key = vec![DOCUMENT_LIFECYCLE_TREE_KEY]; + // The container is shared by every document of the type and outlives + // any one of them: an erase removes records and per-document history + // trees, never this. Flagging it to whoever happened to delete first + // would charge them for structure nobody refunds, and would make the + // committed encoding depend on which document went first. The record + // inside it keeps the deleter's flags, which is what an erase refunds. + // + // The running batch is consulted so that two deletes of different + // documents of the same type, prepared before either is applied, insert + // this one key once rather than twice. + let mut container_operations = Vec::new(); + self.batch_insert_empty_tree_if_not_exists::<0>( + PathKey((std::mem::take(&mut document_type_path), lifecycle_tree_key)), + TreeType::NormalTree, + None, + tree_apply_type, + transaction, + &mut Some(batch_operations), + &mut container_operations, + &platform_version.drive, + )?; + batch_operations.append(&mut container_operations); + + // The record's content is fully known whether or not this is a dry + // run: the deletion time and the revision counts come from the delete + // itself, and the deleter it is flagged to is an input to the delete. + self.batch_insert::<0>( + PathKeyElement(( + lifecycle_path, + document_id.to_vec(), + Element::Item(record.serialize_to_bytes()?, element_flags), + )), + batch_operations, + &platform_version.drive, + ) + } +} diff --git a/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_with_named_type_operations/mod.rs b/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_with_named_type_operations/mod.rs index 264aaaf7a97..b50e9660115 100644 --- a/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_with_named_type_operations/mod.rs +++ b/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_with_named_type_operations/mod.rs @@ -4,7 +4,7 @@ use crate::drive::Drive; use crate::error::drive::DriveError; use crate::error::Error; use crate::fees::op::LowLevelDriveOperation; - +use dpp::block::block_info::BlockInfo; use dpp::data_contract::DataContract; use dpp::identifier::Identifier; @@ -20,10 +20,13 @@ impl Drive { /// * `document_id`: The ID of the document to delete. /// * `contract`: The contract that contains the document. /// * `document_type_name`: The name of the document type. + /// * `block_info`: The block this delete belongs to. + /// * `deleter_id`: The identity credited with the lifecycle record a + /// keep-history delete writes; `None` credits nobody. /// * `previous_batch_operations`: Previous batch operations to include. /// * `estimated_costs_only_with_layer_info`: Estimated costs with layer info. /// * `transaction`: The transaction argument. - /// * `drive_version`: The drive version to select the correct function version to run. + /// * `platform_version`: The platform version to select the correct function version to run. /// /// # Returns /// * `Ok(Vec)` if the operation was successful. @@ -34,11 +37,12 @@ impl Drive { document_id: Identifier, contract: &DataContract, document_type_name: &str, + block_info: &BlockInfo, + deleter_id: Option, previous_batch_operations: Option<&mut Vec>, estimated_costs_only_with_layer_info: &mut Option< HashMap, >, - block_time_ms: u64, transaction: TransactionArg, platform_version: &PlatformVersion, ) -> Result, Error> { @@ -53,9 +57,10 @@ impl Drive { document_id, contract, document_type_name, + block_info, + deleter_id, previous_batch_operations, estimated_costs_only_with_layer_info, - block_time_ms, transaction, platform_version, ), diff --git a/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_with_named_type_operations/v0/mod.rs b/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_with_named_type_operations/v0/mod.rs index cddde01f1b0..47e05699e6b 100644 --- a/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_with_named_type_operations/v0/mod.rs +++ b/packages/rs-drive/src/drive/document/delete/delete_document_for_contract_with_named_type_operations/v0/mod.rs @@ -4,6 +4,7 @@ use grovedb::{EstimatedLayerInformation, TransactionArg}; use std::collections::HashMap; +use dpp::block::block_info::BlockInfo; use dpp::data_contract::DataContract; use crate::drive::Drive; @@ -25,11 +26,12 @@ impl Drive { document_id: Identifier, contract: &DataContract, document_type_name: &str, + block_info: &BlockInfo, + deleter_id: Option, previous_batch_operations: Option<&mut Vec>, estimated_costs_only_with_layer_info: &mut Option< HashMap, >, - block_time_ms: u64, transaction: TransactionArg, platform_version: &PlatformVersion, ) -> Result, Error> { @@ -38,9 +40,10 @@ impl Drive { document_id, contract, document_type, + block_info, + deleter_id, previous_batch_operations, estimated_costs_only_with_layer_info, - block_time_ms, transaction, platform_version, ) diff --git a/packages/rs-drive/src/drive/document/delete/erase_document_for_contract_operations/mod.rs b/packages/rs-drive/src/drive/document/delete/erase_document_for_contract_operations/mod.rs new file mode 100644 index 00000000000..b9891903c4f --- /dev/null +++ b/packages/rs-drive/src/drive/document/delete/erase_document_for_contract_operations/mod.rs @@ -0,0 +1,74 @@ +mod v0; + +use crate::drive::Drive; +use crate::error::drive::DriveError; +use crate::error::Error; +use crate::fees::op::LowLevelDriveOperation; + +use dpp::block::block_info::BlockInfo; +use dpp::data_contract::document_type::DocumentTypeRef; +use dpp::data_contract::DataContract; + +use dpp::identifier::Identifier; +use dpp::version::PlatformVersion; +use grovedb::batch::KeyInfoPath; +use grovedb::{EstimatedLayerInformation, TransactionArg}; +use std::collections::HashMap; + +impl Drive { + /// Prepares the operations for removing a bounded chunk of the retained + /// revisions of a document that has already been deleted. + /// + /// # Parameters + /// * `document_id`: The document whose revisions are being removed. + /// * `contract`: The contract that contains the document. + /// * `document_type`: The type of the document, which must keep history. + /// * `block_info`: The block this erase belongs to. + /// * `estimated_costs_only_with_layer_info`: Estimated costs with layer info. + /// * `transaction`: The transaction argument. + /// * `platform_version`: The platform version to select the correct function version to run. + /// + /// # Returns + /// * `Ok(Vec)` if the operation was successful. + /// * `Err(DriveError::UnknownVersionMismatch)` if the drive version does not match known versions. + #[allow(clippy::too_many_arguments)] + pub fn erase_document_for_contract_operations( + &self, + document_id: Identifier, + contract: &DataContract, + document_type: DocumentTypeRef, + block_info: &BlockInfo, + estimated_costs_only_with_layer_info: &mut Option< + HashMap, + >, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result, Error> { + match platform_version + .drive + .methods + .document + .delete + .erase_document_for_contract_operations + { + Some(0) => self.erase_document_for_contract_operations_v0( + document_id, + contract, + document_type, + block_info, + estimated_costs_only_with_layer_info, + transaction, + platform_version, + ), + Some(version) => Err(Error::Drive(DriveError::UnknownVersionMismatch { + method: "erase_document_for_contract_operations".to_string(), + known_versions: vec![0], + received: version, + })), + None => Err(Error::Drive(DriveError::VersionNotActive { + method: "erase_document_for_contract_operations".to_string(), + known_versions: vec![0], + })), + } + } +} diff --git a/packages/rs-drive/src/drive/document/delete/erase_document_for_contract_operations/v0/mod.rs b/packages/rs-drive/src/drive/document/delete/erase_document_for_contract_operations/v0/mod.rs new file mode 100644 index 00000000000..db7a4d0fe21 --- /dev/null +++ b/packages/rs-drive/src/drive/document/delete/erase_document_for_contract_operations/v0/mod.rs @@ -0,0 +1,432 @@ +use std::collections::HashMap; + +use dpp::block::block_info::BlockInfo; +use dpp::data_contract::accessors::v0::DataContractV0Getters; +use dpp::data_contract::document_type::accessors::DocumentTypeV0Getters; +use dpp::data_contract::document_type::methods::DocumentTypeV0Methods; +use dpp::data_contract::document_type::DocumentTypeRef; +use dpp::data_contract::DataContract; +use dpp::identifier::Identifier; +use dpp::version::PlatformVersion; +use grovedb::batch::key_info::KeyInfo; +use grovedb::batch::KeyInfoPath; +use grovedb::query_result_type::QueryResultType; +use grovedb::{ + Element, EstimatedLayerInformation, MaybeTree, PathQuery, Query, SizedQuery, TransactionArg, + TreeType, +}; + +use crate::drive::document::paths::{document_history_path, document_lifecycle_path}; +use crate::drive::Drive; +use crate::error::drive::DriveError; +use crate::error::Error; +use crate::fees::op::LowLevelDriveOperation; +use crate::util::common::encode::encode_u64; +use crate::util::grove_operations::BatchDeleteApplyType::{ + StatefulBatchDelete, StatelessBatchDelete, +}; +use crate::util::grove_operations::{DirectQueryType, QueryTarget}; +use crate::util::object_size_info::PathKeyElementInfo::{PathKeyElement, PathKeyElementSize}; +use crate::util::storage_flags::StorageFlags; +use dpp::document::lifecycle::{DocumentLifecycleRecord, DOCUMENT_LIFECYCLE_RECORD_MAX_SIZE}; +use dpp::serialization::{PlatformDeserializableTrusted, PlatformSerializable}; + +/// Length of a revision key: a block timestamp followed by a history sequence. +const REVISION_KEY_LENGTH: usize = 16; + +impl Drive { + /// Prepares the operations for removing a bounded chunk of the retained + /// revisions of an already deleted document. + /// + /// Revisions go newest first, so the remnant a partial erasure leaves behind + /// is the document's oldest content and the retained sequence stays + /// contiguous from one. The chunk is bounded by + /// `max_document_revisions_erased_per_transition`, and the enumeration asks + /// for one revision more than it may remove so that it knows, before + /// emitting anything, whether this chunk is the last one. + /// + /// A terminal chunk also removes the lifecycle record and the now empty + /// history subtree; GroveDB establishes that the subtree is empty from the + /// deletes already in this batch and refuses the removal otherwise. A + /// non-terminal first chunk instead overwrites the record with the erasure + /// it authorizes, which is what lets any identity finish the work later. + /// The two never happen together, so one batch never carries two operations + /// on the record's key. + #[inline(always)] + #[allow(clippy::too_many_arguments)] + pub(super) fn erase_document_for_contract_operations_v0( + &self, + document_id: Identifier, + contract: &DataContract, + document_type: DocumentTypeRef, + block_info: &BlockInfo, + estimated_costs_only_with_layer_info: &mut Option< + HashMap, + >, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result, Error> { + if !document_type.documents_keep_history() { + return Err(Error::Drive(DriveError::CorruptedCodeExecution( + "only a document type that keeps history has revisions to erase", + ))); + } + let chunk = platform_version + .system_limits + .max_document_revisions_erased_per_transition + .ok_or(Error::Drive(DriveError::NotSupported( + "erasing retained revisions is not available at this protocol version", + )))?; + + let history_path = document_history_path( + contract.id_ref().as_bytes(), + document_type.name().as_str(), + document_id.as_slice(), + ); + let lifecycle_path = + document_lifecycle_path(contract.id_ref().as_bytes(), document_type.name().as_str()); + // Worst-case sizing: a chunk may hold the type's largest documents, and + // the admission estimate has to cover that whatever this document + // actually stores. + let revision_size = document_type.max_size(platform_version)? as u32; + + let mut batch_operations: Vec = vec![]; + + if let Some(layers) = estimated_costs_only_with_layer_info { + Self::add_estimation_costs_for_erase_document( + document_id, + contract, + document_type, + layers, + platform_version, + )?; + return self.estimated_erase_document_operations( + document_id, + &history_path, + &lifecycle_path, + chunk, + revision_size, + block_info, + batch_operations, + transaction, + platform_version, + ); + } + + // The record exists exactly while a document is deleted or erasing, so + // its presence is what proves the document may lose revisions at all. + // It is read before anything is emitted: a caller that reaches this + // without a delete, or after an erasure has already finished and the + // id was reused, would otherwise strip an active document's history. + let (record, flags) = self.fetch_lifecycle_record_with_flags( + document_id, + &lifecycle_path, + &mut batch_operations, + transaction, + platform_version, + )?; + + let removable = self.enumerate_newest_revision_keys( + &history_path, + chunk, + &mut batch_operations, + transaction, + platform_version, + )?; + if removable.is_empty() { + return Err(Error::Drive(DriveError::CorruptedDriveState( + "a document with a lifecycle record retains no revisions".to_string(), + ))); + } + let terminal = removable.len() <= chunk as usize; + let to_remove = if terminal { + removable.as_slice() + } else { + &removable[..chunk as usize] + }; + + // A leaf delete does not consult the batch it joins, so each one is + // generated against an empty view and appended: building the whole + // running batch once per revision would be quadratic in the chunk size. + let mut single_operation = Vec::with_capacity(2); + for key in to_remove { + single_operation.clear(); + self.batch_delete( + history_path.as_slice().into(), + key, + StatefulBatchDelete { + is_known_to_be_subtree_with_sum: Some(MaybeTree::NotTree), + }, + transaction, + &mut single_operation, + &platform_version.drive, + )?; + batch_operations.append(&mut single_operation); + } + + if terminal { + self.batch_delete( + lifecycle_path.as_slice().into(), + document_id.as_slice(), + StatefulBatchDelete { + is_known_to_be_subtree_with_sum: Some(MaybeTree::NotTree), + }, + transaction, + &mut batch_operations, + &platform_version.drive, + )?; + let mut history_root = history_path; + history_root.pop(); + // Every revision has already been deleted into this batch, which is + // how GroveDB establishes that the subtree is empty. If any survived + // it refuses the removal rather than orphaning their storage. + self.batch_delete( + history_root.as_slice().into(), + document_id.as_slice(), + StatefulBatchDelete { + is_known_to_be_subtree_with_sum: Some(MaybeTree::Tree( + TreeType::ProvableCountTree, + )), + }, + transaction, + &mut batch_operations, + &platform_version.drive, + )?; + } else { + if record.is_erasing() { + // A continuation: the record already carries the erasure this + // chunk is finishing, and nothing but an authorized first chunk + // may write those fields. + return Ok(batch_operations); + } + let (time_ms, revision) = Self::decode_revision_key(&removable[0])?; + // Written with the flags of the record it replaces, so the deleter + // stays the beneficiary of the bytes it paid for; any bytes the new + // fields add are charged to this erase and flagged as such by the + // batch apply. + self.batch_insert::<0>( + PathKeyElement(( + lifecycle_path, + document_id.to_vec(), + Element::Item( + record + .starting_erase_at(block_info.time_ms, time_ms, revision) + .serialize_to_bytes()?, + flags, + ), + )), + &mut batch_operations, + &platform_version.drive, + )?; + } + + Ok(batch_operations) + } + + /// Reads the revision keys of one document newest first, one more than a + /// chunk may remove so the caller can tell a terminal chunk from a partial + /// one before it emits anything. + /// + /// The keys are what this needs, but GroveDB's query surface has no + /// key-only result shape over a range: every result type it exposes carries + /// elements. The revision bodies therefore come back and are dropped, and + /// the estimate below prices that read for what it is. What this does avoid + /// is the running-batch snapshot the general delete-by-query helper rebuilds + /// for every element, which is quadratic in the chunk size. + fn enumerate_newest_revision_keys( + &self, + history_path: &[Vec], + chunk: u16, + batch_operations: &mut Vec, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result>, Error> { + let mut query = Query::new_with_direction(false); + // Exactly the revision key space: the current pointer lives in the + // primary-key tree and is not in this subtree at all. + query.insert_range_from(encode_u64(0)..); + let path_query = PathQuery::new( + history_path.to_vec(), + SizedQuery::new(query, Some(chunk.saturating_add(1)), None), + ); + let (results, _) = self.grove_get_raw_path_query( + &path_query, + transaction, + QueryResultType::QueryKeyElementPairResultType, + batch_operations, + &platform_version.drive, + )?; + Ok(results + .to_key_elements() + .into_iter() + .map(|(key, _)| key) + .collect()) + } + + /// Reads one lifecycle record together with the flags naming the identity + /// its bytes were charged to. + fn fetch_lifecycle_record_with_flags( + &self, + document_id: Identifier, + lifecycle_path: &[Vec], + batch_operations: &mut Vec, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result<(DocumentLifecycleRecord, Option>), Error> { + // A type nothing has been deleted under has no lifecycle tree at all; + // that reads as no record, the same as an id without one. + let element = self + .grove_get_raw_optional( + lifecycle_path.into(), + document_id.as_slice(), + DirectQueryType::StatefulDirectQuery, + transaction, + batch_operations, + &platform_version.drive, + )? + .ok_or(Error::Drive(DriveError::InvalidInput( + "only a deleted document can be erased: it has no lifecycle record".to_string(), + )))?; + let Element::Item(bytes, flags) = element else { + return Err(Error::Drive(DriveError::CorruptedElementType( + "a lifecycle record is not an item", + ))); + }; + Ok(( + DocumentLifecycleRecord::deserialize_from_bytes_trusted(&bytes)?, + flags, + )) + } + + /// Splits a revision key into the block time and history sequence it + /// carries. + fn decode_revision_key(key: &[u8]) -> Result<(u64, u64), Error> { + if key.len() != REVISION_KEY_LENGTH { + return Err(Error::Drive(DriveError::CorruptedDriveState( + "a retained revision key does not carry a time and a sequence".to_string(), + ))); + } + let decode = |bytes: &[u8]| -> u64 { + let mut buffer = [0u8; 8]; + buffer.copy_from_slice(bytes); + // The encoder flips the sign bit so that the ordering of the keys + // matches the ordering of the values. + u64::from_be_bytes(buffer) ^ (1 << 63) + }; + Ok((decode(&key[..8]), decode(&key[8..]))) + } + + /// Prices a full chunk without reading any state. + /// + /// The dry run cannot know how many revisions a document retains, so it + /// charges for a whole chunk of the type's largest documents plus both + /// endings: the record write of a first chunk and the subtree removal of a + /// terminal one. Every erase is therefore admitted against the same + /// worst-case estimate whatever the document's actual history length. + /// + /// The enumeration is priced at one revision more than a chunk removes, + /// because that is what it reads, and at the full body size, because the + /// query surface returns bodies whether or not the caller wants them. The + /// record read every chunk performs first is priced too. + #[allow(clippy::too_many_arguments)] + fn estimated_erase_document_operations( + &self, + document_id: Identifier, + history_path: &[Vec], + lifecycle_path: &[Vec], + chunk: u16, + revision_size: u32, + block_info: &BlockInfo, + mut batch_operations: Vec, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result, Error> { + // The enumeration reads one revision more than the chunk removes, and + // reads each of them whole. + for sequence in 1..=chunk as u64 + 1 { + let mut key = encode_u64(block_info.time_ms); + key.extend(encode_u64(sequence)); + self.grove_get_raw( + history_path.into(), + &key, + DirectQueryType::StatelessDirectQuery { + in_tree_type: TreeType::ProvableCountTree, + query_target: QueryTarget::QueryTargetValue(revision_size), + }, + transaction, + &mut batch_operations, + &platform_version.drive, + )?; + } + + // Every chunk reads the record first; a non-terminal one overwrites it. + self.grove_get_raw( + lifecycle_path.into(), + document_id.as_slice(), + DirectQueryType::StatelessDirectQuery { + in_tree_type: TreeType::NormalTree, + query_target: QueryTarget::QueryTargetValue(DOCUMENT_LIFECYCLE_RECORD_MAX_SIZE), + }, + transaction, + &mut batch_operations, + &platform_version.drive, + )?; + + let mut single_operation = Vec::with_capacity(2); + for sequence in 1..=chunk as u64 { + // Distinct synthetic keys of the real shape: the dry run's batch is + // checked for consistency like any other, so a repeated key would be + // rejected rather than priced. + let mut key = encode_u64(block_info.time_ms); + key.extend(encode_u64(sequence)); + single_operation.clear(); + self.batch_delete( + history_path.into(), + &key, + StatelessBatchDelete { + in_tree_type: TreeType::ProvableCountTree, + estimated_key_size: REVISION_KEY_LENGTH as u32, + estimated_value_size: revision_size, + }, + transaction, + &mut single_operation, + &platform_version.drive, + )?; + batch_operations.append(&mut single_operation); + } + + // A record of the right shape rather than the real one: the dry run + // cannot know the deletion time it carries or the identity it is + // flagged to, only that both are there and how many bytes they take. + let flags_len = StorageFlags::approximate_size(true, None) as usize; + self.batch_insert::<0>( + PathKeyElementSize(( + KeyInfoPath::from_known_owned_path(lifecycle_path.to_vec()), + KeyInfo::KnownKey(document_id.to_vec()), + Element::Item( + vec![0u8; DOCUMENT_LIFECYCLE_RECORD_MAX_SIZE as usize], + Some(vec![0u8; flags_len]), + ), + )), + &mut batch_operations, + &platform_version.drive, + )?; + + let mut history_root = history_path.to_vec(); + history_root.pop(); + self.batch_delete( + history_root.as_slice().into(), + document_id.as_slice(), + StatelessBatchDelete { + in_tree_type: TreeType::NormalTree, + estimated_key_size: 32, + estimated_value_size: TreeType::ProvableCountTree.inner_node_type().cost() + 3, + }, + transaction, + &mut batch_operations, + &platform_version.drive, + )?; + + Ok(batch_operations) + } +} diff --git a/packages/rs-drive/src/drive/document/delete/internal/add_estimation_costs_for_remove_document_to_primary_storage/mod.rs b/packages/rs-drive/src/drive/document/delete/internal/add_estimation_costs_for_remove_document_to_primary_storage/mod.rs index 7fe17619b5d..7bfac71dd9b 100644 --- a/packages/rs-drive/src/drive/document/delete/internal/add_estimation_costs_for_remove_document_to_primary_storage/mod.rs +++ b/packages/rs-drive/src/drive/document/delete/internal/add_estimation_costs_for_remove_document_to_primary_storage/mod.rs @@ -1,4 +1,5 @@ mod v0; +mod v1; use crate::drive::Drive; use crate::error::drive::DriveError; @@ -41,9 +42,15 @@ impl Drive { estimated_costs_only_with_layer_info, platform_version, ), + 1 => Self::add_estimation_costs_for_remove_document_to_primary_storage_v1( + primary_key_path, + document_type, + estimated_costs_only_with_layer_info, + platform_version, + ), version => Err(Error::Drive(DriveError::UnknownVersionMismatch { method: "add_estimation_costs_for_remove_document_to_primary_storage".to_string(), - known_versions: vec![0], + known_versions: vec![0, 1], received: version, })), } diff --git a/packages/rs-drive/src/drive/document/delete/internal/add_estimation_costs_for_remove_document_to_primary_storage/v1/mod.rs b/packages/rs-drive/src/drive/document/delete/internal/add_estimation_costs_for_remove_document_to_primary_storage/v1/mod.rs new file mode 100644 index 00000000000..572f0421384 --- /dev/null +++ b/packages/rs-drive/src/drive/document/delete/internal/add_estimation_costs_for_remove_document_to_primary_storage/v1/mod.rs @@ -0,0 +1,83 @@ +use grovedb::batch::KeyInfoPath; + +use grovedb::EstimatedLayerCount::PotentiallyAtMaxElements; +use grovedb::EstimatedLayerInformation; +use grovedb::EstimatedLayerSizes::{AllReference, Mix}; + +use dpp::data_contract::document_type::DocumentTypeRef; + +use std::collections::HashMap; + +use crate::drive::constants::{ + AVERAGE_NUMBER_OF_UPDATES, AVERAGE_UPDATE_BYTE_COUNT_REQUIRED_SIZE, + DOCUMENT_HISTORY_CURRENT_REFERENCE_PATH_SIZE, +}; +use crate::drive::document::primary_key_tree_type::DocumentTypePrimaryKeyTreeType; +use crate::drive::Drive; +use crate::error::Error; +use crate::util::storage_flags::StorageFlags; + +use dpp::data_contract::document_type::accessors::{DocumentTypeV0Getters, DocumentTypeV2Getters}; +use dpp::version::PlatformVersion; + +impl Drive { + /// Adds the estimation layer for removing one entry from a document type's + /// primary-key tree. + /// + /// A keep-history type stores current pointers there rather than document + /// items, so its layer is sized from the reference shape the writer + /// installs; a summable type's pointers additionally carry the document's + /// contribution. Every other document type is estimated exactly as v0 does. + #[inline(always)] + pub(super) fn add_estimation_costs_for_remove_document_to_primary_storage_v1( + primary_key_path: [&[u8]; 5], + document_type: DocumentTypeRef, + estimated_costs_only_with_layer_info: &mut HashMap, + platform_version: &PlatformVersion, + ) -> Result<(), Error> { + if !document_type.documents_keep_history() { + return Self::add_estimation_costs_for_remove_document_to_primary_storage_v0( + primary_key_path, + document_type, + estimated_costs_only_with_layer_info, + platform_version, + ); + } + + let approximate_size = if document_type.documents_mutable() { + Some(( + AVERAGE_NUMBER_OF_UPDATES as u16, + AVERAGE_UPDATE_BYTE_COUNT_REQUIRED_SIZE, + )) + } else { + None + }; + let flags_size = Some(StorageFlags::approximate_size(true, approximate_size)); + let references = if document_type.documents_summable().is_some() { + Mix { + subtrees_size: None, + items_size: None, + references_size: None, + items_with_sum_item_size: None, + references_with_sum_item_size: Some(( + 32, + DOCUMENT_HISTORY_CURRENT_REFERENCE_PATH_SIZE, + flags_size, + 1, + )), + } + } else { + AllReference(32, DOCUMENT_HISTORY_CURRENT_REFERENCE_PATH_SIZE, flags_size) + }; + estimated_costs_only_with_layer_info.insert( + KeyInfoPath::from_known_path(primary_key_path), + EstimatedLayerInformation { + tree_type: document_type.primary_key_tree_type(platform_version)?, + estimated_layer_count: PotentiallyAtMaxElements, + estimated_layer_sizes: references, + }, + ); + + Ok(()) + } +} diff --git a/packages/rs-drive/src/drive/document/delete/mod.rs b/packages/rs-drive/src/drive/document/delete/mod.rs index e04dc6eb7f7..1e8d2ba6b55 100644 --- a/packages/rs-drive/src/drive/document/delete/mod.rs +++ b/packages/rs-drive/src/drive/document/delete/mod.rs @@ -43,6 +43,7 @@ mod delete_document_for_contract_with_named_type_operations; // This module contains functionality to delete a document for contract operations mod delete_document_for_contract_operations; mod delete_index_only_document_for_contract_operations; +mod erase_document_for_contract_operations; mod internal; @@ -1262,13 +1263,29 @@ mod tests { ); } + /// Protocol 14 refuses the delete outright, and protocol 15 carries it out: + /// the document leaves every ordinary read while its revisions stay where + /// they are. #[test] - fn test_delete_document_keeps_history_returns_error() { + fn test_delete_document_keeps_history_returns_error_at_protocol_14() { + run_keep_history_delete_at_protocol_version(14); + } + + #[test] + fn should_delete_a_keep_history_document_without_touching_its_revisions_at_protocol_15() { + run_keep_history_delete_at_protocol_version(15); + } + + fn run_keep_history_delete_at_protocol_version(protocol_version: u32) { + use crate::drive::document::lifecycle::DocumentLifecycleState; + use dpp::document::DocumentV0Getters; + let drive = setup_drive_with_initial_state_structure(None); let db_transaction = drive.grove.start_transaction(); - let platform_version = PlatformVersion::latest(); + let platform_version = + PlatformVersion::get(protocol_version).expect("expected a known protocol version"); let contract = setup_contract( &drive, @@ -1328,24 +1345,226 @@ mod tests { .try_into() .expect("this be 32 bytes"); - // Attempting to delete a document that keeps history should return an error - let err = drive - .delete_document_for_contract( + let outcome = drive.delete_document_for_contract( + document_id, + &contract, + "person", + BlockInfo::default(), + true, + None, + platform_version, + Some(&EPOCH_CHANGE_FEE_VERSION_TEST), + ); + + if protocol_version < 15 { + assert!(matches!( + outcome.expect_err("expected deleting a history-keeping document to fail"), + Error::Drive(DriveError::InvalidDeletionOfDocumentThatKeepsHistory(_)) + )); + return; + } + + outcome.expect("expected the delete to succeed"); + + let (state, _) = drive + .fetch_document_lifecycle( + &contract, + document_type, + document_id, + None, + None, + platform_version, + ) + .expect("expected to read the lifecycle"); + let DocumentLifecycleState::Deleted(retained) = state else { + panic!("expected the document to be deleted, got {state:?}"); + }; + assert_eq!( + retained.id(), + document_id, + "the retained revision must still be readable" + ); + + let query = DriveDocumentQuery::all_items_query(&contract, document_type, None); + let (documents, _, _) = query + .execute_raw_results_no_proof(&drive, None, None, platform_version) + .expect("expected to query documents"); + assert!( + documents.is_empty(), + "a deleted document must be absent from every ordinary read" + ); + } + + /// A delete entry point that carries only the block time still records the + /// lifecycle entry of a keep-history document at the lifecycle generation: + /// the entry needs the deletion time, which every entry point carries, and + /// a deleter only to credit the record's bytes, which these carry no more + /// than the fee-applying wrappers do. + #[test] + fn should_record_a_keep_history_delete_through_entry_points_without_a_deleter() { + use crate::drive::document::lifecycle::DocumentLifecycleState; + use dpp::document::DocumentV0Getters; + + let platform_version = PlatformVersion::latest(); + let setup = || { + let drive = setup_drive_with_initial_state_structure(None); + let contract = setup_contract( + &drive, + "tests/supporting_files/contract/family/family-contract-with-history.json", + None, + None, + None::, + None, + None, + ); + let document_type = contract + .document_type_for_name("person") + .expect("expected to get document type"); + let owner_id = rand::thread_rng().gen::<[u8; 32]>(); + let person_document = json_document_to_document( + "tests/supporting_files/contract/family/person0.json", + Some(owner_id.into()), + document_type, + platform_version, + ) + .expect("expected to get document"); + drive + .add_document_for_contract( + DocumentAndContractInfo { + owned_document_info: OwnedDocumentInfo { + document_info: DocumentRefInfo(( + &person_document, + Some(Cow::Owned(StorageFlags::SingleEpoch(0))), + )), + owner_id: None, + }, + contract: &contract, + document_type, + }, + false, + BlockInfo::default(), + true, + None, + platform_version, + None, + ) + .expect("expected to insert a document successfully"); + (drive, contract, person_document.id()) + }; + let assert_deleted_at = |drive: &Drive, contract: &DataContract, document_id| { + let document_type = contract + .document_type_for_name("person") + .expect("expected to get document type"); + assert!( + matches!( + drive + .fetch_document_lifecycle( + contract, + document_type, + document_id, + None, + None, + platform_version, + ) + .expect("expected to read the lifecycle") + .0, + DocumentLifecycleState::Deleted(_) + ), + "the delete must record the lifecycle entry" + ); + let (_, record) = drive + .fetch_document_history( + &crate::query::document_history_drive_query::DocumentHistoryDriveQuery { + contract_id: contract.id().to_buffer(), + document_type_name: "person".to_string(), + document_id: document_id.to_buffer(), + filter: crate::query::document_history_drive_query::DocumentHistoryFilter::StartAtTime(0), + limit: Some(1), + }, + document_type, + None, + platform_version, + ) + .map(|history| (history.entries.len(), history.lifecycle)) + .expect("expected to read the history"); + assert_eq!( + record + .expect("the lifecycle is authenticated") + .times + .deleted_at_ms, + 1_000, + "the entry carries the deletion time the entry point was given" + ); + }; + + let (drive, contract, document_id) = setup(); + let document_type = contract + .document_type_for_name("person") + .expect("expected to get document type"); + let operations = drive + .delete_document_for_contract_operations( + document_id, + &contract, + document_type, + &BlockInfo::default_with_time(1_000), + None, + None, + &mut None, + None, + platform_version, + ) + .expect("expected the delete to build"); + drive + .apply_batch_low_level_drive_operations( + None, + None, + operations, + &mut vec![], + &platform_version.drive, + ) + .expect("expected the delete to apply"); + assert_deleted_at(&drive, &contract, document_id); + + let (drive, contract, document_id) = setup(); + let operations = drive + .delete_document_for_contract_with_named_type_operations( document_id, &contract, "person", - BlockInfo::default(), - true, + &BlockInfo::default_with_time(1_000), + None, + None, + &mut None, None, platform_version, - Some(&EPOCH_CHANGE_FEE_VERSION_TEST), ) - .expect_err("expected deleting a history-keeping document to fail"); + .expect("expected the delete to build"); + drive + .apply_batch_low_level_drive_operations( + None, + None, + operations, + &mut vec![], + &platform_version.drive, + ) + .expect("expected the delete to apply"); + assert_deleted_at(&drive, &contract, document_id); - assert!(matches!( - err, - Error::Drive(DriveError::InvalidDeletionOfDocumentThatKeepsHistory(_)) - )); + let (drive, contract, document_id) = setup(); + drive + .delete_document_for_contract_apply_and_add_to_operations( + document_id, + &contract, + "person", + &BlockInfo::default_with_time(1_000), + None, + None, + None, + &mut vec![], + platform_version, + ) + .expect("expected the delete to apply"); + assert_deleted_at(&drive, &contract, document_id); } // ---------- Error-path tests (added for coverage) ---------- @@ -1442,10 +1661,10 @@ mod tests { random_document_id, nonexistent_contract_id, "profile", - &epoch, + &BlockInfo::default_with_epoch(epoch), + None, None, &mut estimated_costs_only_with_layer_info, - 0, None, platform_version, ) diff --git a/packages/rs-drive/src/drive/document/delete/remove_document_from_primary_storage/mod.rs b/packages/rs-drive/src/drive/document/delete/remove_document_from_primary_storage/mod.rs index b158eb7ed33..9d2f5b4b52d 100644 --- a/packages/rs-drive/src/drive/document/delete/remove_document_from_primary_storage/mod.rs +++ b/packages/rs-drive/src/drive/document/delete/remove_document_from_primary_storage/mod.rs @@ -1,4 +1,5 @@ mod v0; +mod v1; use crate::drive::Drive; use crate::error::drive::DriveError; @@ -56,9 +57,18 @@ impl Drive { batch_operations, platform_version, ), + 1 => self.remove_document_from_primary_storage_v1( + document_id, + document_type, + contract_documents_primary_key_path, + estimated_costs_only_with_layer_info, + transaction, + batch_operations, + platform_version, + ), version => Err(Error::Drive(DriveError::UnknownVersionMismatch { method: "remove_document_from_primary_storage".to_string(), - known_versions: vec![0], + known_versions: vec![0, 1], received: version, })), } diff --git a/packages/rs-drive/src/drive/document/delete/remove_document_from_primary_storage/v1/mod.rs b/packages/rs-drive/src/drive/document/delete/remove_document_from_primary_storage/v1/mod.rs new file mode 100644 index 00000000000..72984f48644 --- /dev/null +++ b/packages/rs-drive/src/drive/document/delete/remove_document_from_primary_storage/v1/mod.rs @@ -0,0 +1,109 @@ +use crate::drive::document::primary_key_tree_type::DocumentTypePrimaryKeyTreeType; +use grovedb::batch::KeyInfoPath; + +use grovedb::{EstimatedLayerInformation, MaybeTree, TransactionArg}; + +use dpp::data_contract::document_type::DocumentTypeRef; + +use std::collections::HashMap; + +use crate::util::grove_operations::BatchDeleteApplyType::{ + StatefulBatchDelete, StatelessBatchDelete, +}; + +use crate::drive::constants::DOCUMENT_HISTORY_CURRENT_REFERENCE_PATH_SIZE; +use crate::drive::Drive; + +use crate::error::drive::DriveError; +use crate::error::Error; +use crate::fees::op::LowLevelDriveOperation; + +use dpp::data_contract::document_type::accessors::DocumentTypeV0Getters; +use dpp::data_contract::document_type::methods::DocumentTypeV0Methods; +use dpp::identifier::Identifier; + +use crate::util::type_constants::DEFAULT_HASH_SIZE_U32; +use dpp::version::PlatformVersion; + +impl Drive { + /// Removes one entry from a document type's primary-key tree. + /// + /// A keep-history type's entry is the document's current pointer, a + /// reference rather than an item, so the stateless size the dry run + /// charges comes from the reference shape. Removing it is still removing a + /// leaf: the revisions the pointer names live in the type's history tree, + /// which this never touches. + #[inline(always)] + #[allow(clippy::too_many_arguments)] + pub(super) fn remove_document_from_primary_storage_v1( + &self, + document_id: Identifier, + document_type: DocumentTypeRef, + contract_documents_primary_key_path: [&[u8]; 5], + estimated_costs_only_with_layer_info: &mut Option< + HashMap, + >, + transaction: TransactionArg, + batch_operations: &mut Vec, + platform_version: &PlatformVersion, + ) -> Result<(), Error> { + let keeps_history = document_type.documents_keep_history(); + if keeps_history + && platform_version + .drive + .methods + .document + .insert + .add_document_to_primary_storage + == 0 + { + // Under the layout that writer produces, a keep-history document's + // primary entry is the subtree holding its revisions, and removing + // it as a leaf would unlink that subtree's storage without + // reclaiming it. This version exists only alongside the writer that + // stores a pointer there instead. + return Err(Error::Drive(DriveError::CorruptedCodeExecution( + "a keep-history primary entry can only be removed where the current pointer \ + layout is in use", + ))); + } + + let primary_key_tree_type = document_type.primary_key_tree_type(platform_version)?; + + let apply_type = if estimated_costs_only_with_layer_info.is_some() { + StatelessBatchDelete { + in_tree_type: primary_key_tree_type, + estimated_key_size: DEFAULT_HASH_SIZE_U32, + estimated_value_size: if keeps_history { + DOCUMENT_HISTORY_CURRENT_REFERENCE_PATH_SIZE + } else { + document_type.estimated_size(platform_version)? as u32 + }, + } + } else { + // The entry is a document item, or a current pointer to one; never + // a tree. + StatefulBatchDelete { + is_known_to_be_subtree_with_sum: Some(MaybeTree::NotTree), + } + }; + self.batch_delete( + (&contract_documents_primary_key_path).into(), + document_id.as_slice(), + apply_type, + transaction, + batch_operations, + &platform_version.drive, + )?; + + if let Some(estimated_costs_only_with_layer_info) = estimated_costs_only_with_layer_info { + Self::add_estimation_costs_for_remove_document_to_primary_storage( + contract_documents_primary_key_path, + document_type, + estimated_costs_only_with_layer_info, + platform_version, + )?; + } + Ok(()) + } +} diff --git a/packages/rs-drive/src/drive/document/estimation_costs/add_estimation_costs_for_erase_document/mod.rs b/packages/rs-drive/src/drive/document/estimation_costs/add_estimation_costs_for_erase_document/mod.rs new file mode 100644 index 00000000000..b9de485a608 --- /dev/null +++ b/packages/rs-drive/src/drive/document/estimation_costs/add_estimation_costs_for_erase_document/mod.rs @@ -0,0 +1,61 @@ +mod v0; + +use std::collections::HashMap; + +use dpp::data_contract::document_type::DocumentTypeRef; +use dpp::data_contract::DataContract; +use dpp::identifier::Identifier; +use dpp::version::PlatformVersion; +use grovedb::batch::KeyInfoPath; +use grovedb::EstimatedLayerInformation; + +use crate::drive::Drive; +use crate::error::drive::DriveError; +use crate::error::Error; + +impl Drive { + /// Registers the layers an erase chunk touches. + /// + /// # Parameters + /// * `document_id`: The document whose revisions are being removed. + /// * `contract`: The contract that contains the document. + /// * `document_type`: The type of the document, which must keep history. + /// * `layers`: The estimated layer information the dry run builds. + /// * `platform_version`: The platform version to select the correct function version to run. + /// + /// # Returns + /// * `Ok(())` if the operation was successful. + /// * `Err(DriveError::UnknownVersionMismatch)` if the drive version does not match known versions. + pub(crate) fn add_estimation_costs_for_erase_document( + document_id: Identifier, + contract: &DataContract, + document_type: DocumentTypeRef, + layers: &mut HashMap, + platform_version: &PlatformVersion, + ) -> Result<(), Error> { + match platform_version + .drive + .methods + .document + .delete + .add_estimation_costs_for_erase_document + { + Some(0) => Self::add_estimation_costs_for_erase_document_v0( + document_id, + contract, + document_type, + layers, + platform_version, + ), + Some(version) => Err(Error::Drive(DriveError::UnknownVersionMismatch { + method: "add_estimation_costs_for_erase_document".to_string(), + known_versions: vec![0], + received: version, + })), + None => Err(Error::Drive(DriveError::VersionNotActive { + method: "add_estimation_costs_for_erase_document".to_string(), + known_versions: vec![0], + })), + } + } +} diff --git a/packages/rs-drive/src/drive/document/estimation_costs/add_estimation_costs_for_erase_document/v0/mod.rs b/packages/rs-drive/src/drive/document/estimation_costs/add_estimation_costs_for_erase_document/v0/mod.rs new file mode 100644 index 00000000000..9f75ab787d1 --- /dev/null +++ b/packages/rs-drive/src/drive/document/estimation_costs/add_estimation_costs_for_erase_document/v0/mod.rs @@ -0,0 +1,82 @@ +use std::collections::HashMap; + +use dpp::data_contract::accessors::v0::DataContractV0Getters; +use dpp::data_contract::document_type::accessors::DocumentTypeV0Getters; +use dpp::data_contract::document_type::methods::DocumentTypeV0Methods; +use dpp::data_contract::document_type::DocumentTypeRef; +use dpp::data_contract::DataContract; +use dpp::identifier::Identifier; +use dpp::version::PlatformVersion; +use grovedb::batch::KeyInfoPath; +use grovedb::EstimatedLayerCount::PotentiallyAtMaxElements; +use grovedb::EstimatedLayerInformation; +use grovedb::EstimatedLayerSizes::{AllItems, AllSubtrees}; +use grovedb::EstimatedSumTrees::AllProvableCountTrees; +use grovedb::TreeType; + +use crate::drive::document::paths::{contract_document_type_path_vec, DOCUMENT_HISTORY_TREE_KEY}; +use crate::drive::Drive; +use crate::error::Error; +use crate::util::storage_flags::StorageFlags; + +impl Drive { + /// Registers the layers an erase chunk touches: the type's history tree, + /// the document's own revision tree, and the lifecycle tree the record is + /// written to or removed from. + /// + /// The revision tree is sized for a document that may retain far more + /// revisions than a chunk removes, so its height is bounded rather than + /// derived from the chunk size; an estimate keyed to the chunk would + /// understate the merk path of a long history. + #[inline(always)] + pub(super) fn add_estimation_costs_for_erase_document_v0( + document_id: Identifier, + contract: &DataContract, + document_type: DocumentTypeRef, + layers: &mut HashMap, + platform_version: &PlatformVersion, + ) -> Result<(), Error> { + Self::add_estimation_costs_for_levels_up_to_contract_document_type_excluded( + contract, + layers, + &platform_version.drive, + )?; + Self::add_estimation_costs_for_lifecycle_record( + contract, + document_type, + layers, + platform_version, + )?; + + let flags_size = Some(StorageFlags::approximate_size(true, None)); + let mut history_root = contract_document_type_path_vec( + contract.id_ref().as_bytes(), + document_type.name().as_str(), + ); + history_root.push(vec![DOCUMENT_HISTORY_TREE_KEY]); + layers.insert( + KeyInfoPath::from_known_owned_path(history_root.clone()), + EstimatedLayerInformation { + tree_type: TreeType::NormalTree, + estimated_layer_count: PotentiallyAtMaxElements, + estimated_layer_sizes: AllSubtrees(32, AllProvableCountTrees, flags_size), + }, + ); + + let mut history_path = history_root; + history_path.push(document_id.to_vec()); + layers.insert( + KeyInfoPath::from_known_owned_path(history_path), + EstimatedLayerInformation { + tree_type: TreeType::ProvableCountTree, + estimated_layer_count: PotentiallyAtMaxElements, + estimated_layer_sizes: AllItems( + 16, + document_type.max_size(platform_version)? as u32, + flags_size, + ), + }, + ); + Ok(()) + } +} diff --git a/packages/rs-drive/src/drive/document/estimation_costs/add_estimation_costs_for_lifecycle_record/mod.rs b/packages/rs-drive/src/drive/document/estimation_costs/add_estimation_costs_for_lifecycle_record/mod.rs new file mode 100644 index 00000000000..6d1fc26be2c --- /dev/null +++ b/packages/rs-drive/src/drive/document/estimation_costs/add_estimation_costs_for_lifecycle_record/mod.rs @@ -0,0 +1,60 @@ +mod v0; + +use std::collections::HashMap; + +use dpp::data_contract::document_type::DocumentTypeRef; +use dpp::data_contract::DataContract; +use dpp::version::PlatformVersion; +use grovedb::batch::KeyInfoPath; +use grovedb::EstimatedLayerInformation; + +use crate::drive::Drive; +use crate::error::drive::DriveError; +use crate::error::Error; + +impl Drive { + /// Registers the layers a lifecycle record write or removal touches. + /// + /// The record is written by a delete of a keep-history document and + /// removed by the terminal erase chunk, so both estimate through this one + /// entry point; its own version slot keeps a later change to the layer + /// counts or record size from moving the estimates of the generations + /// that already rely on it. + /// + /// # Parameters + /// * `contract`: The contract that contains the document type. + /// * `document_type`: The type whose lifecycle tree holds the record. + /// * `layers`: The estimated layer information the dry run builds. + /// * `platform_version`: The platform version to select the correct function version to run. + /// + /// # Returns + /// * `Ok(())` if the operation was successful. + /// * `Err(DriveError::UnknownVersionMismatch)` if the drive version does not match known versions. + pub(crate) fn add_estimation_costs_for_lifecycle_record( + contract: &DataContract, + document_type: DocumentTypeRef, + layers: &mut HashMap, + platform_version: &PlatformVersion, + ) -> Result<(), Error> { + match platform_version + .drive + .methods + .document + .delete + .add_estimation_costs_for_lifecycle_record + { + Some(0) => { + Self::add_estimation_costs_for_lifecycle_record_v0(contract, document_type, layers) + } + Some(version) => Err(Error::Drive(DriveError::UnknownVersionMismatch { + method: "add_estimation_costs_for_lifecycle_record".to_string(), + known_versions: vec![0], + received: version, + })), + None => Err(Error::Drive(DriveError::VersionNotActive { + method: "add_estimation_costs_for_lifecycle_record".to_string(), + known_versions: vec![0], + })), + } + } +} diff --git a/packages/rs-drive/src/drive/document/estimation_costs/add_estimation_costs_for_lifecycle_record/v0/mod.rs b/packages/rs-drive/src/drive/document/estimation_costs/add_estimation_costs_for_lifecycle_record/v0/mod.rs new file mode 100644 index 00000000000..258a0aad544 --- /dev/null +++ b/packages/rs-drive/src/drive/document/estimation_costs/add_estimation_costs_for_lifecycle_record/v0/mod.rs @@ -0,0 +1,61 @@ +use std::collections::HashMap; + +use dpp::data_contract::accessors::v0::DataContractV0Getters; +use dpp::data_contract::document_type::accessors::DocumentTypeV0Getters; +use dpp::data_contract::document_type::DocumentTypeRef; +use dpp::data_contract::DataContract; +use grovedb::batch::KeyInfoPath; +use grovedb::EstimatedLayerCount::{ApproximateElements, PotentiallyAtMaxElements}; +use grovedb::EstimatedLayerInformation; +use grovedb::EstimatedLayerSizes::{AllItems, AllSubtrees}; +use grovedb::EstimatedSumTrees::NoSumTrees; +use grovedb::TreeType; + +use crate::drive::document::paths::{contract_document_type_path_vec, document_lifecycle_path}; +use crate::drive::Drive; +use crate::error::Error; +use crate::util::storage_flags::StorageFlags; +use dpp::document::lifecycle::DOCUMENT_LIFECYCLE_RECORD_MAX_SIZE; + +impl Drive { + /// Registers the layers a lifecycle record write or removal touches: the + /// document type itself, whose lifecycle tree may have to be created, and + /// the lifecycle tree holding one fixed-size record per deleted document. + /// + /// A dry run that leaves either layer unregistered fails outright, so both + /// are always registered even when the tree already exists. + pub(super) fn add_estimation_costs_for_lifecycle_record_v0( + contract: &DataContract, + document_type: DocumentTypeRef, + layers: &mut HashMap, + ) -> Result<(), Error> { + let flags_size = Some(StorageFlags::approximate_size(true, None)); + layers.insert( + KeyInfoPath::from_known_owned_path(contract_document_type_path_vec( + contract.id_ref().as_bytes(), + document_type.name().as_str(), + )), + EstimatedLayerInformation { + tree_type: TreeType::NormalTree, + // The primary-key tree, the lifecycle tree, the history tree + // and one tree per index. + estimated_layer_count: ApproximateElements( + document_type.indexes().len() as u32 + 3, + ), + estimated_layer_sizes: AllSubtrees(1, NoSumTrees, flags_size), + }, + ); + layers.insert( + KeyInfoPath::from_known_owned_path(document_lifecycle_path( + contract.id_ref().as_bytes(), + document_type.name().as_str(), + )), + EstimatedLayerInformation { + tree_type: TreeType::NormalTree, + estimated_layer_count: PotentiallyAtMaxElements, + estimated_layer_sizes: AllItems(32, DOCUMENT_LIFECYCLE_RECORD_MAX_SIZE, flags_size), + }, + ); + Ok(()) + } +} diff --git a/packages/rs-drive/src/drive/document/estimation_costs/mod.rs b/packages/rs-drive/src/drive/document/estimation_costs/mod.rs index 4a3acee1aed..d8e89847f8a 100644 --- a/packages/rs-drive/src/drive/document/estimation_costs/mod.rs +++ b/packages/rs-drive/src/drive/document/estimation_costs/mod.rs @@ -4,4 +4,8 @@ mod add_estimation_costs_for_add_document_to_primary_storage; mod add_estimation_costs_for_add_contested_document_to_primary_storage; +mod add_estimation_costs_for_lifecycle_record; + +mod add_estimation_costs_for_erase_document; + pub(crate) mod estimated_sum_trees_for_value_tree_type; diff --git a/packages/rs-drive/src/drive/document/insert/add_document_for_contract/time_range_index_e2e_tests.rs b/packages/rs-drive/src/drive/document/insert/add_document_for_contract/time_range_index_e2e_tests.rs index 5deaf230552..7282109e6fd 100644 --- a/packages/rs-drive/src/drive/document/insert/add_document_for_contract/time_range_index_e2e_tests.rs +++ b/packages/rs-drive/src/drive/document/insert/add_document_for_contract/time_range_index_e2e_tests.rs @@ -505,6 +505,7 @@ fn assert_ttl_document_batch(case: TtlBatchCase, ttl: bool) { .map(|i| { DriveOperation::DocumentOperation(DocumentOperationType::DeleteDocument { document_id: docs[i].id(), + deleter_id: None, contract_info: DataContractInfo::BorrowedDataContract(&contract), document_type_info: DocumentTypeInfo::DocumentTypeRef(dt), }) @@ -736,6 +737,7 @@ fn raw_conversion_refuses_ttl_operations_and_requires_a_transaction() { vec![DriveOperation::DocumentOperation( DocumentOperationType::DeleteDocument { document_id: doc.id(), + deleter_id: None, contract_info: DataContractInfo::BorrowedDataContract(&contract), document_type_info: DocumentTypeInfo::DocumentTypeRef(dt), }, @@ -3819,9 +3821,10 @@ fn ttl_budget_boundary_after_zero_tree_keeps_deletes_exact() { document.id(), &contract, document_type, + &BlockInfo::default_with_time(after_expiry_ms), + None, None, &mut None, - after_expiry_ms, None, platform_version, ) @@ -4887,6 +4890,7 @@ fn apply_drive_operations_rolls_back_ttl_preparation_when_conversion_fails_witho vec![DriveOperation::DocumentOperation( DocumentOperationType::DeleteDocument { document_id: missing, + deleter_id: None, contract_info: DataContractInfo::BorrowedDataContract(&contract), document_type_info: DocumentTypeInfo::DocumentTypeRef(document_type), }, diff --git a/packages/rs-drive/src/drive/document/insert/add_document_for_contract_operations/mod.rs b/packages/rs-drive/src/drive/document/insert/add_document_for_contract_operations/mod.rs index ba4d552f4be..9c9908a6a56 100644 --- a/packages/rs-drive/src/drive/document/insert/add_document_for_contract_operations/mod.rs +++ b/packages/rs-drive/src/drive/document/insert/add_document_for_contract_operations/mod.rs @@ -1,5 +1,6 @@ mod v0; mod v1; +mod v2; use crate::drive::Drive; use crate::error::drive::DriveError; @@ -91,9 +92,21 @@ impl Drive { transaction, platform_version, ), + // v2: a fresh insert of a keep-history document never lets the + // caller's override skip the primary-storage checks, so a deleted + // document's retained revisions cannot be written over. + 2 => self.add_document_for_contract_operations_v2( + document_and_contract_info, + override_document, + block_info, + previous_batch_operations, + estimated_costs_only_with_layer_info, + transaction, + platform_version, + ), version => Err(Error::Drive(DriveError::UnknownVersionMismatch { method: "add_document_for_contract_operations".to_string(), - known_versions: vec![0, 1], + known_versions: vec![0, 1, 2], received: version, })), } diff --git a/packages/rs-drive/src/drive/document/insert/add_document_for_contract_operations/v2/mod.rs b/packages/rs-drive/src/drive/document/insert/add_document_for_contract_operations/v2/mod.rs new file mode 100644 index 00000000000..05be13227e7 --- /dev/null +++ b/packages/rs-drive/src/drive/document/insert/add_document_for_contract_operations/v2/mod.rs @@ -0,0 +1,205 @@ +use crate::drive::document::paths::contract_documents_primary_key_path; +use crate::drive::document::primary_key_tree_type::DocumentTypePrimaryKeyTreeType; +use crate::drive::Drive; +use crate::error::Error; +use crate::fees::op::LowLevelDriveOperation; +use crate::util::grove_operations::DirectQueryType::{StatefulDirectQuery, StatelessDirectQuery}; +use crate::util::grove_operations::QueryTarget::QueryTargetValue; +use crate::util::object_size_info::{DocumentAndContractInfo, DocumentInfoV0Methods}; +use dpp::block::block_info::BlockInfo; +use dpp::data_contract::accessors::v0::DataContractV0Getters; +use dpp::data_contract::document_type::accessors::{DocumentTypeV0Getters, DocumentTypeV2Getters}; +use dpp::data_contract::document_type::methods::DocumentTypeV0Methods; + +use dpp::version::PlatformVersion; +use grovedb::batch::KeyInfoPath; +use grovedb::{EstimatedLayerInformation, TransactionArg}; +use std::collections::HashMap; + +impl Drive { + /// Gathers the operations to add a document to a contract. + /// + /// v2 is v1 with one difference in how a fresh insert of a keep-history + /// document reaches primary storage: `override_document` no longer lets + /// the primary-storage writer skip its own checks once the document turned + /// out not to exist. Under the keep-history layout a deleted document has + /// no current entry but still retains its revisions, and the writer + /// refuses to create over them only when it is allowed to check; a + /// caller's override speaks for the current entry alone, so the insert + /// path does not forward it for such a type. Types that keep no history + /// have nothing behind a missing entry and keep v1's cheaper write. + #[inline(always)] + #[allow(clippy::too_many_arguments)] + pub(super) fn add_document_for_contract_operations_v2( + &self, + document_and_contract_info: DocumentAndContractInfo, + override_document: bool, + block_info: &BlockInfo, + previous_batch_operations: &mut Option<&mut Vec>, + estimated_costs_only_with_layer_info: &mut Option< + HashMap, + >, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result, Error> { + let mut batch_operations: Vec = vec![]; + + // indexOnly document types have no primary-storage row and no + // primary-key tree at all — the index entries are the rows, and + // updates are impossible by construction (immutable, no revision). + // `index_only()` can only be true on a PV14+ contract (the grammar + // rejects the keyword below meta-schema v3), so this branch is + // unreachable for every historical document. + if document_and_contract_info.document_type.index_only() { + if let Some(estimated_costs_only_with_layer_info) = estimated_costs_only_with_layer_info + { + Self::add_estimation_costs_for_levels_up_to_contract_document_type_excluded( + document_and_contract_info.contract, + estimated_costs_only_with_layer_info, + &platform_version.drive, + )?; + } + + self.add_indices_for_top_index_level_for_contract_operations( + &document_and_contract_info, + previous_batch_operations, + estimated_costs_only_with_layer_info, + block_info.time_ms, + transaction, + &mut batch_operations, + platform_version, + )?; + + self.add_preallocated_index_tree_operations_for_referring_types( + &document_and_contract_info, + previous_batch_operations, + estimated_costs_only_with_layer_info, + transaction, + &mut batch_operations, + platform_version, + )?; + + return Ok(batch_operations); + } + + let primary_key_path = contract_documents_primary_key_path( + document_and_contract_info.contract.id_ref().as_bytes(), + document_and_contract_info.document_type.name().as_str(), + ); + + let primary_key_tree_type = document_and_contract_info + .document_type + .primary_key_tree_type(platform_version)?; + + // Apply means stateful query + let query_type = if estimated_costs_only_with_layer_info.is_none() { + StatefulDirectQuery + } else { + StatelessDirectQuery { + in_tree_type: primary_key_tree_type, + query_target: QueryTargetValue( + document_and_contract_info + .document_type + .estimated_size(platform_version)? as u32, + ), + } + }; + + // To update but not create: + + // 1. Override should be allowed + let could_be_update = override_document; + + // 2. Is not a dry run + let could_be_update = could_be_update + && !document_and_contract_info + .owned_document_info + .document_info + .is_document_size(); + + // 3. Document exists in storage + let is_update = could_be_update + && self.grove_has_raw( + primary_key_path.as_ref().into(), + document_and_contract_info + .owned_document_info + .document_info + .id_key_value_info() + .as_key_ref_request()?, + query_type, + transaction, + &mut batch_operations, + &platform_version.drive, + )?; + + if is_update { + let update_operations = self + .update_document_for_contract_operations_without_ttl_drain( + document_and_contract_info, + block_info, + previous_batch_operations, + estimated_costs_only_with_layer_info, + transaction, + platform_version, + )?; + + batch_operations.extend(update_operations); + + return Ok(batch_operations); + } + + // if we are trying to get estimated costs we need to add the upper levels + if let Some(estimated_costs_only_with_layer_info) = estimated_costs_only_with_layer_info { + Self::add_estimation_costs_for_levels_up_to_contract_document_type_excluded( + document_and_contract_info.contract, + estimated_costs_only_with_layer_info, + &platform_version.drive, + )?; + } + + // The document has no current entry, whatever the caller assumed. For + // a keep-history type that entry is not the whole story, so the writer + // performs its own checks, including the refusal to create over a + // deleted document's retained revisions. + let insert_without_check = override_document + && !document_and_contract_info + .document_type + .documents_keep_history(); + self.add_document_to_primary_storage( + &document_and_contract_info, + block_info, + insert_without_check, + estimated_costs_only_with_layer_info, + transaction, + &mut batch_operations, + platform_version, + )?; + + self.add_indices_for_top_index_level_for_contract_operations( + &document_and_contract_info, + previous_batch_operations, + estimated_costs_only_with_layer_info, + block_info.time_ms, + transaction, + &mut batch_operations, + platform_version, + )?; + + // If any indexOnly document type of this contract carries a + // `preallocated` index bound to this document type through a + // refersTo declaration, create that index's trees for entries + // referencing this document now — see + // `add_preallocated_index_tree_operations`. A no-op for every + // contract without the (PV14+, indexOnly-only) flag. + self.add_preallocated_index_tree_operations_for_referring_types( + &document_and_contract_info, + previous_batch_operations, + estimated_costs_only_with_layer_info, + transaction, + &mut batch_operations, + platform_version, + )?; + + Ok(batch_operations) + } +} diff --git a/packages/rs-drive/src/drive/document/insert/add_document_to_primary_storage/v1/mod.rs b/packages/rs-drive/src/drive/document/insert/add_document_to_primary_storage/v1/mod.rs index 072102612e1..d9ab8dbcf63 100644 --- a/packages/rs-drive/src/drive/document/insert/add_document_to_primary_storage/v1/mod.rs +++ b/packages/rs-drive/src/drive/document/insert/add_document_to_primary_storage/v1/mod.rs @@ -100,7 +100,7 @@ impl Drive { BatchInsertTreeApplyType::StatefulBatchInsertTree }; if let Some(document) = document { - self.batch_insert_empty_tree_if_not_exists::<0>( + let created = self.batch_insert_empty_tree_if_not_exists::<0>( PathKey((history_root, document.id().to_vec())), TreeType::ProvableCountTree, flags, @@ -110,6 +110,21 @@ impl Drive { operations, version, )?; + // An id whose revisions are still retained is taken, even though + // nothing in the primary-key tree says so. Appending to that + // history would silently merge a new document into a deleted one's + // record, so it is refused here rather than only in transition + // validation: this is the guard that covers writers outside it. + // Only the update path asks to write without checks, and its + // document holds the id already, so its history is meant to exist; + // the insert path always lets this check run. + // A dry run skips the probe, which reports the tree as absent, and + // pays for it as a fixed cost so estimation and execution agree. + if !created && !insert_without_check { + return Err(Error::Drive(DriveError::CorruptedDocumentAlreadyExists( + "a document of this id still retains revisions and can not be created until they are erased", + ))); + } } else { operations.push( LowLevelDriveOperation::for_estimated_path_key_empty_provable_count_tree( diff --git a/packages/rs-drive/src/drive/document/lifecycle/fetch/mod.rs b/packages/rs-drive/src/drive/document/lifecycle/fetch/mod.rs new file mode 100644 index 00000000000..3dce314e1a3 --- /dev/null +++ b/packages/rs-drive/src/drive/document/lifecycle/fetch/mod.rs @@ -0,0 +1,91 @@ +mod v0; + +use dpp::block::epoch::Epoch; +use dpp::data_contract::document_type::DocumentTypeRef; +use dpp::data_contract::DataContract; +use dpp::document::Document; +use dpp::fee::fee_result::FeeResult; +use dpp::identifier::Identifier; +use dpp::version::PlatformVersion; +use grovedb::TransactionArg; + +use crate::drive::Drive; +use crate::error::drive::DriveError; +use crate::error::Error; + +/// Where one keep-history document stands in its lifecycle. +/// +/// Ordinary reads cannot tell a deleted document from one that never existed — +/// both are absent from the primary-key tree — so every check that must +/// distinguish them goes through this. +#[derive(Debug, Clone, PartialEq)] +pub enum DocumentLifecycleState { + /// The document is current and visible to ordinary reads. + Active(Box), + /// The document has been deleted and its revisions are retained. Carries + /// the newest of them, which is where a deleted document's owner is read + /// from. + Deleted(Box), + /// An authorized erasure has started and has not finished. Anyone may + /// continue it, so nothing about the document's owner is needed here. + Erasing, + /// No document, no retained revisions and no record: the id is free. + Absent, +} + +impl DocumentLifecycleState { + /// Whether the id is taken, which is what a create has to know. + pub fn is_present(&self) -> bool { + !matches!(self, DocumentLifecycleState::Absent) + } +} + +impl Drive { + /// Classifies one keep-history document as active, deleted, erasing or + /// absent, and returns the fee for the reads it performed. + /// + /// The reads are ordered by how likely they are to settle the question: the + /// ordinary by-id read answers it for every live document, and only a miss + /// pays for the lifecycle record. A deleted document costs one read more, + /// for the newest revision it retains. + /// + /// This is deliberately not folded into the by-id fetch every document + /// action already performs: only stateful validation of an action on a + /// keep-history type needs the extra reads, and every other document fetch + /// would otherwise pay for them. + pub fn fetch_document_lifecycle( + &self, + contract: &DataContract, + document_type: DocumentTypeRef, + document_id: Identifier, + epoch: Option<&Epoch>, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result<(DocumentLifecycleState, FeeResult), Error> { + match platform_version + .drive + .methods + .document + .query + .fetch_document_lifecycle + { + Some(0) => self.fetch_document_lifecycle_v0( + contract, + document_type, + document_id, + epoch, + transaction, + platform_version, + ), + Some(version) => Err(Error::Drive(DriveError::UnknownVersionMismatch { + method: "fetch_document_lifecycle".to_string(), + known_versions: vec![0], + received: version, + })), + None => Err(Error::Drive(DriveError::VersionNotActive { + method: "fetch_document_lifecycle".to_string(), + known_versions: vec![0], + })), + } + } +} diff --git a/packages/rs-drive/src/drive/document/lifecycle/fetch/v0/mod.rs b/packages/rs-drive/src/drive/document/lifecycle/fetch/v0/mod.rs new file mode 100644 index 00000000000..d3b5c2d6383 --- /dev/null +++ b/packages/rs-drive/src/drive/document/lifecycle/fetch/v0/mod.rs @@ -0,0 +1,160 @@ +use dpp::block::epoch::Epoch; +use dpp::data_contract::accessors::v0::DataContractV0Getters; +use dpp::data_contract::document_type::accessors::DocumentTypeV0Getters; +use dpp::data_contract::document_type::DocumentTypeRef; +use dpp::data_contract::DataContract; +use dpp::document::serialization_traits::DocumentPlatformConversionMethodsV0; +use dpp::document::Document; +use dpp::fee::fee_result::FeeResult; +use dpp::identifier::Identifier; +use dpp::version::PlatformVersion; +use grovedb::query_result_type::QueryResultType; +use grovedb::{Element, PathQuery, Query, SizedQuery, TransactionArg}; + +use crate::drive::document::lifecycle::fetch::DocumentLifecycleState; +use crate::drive::document::paths::{ + contract_documents_primary_key_path, document_history_path, document_lifecycle_path, +}; +use crate::drive::Drive; +use crate::error::drive::DriveError; +use crate::error::Error; +use crate::fees::op::LowLevelDriveOperation; +use crate::util::grove_operations::{DirectQueryType, QueryType}; +use dpp::document::lifecycle::DocumentLifecycleRecord; +use dpp::serialization::PlatformDeserializableTrusted; + +impl Drive { + #[inline(always)] + pub(super) fn fetch_document_lifecycle_v0( + &self, + contract: &DataContract, + document_type: DocumentTypeRef, + document_id: Identifier, + epoch: Option<&Epoch>, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result<(DocumentLifecycleState, FeeResult), Error> { + if !document_type.documents_keep_history() { + return Err(Error::Drive(DriveError::CorruptedCodeExecution( + "only a document type that keeps history has a lifecycle to fetch", + ))); + } + let mut operations: Vec = vec![]; + let state = self.read_document_lifecycle( + contract, + document_type, + document_id, + &mut operations, + transaction, + platform_version, + )?; + let fee = match epoch { + Some(epoch) => Drive::calculate_fee( + None, + Some(operations), + epoch, + self.config.epochs_per_era, + platform_version, + None, + )?, + None => FeeResult::default(), + }; + Ok((state, fee)) + } + + fn read_document_lifecycle( + &self, + contract: &DataContract, + document_type: DocumentTypeRef, + document_id: Identifier, + operations: &mut Vec, + transaction: TransactionArg, + platform_version: &PlatformVersion, + ) -> Result { + let primary_path = contract_documents_primary_key_path( + contract.id_ref().as_bytes(), + document_type.name().as_str(), + ); + let pointer = self.grove_get_raw_optional( + (&primary_path).into(), + document_id.as_slice(), + DirectQueryType::StatefulDirectQuery, + transaction, + operations, + &platform_version.drive, + )?; + if pointer.is_some() { + let element = self + .grove_get( + (&primary_path).into(), + document_id.as_slice(), + QueryType::StatefulQuery, + transaction, + operations, + &platform_version.drive, + )? + .ok_or(Error::Drive(DriveError::CorruptedDriveState( + "a current pointer resolved to nothing".to_string(), + )))?; + let Element::Item(bytes, _) = element else { + return Err(Error::Drive(DriveError::CorruptedElementType( + "a current pointer did not resolve to a document", + ))); + }; + return Ok(DocumentLifecycleState::Active(Box::new( + Document::from_bytes(&bytes, document_type, platform_version)?, + ))); + } + + let lifecycle_path = + document_lifecycle_path(contract.id_ref().as_bytes(), document_type.name().as_str()); + // A document type whose documents have never been deleted has no + // lifecycle tree at all, so a missing path means the same as a missing + // key: the id is free. + let record_bytes = self.grove_get_raw_optional_item( + lifecycle_path.as_slice().into(), + document_id.as_slice(), + DirectQueryType::StatefulDirectQuery, + transaction, + operations, + &platform_version.drive, + )?; + let Some(record_bytes) = record_bytes else { + return Ok(DocumentLifecycleState::Absent); + }; + let record = DocumentLifecycleRecord::deserialize_from_bytes_trusted(&record_bytes)?; + if record.is_erasing() { + return Ok(DocumentLifecycleState::Erasing); + } + + // A deleted document's owner is read from the newest revision it still + // retains: a plain reverse range over the revision key space, immune to + // the composite-key bounds a time selector has to get right. + let mut query = Query::new_with_direction(false); + query.insert_all(); + let path_query = PathQuery::new( + document_history_path( + contract.id_ref().as_bytes(), + document_type.name().as_str(), + document_id.as_slice(), + ), + SizedQuery::new(query, Some(1), None), + ); + let (results, _) = self.grove_get_raw_path_query( + &path_query, + transaction, + QueryResultType::QueryKeyElementPairResultType, + operations, + &platform_version.drive, + )?; + let Some((_, Element::Item(bytes, _))) = results.to_key_elements().into_iter().next() + else { + return Err(Error::Drive(DriveError::CorruptedDriveState( + "a deleted document retains no revision to read its owner from".to_string(), + ))); + }; + Ok(DocumentLifecycleState::Deleted(Box::new( + Document::from_bytes(&bytes, document_type, platform_version)?, + ))) + } +} diff --git a/packages/rs-drive/src/drive/document/lifecycle/mod.rs b/packages/rs-drive/src/drive/document/lifecycle/mod.rs new file mode 100644 index 00000000000..c3576b7906d --- /dev/null +++ b/packages/rs-drive/src/drive/document/lifecycle/mod.rs @@ -0,0 +1,19 @@ +//! The per-type tree that holds the lifecycle records of keep-history +//! documents, and the reads over it. +//! +//! The record itself is `dpp::document::lifecycle::DocumentLifecycleRecord`. +//! It lives in its own tree under the document type rather than inside the +//! document's history, so every key in the history stays in the revision +//! domain and deleted documents stay enumerable per type. + +#[cfg(feature = "server")] +mod fetch; + +#[cfg(feature = "server")] +mod refund_recipients; + +#[cfg(all(test, feature = "server", feature = "verify"))] +mod tests; + +#[cfg(feature = "server")] +pub use fetch::DocumentLifecycleState; diff --git a/packages/rs-drive/src/drive/document/lifecycle/refund_recipients.rs b/packages/rs-drive/src/drive/document/lifecycle/refund_recipients.rs new file mode 100644 index 00000000000..f6b0aeeb7bc --- /dev/null +++ b/packages/rs-drive/src/drive/document/lifecycle/refund_recipients.rs @@ -0,0 +1,85 @@ +//! Pricing for the balance work an erase's refunds cause outside its own fee. +//! +//! Refund credits are applied after the fee result is formed, as separate +//! balance operations against identities that had nothing to do with the +//! transition. A bound on how many revisions one erase removes limits that work +//! but does not pay for it, so it is priced here and billed to the submitter +//! through the execution context, which feeds both the admission estimate and +//! the fee actually charged. + +use std::collections::HashMap; + +use dpp::block::epoch::Epoch; +use dpp::fee::fee_result::FeeResult; +use dpp::version::PlatformVersion; +use grovedb::batch::KeyInfoPath; +use grovedb::EstimatedLayerInformation; + +use crate::drive::Drive; +use crate::error::drive::DriveError; +use crate::error::Error; +use crate::fees::op::LowLevelDriveOperation; + +/// Beneficiaries an erase can credit beyond the revisions themselves: the +/// lifecycle record's writer and the per-document history subtree's creator, +/// both of which a terminal chunk removes. Delete already removed the current +/// pointer, so erase cannot refund it again. +const STRUCTURAL_REFUND_RECIPIENTS: u64 = 2; + +impl Drive { + /// Prices the worst-case number of third-party balance updates one erase + /// can cause. + /// + /// Every recipient does the same shape of work — read the balance, read the + /// debt, write one or both — so the cost of one is measured through the + /// ordinary estimation path and multiplied by how many there can be. The + /// measurement performs no balance mutation and touches no real identity: + /// it runs the balance update in estimation mode, where the reads are + /// average-case costs rather than state. + pub fn erase_refund_recipient_cost( + &self, + epoch: &Epoch, + platform_version: &PlatformVersion, + ) -> Result { + let chunk = platform_version + .system_limits + .max_document_revisions_erased_per_transition + .ok_or(Error::Drive(DriveError::NotSupported( + "erasing retained revisions is not available at this protocol version", + )))?; + let recipients = chunk as u64 + STRUCTURAL_REFUND_RECIPIENTS; + + let mut layers: Option> = + Some(HashMap::new()); + let batch = self.add_to_identity_balance_operations( + [0u8; 32], + 1, + &mut layers, + None, + platform_version, + )?; + let mut operations: Vec = vec![]; + self.apply_batch_low_level_drive_operations( + layers, + None, + batch, + &mut operations, + &platform_version.drive, + )?; + let one = Drive::calculate_fee( + None, + Some(operations), + epoch, + self.config.epochs_per_era, + platform_version, + None, + )?; + + Ok(FeeResult { + storage_fee: one.storage_fee.saturating_mul(recipients), + processing_fee: one.processing_fee.saturating_mul(recipients), + fee_refunds: Default::default(), + removed_bytes_from_system: 0, + }) + } +} diff --git a/packages/rs-drive/src/drive/document/lifecycle/tests.rs b/packages/rs-drive/src/drive/document/lifecycle/tests.rs new file mode 100644 index 00000000000..2951eb4ddcc --- /dev/null +++ b/packages/rs-drive/src/drive/document/lifecycle/tests.rs @@ -0,0 +1,2041 @@ +//! The storage side of the keep-history document lifecycle: what a delete +//! leaves behind, what an erase removes, and what a chunked erasure looks like +//! from one transition to the next. + +use super::fetch::DocumentLifecycleState; +use crate::drive::document::paths::{document_history_path, document_lifecycle_path}; +use crate::drive::Drive; +use crate::error::drive::DriveError; +use crate::error::Error; +use crate::query::document_history_drive_query::{ + DocumentHistoryDriveQuery, DocumentHistoryFilter, DocumentHistoryLifecycle, + DocumentHistoryState, +}; +use crate::query::{ + DriveDocumentQuery, SingleDocumentDriveQuery, SingleDocumentDriveQueryContestedStatus, +}; +use crate::util::grove_operations::BatchDeleteApplyType::StatefulBatchDelete; +use crate::util::object_size_info::{DocumentAndContractInfo, DocumentInfo, OwnedDocumentInfo}; +use crate::util::storage_flags::StorageFlags; +use crate::util::test_helpers::setup::setup_drive_with_initial_state_structure; +use crate::verify::document::DocumentHistoryProof; +use dpp::block::block_info::BlockInfo; +use dpp::data_contract::accessors::v0::DataContractV0Getters; +use dpp::data_contract::document_type::DocumentTypeRef; +use dpp::data_contract::DataContract; +use dpp::document::lifecycle::DocumentLifecycleRecord; +use dpp::document::{DocumentV0Getters, DocumentV0Setters}; +use dpp::identifier::Identifier; +use dpp::serialization::PlatformDeserializableUntrusted; +use dpp::tests::json_document::{json_document_to_contract, json_document_to_document}; +use dpp::version::PlatformVersion; +use grovedb::{MaybeTree, TransactionArg, TreeType}; +use std::borrow::Cow; +use std::collections::HashMap; + +const FAMILY_HISTORY_CONTRACT: &str = + "tests/supporting_files/contract/family/family-contract-with-history.json"; +const PERSON: &str = "tests/supporting_files/contract/family/person0.json"; + +fn latest() -> &'static PlatformVersion { + PlatformVersion::latest() +} + +fn chunk_size() -> u64 { + latest() + .system_limits + .max_document_revisions_erased_per_transition + .expect("protocol 15 bounds the erase chunk") as u64 +} + +/// Applies the family contract and writes `revisions` revisions of one person, +/// each at its own block time so every revision key is distinct. +fn setup_history(revisions: u64, owner: [u8; 32]) -> (Drive, DataContract, Identifier) { + let version = latest(); + let drive = setup_drive_with_initial_state_structure(None); + let contract = json_document_to_contract(FAMILY_HISTORY_CONTRACT, false, version) + .expect("expected the family history contract"); + drive + .apply_contract(&contract, BlockInfo::default(), true, None, None, version) + .expect("expected to apply the contract"); + let document_type = contract + .document_type_for_name("person") + .expect("expected the person type"); + let mut document = + json_document_to_document(PERSON, Some(owner.into()), document_type, version) + .expect("expected a person document"); + let flags = Some(Cow::Owned(StorageFlags::new_single_epoch(0, Some(owner)))); + for revision in 1..=revisions { + document.set_revision(Some(revision)); + drive + .add_document_for_contract( + DocumentAndContractInfo { + owned_document_info: OwnedDocumentInfo { + document_info: DocumentInfo::DocumentRefInfo((&document, flags.clone())), + owner_id: None, + }, + contract: &contract, + document_type, + }, + revision > 1, + BlockInfo::default_with_time(1_000 + revision), + true, + None, + version, + None, + ) + .expect("expected to write a revision"); + } + let id = document.id(); + (drive, contract, id) +} + +fn document_type_of(contract: &DataContract) -> DocumentTypeRef<'_> { + contract + .document_type_for_name("person") + .expect("expected the person type") +} + +fn lifecycle_of(drive: &Drive, contract: &DataContract, id: Identifier) -> DocumentLifecycleState { + drive + .fetch_document_lifecycle( + contract, + document_type_of(contract), + id, + None, + None, + latest(), + ) + .expect("expected to read the lifecycle") + .0 +} + +fn record_of( + drive: &Drive, + contract: &DataContract, + id: Identifier, +) -> Option { + let path = document_lifecycle_path(contract.id_ref().as_bytes(), "person"); + drive + .grove_get_raw_optional_item( + path.as_slice().into(), + id.as_slice(), + crate::util::grove_operations::DirectQueryType::StatefulDirectQuery, + None, + &mut vec![], + &latest().drive, + ) + .expect("expected to read the lifecycle tree") + .map(|bytes| { + DocumentLifecycleRecord::deserialize_from_bytes_untrusted(&bytes) + .expect("expected a valid record") + }) +} + +fn visible_document_count(drive: &Drive, contract: &DataContract) -> usize { + let query = DriveDocumentQuery::all_items_query(contract, document_type_of(contract), None); + query + .execute_raw_results_no_proof(drive, None, None, latest()) + .expect("expected to query documents") + .0 + .len() +} + +fn history_metadata( + drive: &Drive, + contract: &DataContract, + id: Identifier, +) -> DocumentHistoryLifecycle { + let query = DocumentHistoryDriveQuery { + contract_id: contract.id().to_buffer(), + document_type_name: "person".into(), + document_id: id.to_buffer(), + filter: DocumentHistoryFilter::StartAtTime(0), + limit: Some(10), + }; + let page = drive + .fetch_document_history(&query, document_type_of(contract), None, latest()) + .expect("expected to fetch history"); + let proof = drive + .prove_document_history(&query, document_type_of(contract), None, latest()) + .expect("expected to prove history"); + let (_, verified) = + Drive::verify_document_history(&query, &proof, document_type_of(contract), latest()) + .expect("expected the proof to verify"); + assert_eq!( + verified, page, + "the proved history must match the read history" + ); + verified + .lifecycle + .expect("the latest protocol authenticates lifecycle metadata") +} + +fn delete( + drive: &Drive, + contract: &DataContract, + id: Identifier, + deleter: Identifier, + time_ms: u64, +) -> dpp::fee::fee_result::FeeResult { + let mut operations = vec![]; + let block_info = BlockInfo::default_with_time(time_ms); + let batch = drive + .delete_document_for_contract_operations( + id, + contract, + document_type_of(contract), + &block_info, + Some(deleter), + None, + &mut None, + None, + latest(), + ) + .expect("expected to build the delete operations"); + drive + .apply_batch_low_level_drive_operations(None, None, batch, &mut operations, &latest().drive) + .expect("expected to apply the delete"); + Drive::calculate_fee( + None, + Some(operations), + &block_info.epoch, + drive.config.epochs_per_era, + latest(), + None, + ) + .expect("expected a fee") +} + +fn erase( + drive: &Drive, + contract: &DataContract, + id: Identifier, + time_ms: u64, +) -> dpp::fee::fee_result::FeeResult { + let mut operations = vec![]; + let block_info = BlockInfo::default_with_time(time_ms); + let batch = drive + .erase_document_for_contract_operations( + id, + contract, + document_type_of(contract), + &block_info, + &mut None, + None, + latest(), + ) + .expect("expected to build the erase operations"); + drive + .apply_batch_low_level_drive_operations(None, None, batch, &mut operations, &latest().drive) + .expect("expected to apply the erase"); + Drive::calculate_fee( + None, + Some(operations), + &block_info.epoch, + drive.config.epochs_per_era, + latest(), + None, + ) + .expect("expected a fee") +} + +/// The admission estimate of an erase, which knows nothing about the document +/// it will act on. +fn estimated_erase_fee( + drive: &Drive, + contract: &DataContract, + id: Identifier, +) -> dpp::fee::fee_result::FeeResult { + let mut operations = vec![]; + let mut layers = Some(HashMap::new()); + let block_info = BlockInfo::default_with_time(9_000); + let batch = drive + .erase_document_for_contract_operations( + id, + contract, + document_type_of(contract), + &block_info, + &mut layers, + None, + latest(), + ) + .expect("expected to build the estimated erase operations"); + drive + .apply_batch_low_level_drive_operations( + layers, + None, + batch, + &mut operations, + &latest().drive, + ) + .expect("expected to price the erase"); + Drive::calculate_fee( + None, + Some(operations), + &block_info.epoch, + drive.config.epochs_per_era, + latest(), + None, + ) + .expect("expected a fee") +} + +#[test] +fn should_hide_a_deleted_document_while_keeping_every_revision_readable() { + let owner = [7u8; 32]; + let (drive, contract, id) = setup_history(3, owner); + assert_eq!(visible_document_count(&drive, &contract), 1); + + delete(&drive, &contract, id, Identifier::new(owner), 5_000); + + assert_eq!( + visible_document_count(&drive, &contract), + 0, + "a deleted document must be absent from every ordinary read" + ); + let record = record_of(&drive, &contract, id).expect("a delete writes a lifecycle record"); + assert_eq!(record.deleted_at_ms(), 5_000); + assert!(!record.is_erasing()); + + let lifecycle = history_metadata(&drive, &contract, id); + assert_eq!(lifecycle.state, DocumentHistoryState::Deleted); + assert_eq!( + lifecycle.remaining_revisions, 3, + "a delete removes no revision" + ); + assert_eq!(lifecycle.times.deleted_at_ms, 5_000); + assert_eq!(lifecycle.times.erasing_started_at_ms, 0); + + assert!(matches!( + lifecycle_of(&drive, &contract, id), + DocumentLifecycleState::Deleted(_) + )); +} + +#[test] +fn should_erase_a_history_shorter_than_a_chunk_in_one_transition() { + let owner = [8u8; 32]; + let (drive, contract, id) = setup_history(1, owner); + delete(&drive, &contract, id, Identifier::new(owner), 5_000); + + erase(&drive, &contract, id, 6_000); + + assert!( + record_of(&drive, &contract, id).is_none(), + "the terminal chunk removes the record" + ); + assert!(matches!( + lifecycle_of(&drive, &contract, id), + DocumentLifecycleState::Absent + )); + let lifecycle = history_metadata(&drive, &contract, id); + assert_eq!(lifecycle.state, DocumentHistoryState::Absent); + assert_eq!(lifecycle.remaining_revisions, 0); +} + +#[test] +fn should_erase_a_history_of_exactly_one_chunk_in_one_transition() { + let chunk = chunk_size(); + let owner = [9u8; 32]; + let (drive, contract, id) = setup_history(chunk, owner); + delete(&drive, &contract, id, Identifier::new(owner), 5_000); + + erase(&drive, &contract, id, 6_000); + + assert!(record_of(&drive, &contract, id).is_none()); + assert_eq!( + history_metadata(&drive, &contract, id).state, + DocumentHistoryState::Absent + ); +} + +#[test] +fn should_commit_a_longer_history_to_erasure_and_finish_it_in_the_next_chunk() { + let chunk = chunk_size(); + let owner = [10u8; 32]; + let (drive, contract, id) = setup_history(chunk + 1, owner); + delete(&drive, &contract, id, Identifier::new(owner), 5_000); + + erase(&drive, &contract, id, 6_000); + + let record = record_of(&drive, &contract, id).expect("a partial chunk keeps the record"); + assert!(record.is_erasing(), "the first chunk commits the erasure"); + assert_eq!(record.deleted_at_ms(), 5_000, "the deletion time survives"); + assert_eq!(record.erasing_started_at_ms(), 6_000); + assert_eq!( + record.erasing_from_revision(), + chunk + 1, + "the erasure started from the newest revision" + ); + assert_eq!(record.erasing_from_time_ms(), 1_000 + chunk + 1); + + let lifecycle = history_metadata(&drive, &contract, id); + assert_eq!(lifecycle.state, DocumentHistoryState::Erasing); + assert_eq!( + lifecycle.remaining_revisions, 1, + "the oldest revision is what a partial erasure leaves behind" + ); + assert_eq!(lifecycle.times.erasing_from_revision, chunk + 1); + assert!(matches!( + lifecycle_of(&drive, &contract, id), + DocumentLifecycleState::Erasing + )); + + erase(&drive, &contract, id, 7_000); + + assert!(record_of(&drive, &contract, id).is_none()); + assert_eq!( + history_metadata(&drive, &contract, id).state, + DocumentHistoryState::Absent + ); +} + +#[test] +fn should_leave_the_record_untouched_across_a_continuation() { + let chunk = chunk_size(); + let owner = [11u8; 32]; + let (drive, contract, id) = setup_history(chunk * 2 + 1, owner); + delete(&drive, &contract, id, Identifier::new(owner), 5_000); + + erase(&drive, &contract, id, 6_000); + let after_start = record_of(&drive, &contract, id).expect("the erasure is committed"); + + erase(&drive, &contract, id, 7_000); + let after_continuation = + record_of(&drive, &contract, id).expect("the erasure is not finished yet"); + + assert_eq!( + after_start, after_continuation, + "a continuation carries no authorization of its own and writes nothing to the record" + ); + assert_eq!( + history_metadata(&drive, &contract, id).remaining_revisions, + 1 + ); + + erase(&drive, &contract, id, 8_000); + assert_eq!( + history_metadata(&drive, &contract, id).state, + DocumentHistoryState::Absent + ); +} + +#[test] +fn should_refuse_to_create_a_document_whose_revisions_are_still_retained() { + let owner = [12u8; 32]; + let (drive, contract, id) = setup_history(2, owner); + delete(&drive, &contract, id, Identifier::new(owner), 5_000); + + let version = latest(); + let document_type = document_type_of(&contract); + let mut document = + json_document_to_document(PERSON, Some(owner.into()), document_type, version).unwrap(); + document.set_revision(Some(1)); + + let error = drive + .add_document_for_contract( + DocumentAndContractInfo { + owned_document_info: OwnedDocumentInfo { + document_info: DocumentInfo::DocumentRefInfo((&document, None)), + owner_id: None, + }, + contract: &contract, + document_type, + }, + // The contested award path reaches the writer this way: nothing + // above it has checked whether the id is free. + false, + BlockInfo::default_with_time(6_000), + true, + None, + version, + None, + ) + .expect_err("a reserved id must not be creatable"); + assert!(matches!( + error, + Error::Drive(DriveError::CorruptedDocumentAlreadyExists(_)) + )); + + // Once the revisions are gone the id is free again. + erase(&drive, &contract, id, 7_000); + drive + .add_document_for_contract( + DocumentAndContractInfo { + owned_document_info: OwnedDocumentInfo { + document_info: DocumentInfo::DocumentRefInfo((&document, None)), + owner_id: None, + }, + contract: &contract, + document_type, + }, + false, + BlockInfo::default_with_time(8_000), + true, + None, + version, + None, + ) + .expect("an erased id is reusable"); + assert_eq!(visible_document_count(&drive, &contract), 1); +} + +/// `override_document` tells the writer the caller has already checked the +/// primary-key entry, so it may skip that check on the current pointer. A +/// deleted document has no current pointer but still retains its revisions, +/// and nothing the caller checked says so; the retained-history refusal must +/// therefore hold whatever the caller claims about the pointer. +#[test] +fn should_refuse_to_create_over_retained_revisions_even_when_the_caller_overrides() { + let owner = [14u8; 32]; + let (drive, contract, id) = setup_history(2, owner); + delete(&drive, &contract, id, Identifier::new(owner), 5_000); + + let version = latest(); + let document_type = document_type_of(&contract); + let mut document = + json_document_to_document(PERSON, Some(owner.into()), document_type, version).unwrap(); + document.set_revision(Some(1)); + + let error = drive + .add_document_for_contract( + DocumentAndContractInfo { + owned_document_info: OwnedDocumentInfo { + document_info: DocumentInfo::DocumentRefInfo((&document, None)), + owner_id: None, + }, + contract: &contract, + document_type, + }, + true, + BlockInfo::default_with_time(6_000), + true, + None, + version, + None, + ) + .expect_err("an override must not merge a new document into a deleted one's history"); + assert!(matches!( + error, + Error::Drive(DriveError::CorruptedDocumentAlreadyExists(_)) + )); + assert_eq!( + history_metadata(&drive, &contract, id).remaining_revisions, + 2, + "the refused write must leave the retained history untouched" + ); + assert!(matches!( + lifecycle_of(&drive, &contract, id), + DocumentLifecycleState::Deleted(_) + )); +} + +#[test] +fn should_refuse_to_drop_a_history_subtree_while_a_revision_survives() { + let owner = [13u8; 32]; + let (drive, contract, id) = setup_history(2, owner); + delete(&drive, &contract, id, Identifier::new(owner), 5_000); + + // The removal of the subtree on its own, without the revision deletes an + // erase puts in the same batch. GroveDB has nothing to conclude emptiness + // from and must refuse rather than orphan the revisions' storage. + let mut history_root = document_history_path(contract.id_ref().as_bytes(), "person", &[]); + history_root.pop(); + let mut operations = vec![]; + let batch_result = drive.batch_delete( + history_root.as_slice().into(), + id.as_slice(), + StatefulBatchDelete { + is_known_to_be_subtree_with_sum: Some(MaybeTree::Tree(TreeType::ProvableCountTree)), + }, + None, + &mut operations, + &latest().drive, + ); + assert!( + batch_result.is_err(), + "a populated history subtree must not be removable" + ); +} + +#[test] +fn should_refund_each_removed_revision_to_the_identity_that_wrote_it() { + let owner = [14u8; 32]; + let (drive, contract, id) = setup_history(3, owner); + delete(&drive, &contract, id, Identifier::new(owner), 5_000); + + let fee = erase(&drive, &contract, id, 6_000); + + let refunded: u64 = fee + .fee_refunds + .0 + .get(&owner) + .map(|per_epoch| per_epoch.values().sum()) + .unwrap_or_default(); + assert!( + refunded > 0, + "the revisions' writer must be credited for the bytes the erase removed" + ); +} + +#[test] +fn should_price_an_erase_for_a_full_chunk_whatever_the_document_retains() { + let chunk = chunk_size(); + for revisions in [1, chunk, chunk + 1] { + let owner = [15u8; 32]; + let (drive, contract, id) = setup_history(revisions, owner); + delete(&drive, &contract, id, Identifier::new(owner), 5_000); + + let estimated = estimated_erase_fee(&drive, &contract, id); + let actual = erase(&drive, &contract, id, 6_000); + + assert!( + estimated.processing_fee >= actual.processing_fee, + "the admission estimate under-charged a {revisions}-revision erase: {} < {}", + estimated.processing_fee, + actual.processing_fee + ); + } +} + +/// The history a delete leaves behind is still fully readable, page by page and +/// with a proof, including the page that runs past its end. +#[test] +fn should_page_and_prove_the_history_of_a_deleted_document() { + let owner = [16u8; 32]; + let (drive, contract, id) = setup_history(12, owner); + delete(&drive, &contract, id, Identifier::new(owner), 5_000); + + let mut selector = DocumentHistoryFilter::StartAtTime(0); + let mut seen = vec![]; + loop { + let query = DocumentHistoryDriveQuery { + contract_id: contract.id().to_buffer(), + document_type_name: "person".into(), + document_id: id.to_buffer(), + filter: selector, + limit: Some(10), + }; + let page = drive + .fetch_document_history(&query, document_type_of(&contract), None, latest()) + .expect("expected to fetch a page"); + let proof = drive + .prove_document_history(&query, document_type_of(&contract), None, latest()) + .expect("expected to prove a page"); + let (_, verified) = + Drive::verify_document_history(&query, &proof, document_type_of(&contract), latest()) + .expect("expected the page proof to verify"); + let lifecycle = verified + .lifecycle + .expect("the latest protocol authenticates lifecycle metadata"); + assert_eq!(lifecycle.state, DocumentHistoryState::Deleted); + assert_eq!(lifecycle.remaining_revisions, 12); + assert_eq!(lifecycle.times.deleted_at_ms, 5_000); + seen.extend(page.entries.iter().map(|entry| entry.revision)); + let Some(last) = page.entries.last() else { + break; + }; + selector = DocumentHistoryFilter::StartAfter { + time_ms: last.time_ms, + revision: last.revision, + }; + } + assert_eq!(seen, (1..=12).collect::>()); +} + +/// Every document type parsed by an earlier grammar has no lifecycle tree, so +/// the read has to treat a missing tree the same as a missing key. +#[test] +fn should_report_a_document_with_no_lifecycle_tree_as_absent() { + let owner = [17u8; 32]; + let (drive, contract, _) = setup_history(1, owner); + let unknown = Identifier::new([99u8; 32]); + assert!(matches!( + lifecycle_of(&drive, &contract, unknown), + DocumentLifecycleState::Absent + )); +} + +#[test] +fn should_reject_an_erase_of_a_document_that_retains_nothing() { + let owner = [18u8; 32]; + let (drive, contract, id) = setup_history(1, owner); + delete(&drive, &contract, id, Identifier::new(owner), 5_000); + erase(&drive, &contract, id, 6_000); + + let error = drive + .erase_document_for_contract_operations( + id, + &contract, + document_type_of(&contract), + &BlockInfo::default_with_time(7_000), + &mut None, + None as TransactionArg, + latest(), + ) + .expect_err("there is nothing left to erase"); + // The terminal chunk removed the record along with the revisions, so the + // id reads as never deleted and the refusal is the not-deleted one. + assert!(matches!(error, Error::Drive(DriveError::InvalidInput(_)))); +} + +/// The two record shapes must occupy the same number of bytes: an erase start +/// overwrites the record in place, and a different size would re-price its +/// storage and drop the deleter's flags. +#[test] +fn should_keep_the_deleter_as_the_records_beneficiary_across_an_erase_start() { + let chunk = chunk_size(); + let owner = [19u8; 32]; + let deleter = Identifier::new([20u8; 32]); + let (drive, contract, id) = setup_history(chunk + 1, owner); + delete(&drive, &contract, id, deleter, 5_000); + + let path = document_lifecycle_path(contract.id_ref().as_bytes(), "person"); + let flags_before = drive + .grove + .get_raw( + path.as_slice().into(), + id.as_slice(), + None, + &latest().drive.grove_version, + ) + .unwrap() + .expect("the record exists"); + + erase(&drive, &contract, id, 6_000); + + let flags_after = drive + .grove + .get_raw( + path.as_slice().into(), + id.as_slice(), + None, + &latest().drive.grove_version, + ) + .unwrap() + .expect("the record survives a partial erasure"); + let (grovedb::Element::Item(before, before_flags), grovedb::Element::Item(after, after_flags)) = + (flags_before, flags_after) + else { + panic!("a lifecycle record is an item"); + }; + assert_ne!(before, after, "the erase start rewrote the record"); + assert_eq!( + before_flags, after_flags, + "the deleter must stay the beneficiary of the record's bytes" + ); +} + +/// A revision written by one identity and a revision written by another are +/// refunded separately, so an erasure credits whoever paid for each byte. +#[test] +fn should_refund_distinct_writers_separately() { + let first = [21u8; 32]; + let second = [22u8; 32]; + let version = latest(); + let (drive, contract, id) = setup_history(1, first); + let document_type = document_type_of(&contract); + let mut document = + json_document_to_document(PERSON, Some(first.into()), document_type, version).unwrap(); + document.set_revision(Some(2)); + drive + .add_document_for_contract( + DocumentAndContractInfo { + owned_document_info: OwnedDocumentInfo { + document_info: DocumentInfo::DocumentRefInfo(( + &document, + Some(Cow::Owned(StorageFlags::new_single_epoch(0, Some(second)))), + )), + owner_id: None, + }, + contract: &contract, + document_type, + }, + true, + BlockInfo::default_with_time(2_000), + true, + None, + version, + None, + ) + .expect("expected a second writer's revision"); + + delete(&drive, &contract, id, Identifier::new(first), 5_000); + let fee = erase(&drive, &contract, id, 6_000); + + assert!( + fee.fee_refunds.0.contains_key(&first) && fee.fee_refunds.0.contains_key(&second), + "each writer must be credited for the revision they paid for; got {:?}", + fee.fee_refunds.0.keys().collect::>() + ); +} + +#[test] +fn should_read_the_newest_retained_revision_of_a_deleted_document() { + let owner = [23u8; 32]; + let (drive, contract, id) = setup_history(4, owner); + delete(&drive, &contract, id, Identifier::new(owner), 5_000); + + let DocumentLifecycleState::Deleted(newest) = lifecycle_of(&drive, &contract, id) else { + panic!("expected a deleted document"); + }; + assert_eq!( + newest.revision(), + Some(4), + "an erase start is authorized against the newest revision that survives" + ); + assert_eq!(newest.owner_id().to_buffer(), owner); +} + +/// A `Document` is only ever written under a keep-history type through the +/// history tree, so a deleted document's revisions must not be reachable by id. +#[test] +fn should_not_resolve_a_deleted_document_by_id() { + let owner = [24u8; 32]; + let (drive, contract, id) = setup_history(2, owner); + delete(&drive, &contract, id, Identifier::new(owner), 5_000); + + let query = SingleDocumentDriveQuery { + contract_id: contract.id().to_buffer(), + document_type_name: "person".into(), + document_type_keeps_history: true, + document_id: id.to_buffer(), + block_time_ms: None, + contested_status: SingleDocumentDriveQueryContestedStatus::NotContested, + }; + let path_query = query + .construct_path_query(latest()) + .expect("expected a by-id path query"); + let (results, _) = drive + .grove_get_raw_path_query( + &path_query, + None, + grovedb::query_result_type::QueryResultType::QueryKeyElementPairResultType, + &mut vec![], + &latest().drive, + ) + .expect("expected the by-id read to run"); + assert!( + results.to_key_elements().is_empty(), + "a deleted document is not fetchable by id" + ); +} + +/// The counted and summed entry of a keep-history type is its current pointer, +/// so removing it decrements both aggregates by construction and a deleted +/// document stops contributing without any bookkeeping of its own. +#[test] +fn should_decrement_the_count_and_the_sum_when_a_keep_history_document_is_deleted() { + use crate::drive::document::paths::contract_document_type_path_vec; + use dpp::data_contract::document_type::accessors::DocumentTypeV2Getters; + use dpp::data_contract::DataContractFactory; + use dpp::document::document_factory::DocumentFactory; + use dpp::platform_value::platform_value; + use grovedb::Element; + + let version = latest(); + let schema = platform_value!({ + "type": "object", + "documentsKeepHistory": true, + "canBeDeleted": true, + "documentsCountable": true, + "documentsSummable": "amount", + "properties": { + "amount": {"type": "integer", "minimum": 0, "maximum": 4294967295i64, "position": 0}, + }, + "required": ["amount"], + "additionalProperties": false, + "indices": [{"name": "amount", "properties": [{"amount": "asc"}]}], + }); + let contract = DataContractFactory::new(version.protocol_version) + .expect("expected a contract factory") + .create_with_value_config( + [7; 32].into(), + 0, + platform_value!({ "tip": schema }), + None, + None, + ) + .expect("a countable summable keep-history type must parse") + .data_contract_owned(); + + let drive = setup_drive_with_initial_state_structure(None); + drive + .apply_contract(&contract, BlockInfo::default(), true, None, None, version) + .expect("expected to apply the contract"); + let document_type = contract + .document_type_for_name("tip") + .expect("expected the tip type"); + assert!(document_type.documents_countable()); + + let mut document = DocumentFactory::new(version.protocol_version) + .expect("expected a document factory") + .create_document( + &contract, + [7; 32].into(), + "tip".into(), + platform_value!({"amount": 5}), + ) + .expect("expected a tip"); + document.set_id([9; 32].into()); + for revision in 1..=3u64 { + document.set_revision(Some(revision)); + drive + .add_document_for_contract( + DocumentAndContractInfo { + owned_document_info: OwnedDocumentInfo { + document_info: DocumentInfo::DocumentRefInfo((&document, None)), + owner_id: None, + }, + contract: &contract, + document_type, + }, + revision > 1, + BlockInfo::default_with_time(1_000 + revision), + true, + None, + version, + None, + ) + .expect("expected to write a revision"); + } + + let type_path = contract_document_type_path_vec(contract.id_ref().as_bytes(), "tip"); + let aggregates = || { + let element = drive + .grove + .get_raw( + type_path.as_slice().into(), + &[0], + None, + &latest().drive.grove_version, + ) + .value + .expect("expected the primary-key tree"); + let Element::CountSumTree(_, count, sum, _) = element else { + panic!("a countable summable type stores its documents in a count-sum tree"); + }; + (count, sum) + }; + assert_eq!( + aggregates(), + (1, 5), + "one live document contributing its amount, whatever its history holds" + ); + + let batch = drive + .delete_document_for_contract_operations( + document.id(), + &contract, + document_type, + &BlockInfo::default_with_time(5_000), + Some(Identifier::new([7; 32])), + None, + &mut None, + None, + version, + ) + .expect("expected to delete"); + drive + .apply_batch_low_level_drive_operations(None, None, batch, &mut vec![], &version.drive) + .expect("expected to apply the delete"); + + assert_eq!( + aggregates(), + (0, 0), + "a deleted document contributes to neither aggregate, though its revisions remain" + ); + + // Its index entries are gone too, so the value it held is free for another + // document to take. + let mut index_path = type_path.clone(); + index_path.push(b"amount".to_vec()); + let mut query = grovedb::Query::new(); + query.insert_all(); + let (results, _) = drive + .grove_get_raw_path_query( + &grovedb::PathQuery::new(index_path, grovedb::SizedQuery::new(query, Some(10), None)), + None, + grovedb::query_result_type::QueryResultType::QueryKeyElementPairResultType, + &mut vec![], + &version.drive, + ) + .expect("expected to read the index"); + assert!( + results.to_key_elements().is_empty(), + "a delete removes every index reference that led to the document" + ); +} + +/// The per-type container is shared by every document of the type and outlives +/// any one of them, so it belongs to nobody: an erase removes records and +/// per-document history trees, never this. The record inside it is the byte an +/// erase does refund, and that one names the deleter. +#[test] +fn should_leave_the_lifecycle_container_unflagged_while_the_record_names_the_deleter() { + use crate::drive::document::paths::contract_document_type_path_vec; + + let owner = [25u8; 32]; + let deleter = Identifier::new([26u8; 32]); + let (drive, contract, id) = setup_history(2, owner); + delete(&drive, &contract, id, deleter, 5_000); + + let mut type_path = contract_document_type_path_vec(contract.id_ref().as_bytes(), "person"); + let container = drive + .grove + .get_raw( + type_path.as_slice().into(), + &[crate::drive::document::paths::DOCUMENT_LIFECYCLE_TREE_KEY], + None, + &latest().drive.grove_version, + ) + .value + .expect("the first delete creates the container"); + assert_eq!( + container.get_flags(), + &None, + "the shared container must belong to nobody" + ); + + type_path.push(vec![ + crate::drive::document::paths::DOCUMENT_LIFECYCLE_TREE_KEY, + ]); + let record = drive + .grove + .get_raw( + type_path.as_slice().into(), + id.as_slice(), + None, + &latest().drive.grove_version, + ) + .value + .expect("the record exists"); + let flags = StorageFlags::map_cow_some_element_flags_ref(record.get_flags()) + .expect("the record's flags must decode") + .expect("the record names its deleter"); + assert_eq!( + flags.owner_id(), + Some(&deleter.to_buffer()), + "the record is the byte an erase refunds, and it belongs to the deleter" + ); +} + +/// Only the first delete of a type pays for the container. A second one finds +/// it already there and pays for its record alone, so the storage difference +/// between the two is exactly the container. +#[test] +fn should_charge_the_container_to_the_first_delete_of_a_type_only() { + let owner = [27u8; 32]; + let version = latest(); + let (drive, contract, first_id) = setup_history(1, owner); + let document_type = document_type_of(&contract); + + // A second document of the same type, so the second delete finds the + // container already in place. + let mut second = + json_document_to_document(PERSON, Some(owner.into()), document_type, version).unwrap(); + second.set_id(Identifier::new([28u8; 32])); + second.set_revision(Some(1)); + drive + .add_document_for_contract( + DocumentAndContractInfo { + owned_document_info: OwnedDocumentInfo { + document_info: DocumentInfo::DocumentRefInfo(( + &second, + Some(Cow::Owned(StorageFlags::new_single_epoch(0, Some(owner)))), + )), + owner_id: None, + }, + contract: &contract, + document_type, + }, + false, + BlockInfo::default_with_time(2_000), + true, + None, + version, + None, + ) + .expect("expected a second document"); + + let deleter = Identifier::new([29u8; 32]); + let first = delete(&drive, &contract, first_id, deleter, 5_000); + let later = delete(&drive, &contract, second.id(), deleter, 6_000); + + assert!( + first.storage_fee > later.storage_fee, + "the first delete pays for the container as well as its record: {} vs {}", + first.storage_fee, + later.storage_fee + ); +} + +/// The declared feature slot has to actually route: a table that selects a +/// version this code does not implement must fail loudly rather than silently +/// keep using the old estimate. +#[test] +fn should_reject_an_unsupported_erase_estimation_version() { + use std::collections::HashMap; + + let owner = [30u8; 32]; + let (_drive, contract, id) = setup_history(1, owner); + let mut version = latest().clone(); + assert_eq!( + version + .drive + .methods + .document + .delete + .add_estimation_costs_for_erase_document, + Some(0), + "protocol 15 selects the only implementation there is" + ); + version + .drive + .methods + .document + .delete + .add_estimation_costs_for_erase_document = Some(1); + + let error = Drive::add_estimation_costs_for_erase_document( + id, + &contract, + document_type_of(&contract), + &mut HashMap::new(), + &version, + ) + .expect_err("an unimplemented version must be refused"); + assert!(matches!( + error, + Error::Drive(DriveError::UnknownVersionMismatch { .. }) + )); +} + +/// The recipient work is priced from the ordinary balance-update path and +/// scales with the number of beneficiaries one erase can credit, without +/// touching any identity's balance. +#[test] +fn should_price_the_refund_recipients_an_erase_can_credit() { + use dpp::block::epoch::Epoch; + + let drive = setup_drive_with_initial_state_structure(None); + let epoch = Epoch::new(0).unwrap(); + let version = latest(); + + let priced = drive + .erase_refund_recipient_cost(&epoch, version) + .expect("expected a price for the recipient work"); + assert!( + priced.processing_fee > 0, + "crediting a beneficiary is work somebody has to pay for" + ); + assert!( + priced.fee_refunds.0.is_empty() && priced.storage_fee == 0, + "pricing must not invent a refund or a storage charge of its own" + ); + + let mut no_revisions = version.clone(); + no_revisions + .system_limits + .max_document_revisions_erased_per_transition = Some(0); + let structural = drive + .erase_refund_recipient_cost(&epoch, &no_revisions) + .expect("expected the structural-recipient price"); + let mut one_revision = no_revisions.clone(); + one_revision + .system_limits + .max_document_revisions_erased_per_transition = Some(1); + let with_one_revision = drive + .erase_refund_recipient_cost(&epoch, &one_revision) + .expect("expected the one-revision price"); + let one_recipient = with_one_revision + .processing_fee + .checked_sub(structural.processing_fee) + .expect("one additional recipient increases the price"); + assert_eq!( + structural.processing_fee, + one_recipient * 2, + "terminal erase refunds only the lifecycle record and history subtree" + ); + assert_eq!( + priced.processing_fee, + one_recipient * (chunk_size() + 2), + "the full estimate prices every revision plus two structural recipients" + ); + + // Every balance in the drive is untouched: this is a measurement, not a + // mutation. + assert!( + drive + .fetch_identity_balance(default_owner_id(), None, version) + .expect("expected the balance read to run") + .is_none(), + "pricing must not create the identity it measures against" + ); +} + +/// The synthetic identity the price is measured against. +fn default_owner_id() -> [u8; 32] { + [0u8; 32] +} + +/// Revisions written in different epochs are refunded under the epoch each was +/// written in, and the payout tail decides the amount: a byte written recently +/// has more of its one-time charge still unspent than one written long ago, so +/// it refunds more. +#[test] +fn should_refund_each_writer_under_the_epoch_they_wrote_in() { + use dpp::block::epoch::Epoch; + + let early_writer = [40u8; 32]; + let late_writer = [41u8; 32]; + let version = latest(); + let drive = setup_drive_with_initial_state_structure(None); + let contract = json_document_to_contract(FAMILY_HISTORY_CONTRACT, false, version).unwrap(); + drive + .apply_contract(&contract, BlockInfo::default(), true, None, None, version) + .expect("expected to apply the contract"); + let document_type = document_type_of(&contract); + let mut document = + json_document_to_document(PERSON, Some(early_writer.into()), document_type, version) + .unwrap(); + + let write = |revision: u64, writer: [u8; 32], epoch_index: u16, first: bool| { + let mut document = document.clone(); + document.set_revision(Some(revision)); + drive + .add_document_for_contract( + DocumentAndContractInfo { + owned_document_info: OwnedDocumentInfo { + document_info: DocumentInfo::DocumentRefInfo(( + &document, + Some(Cow::Owned(StorageFlags::new_single_epoch( + epoch_index, + Some(writer), + ))), + )), + owner_id: None, + }, + contract: &contract, + document_type, + }, + !first, + BlockInfo { + epoch: Epoch::new(epoch_index).unwrap(), + ..BlockInfo::default_with_time(1_000 + revision) + }, + true, + None, + version, + None, + ) + .expect("expected to write a revision"); + }; + // The first revision also creates the document's history subtree and its + // index references, so it is not the same amount of storage as an appended + // one. A third identity pays for it, leaving the two compared below as one + // appended revision each and differing only in the epoch they landed in. + write(1, [39u8; 32], 0, true); + write(2, early_writer, 0, false); + write(3, late_writer, 6, false); + document.set_revision(Some(3)); + let id = document.id(); + + let block_info = BlockInfo { + epoch: Epoch::new(6).unwrap(), + ..BlockInfo::default_with_time(5_000) + }; + let batch = drive + .delete_document_for_contract_operations( + id, + &contract, + document_type, + &block_info, + Some(Identifier::new(early_writer)), + None, + &mut None, + None, + version, + ) + .expect("expected to delete"); + drive + .apply_batch_low_level_drive_operations(None, None, batch, &mut vec![], &version.drive) + .expect("expected to apply the delete"); + + let fee = erase_at(&drive, &contract, id, &block_info); + + let early = fee + .fee_refunds + .0 + .get(&early_writer) + .expect("the first writer must be credited"); + let late = fee + .fee_refunds + .0 + .get(&late_writer) + .expect("the second writer must be credited"); + // Every byte is refunded under the epoch it was written in, so the first + // writer appears twice: once for the revision it wrote in epoch 0, and once + // for the lifecycle record it wrote as the deleter in epoch 6. + assert_eq!( + early.keys().copied().collect::>(), + vec![0, 6], + "the first writer's revision and the record it wrote as deleter" + ); + assert_eq!( + late.keys().copied().collect::>(), + vec![6], + "the second writer wrote only its revision, in epoch 6" + ); + + let early_revision = early[&0]; + let late_revision = late[&6]; + assert!( + late_revision > early_revision, + "the same revision written six epochs later has more of its one-time \ + charge still unspent, so it refunds more: {late_revision} is not above {early_revision}" + ); +} + +/// Erases at a caller-chosen block, so a test can put the write and the erase +/// in different epochs. +fn erase_at( + drive: &Drive, + contract: &DataContract, + id: Identifier, + block_info: &BlockInfo, +) -> dpp::fee::fee_result::FeeResult { + let mut operations = vec![]; + let batch = drive + .erase_document_for_contract_operations( + id, + contract, + document_type_of(contract), + block_info, + &mut None, + None, + latest(), + ) + .expect("expected to build the erase operations"); + drive + .apply_batch_low_level_drive_operations(None, None, batch, &mut operations, &latest().drive) + .expect("expected to apply the erase"); + Drive::calculate_fee( + None, + Some(operations), + &block_info.epoch, + drive.config.epochs_per_era, + latest(), + None, + ) + .expect("expected a fee") +} + +/// A refund is a credit to a real identity, not just a map entry: the amount +/// the fee result names is the amount the beneficiary's balance gains. +#[test] +fn should_credit_a_refund_to_the_beneficiarys_balance() { + let owner = [42u8; 32]; + let version = latest(); + let (drive, contract, id) = setup_history(3, owner); + drive + .add_new_identity( + dpp::identity::Identity::create_basic_identity(Identifier::new(owner), version) + .expect("expected a basic identity"), + false, + &BlockInfo::default(), + true, + None, + version, + ) + .expect("expected to create the beneficiary"); + let before = drive + .fetch_identity_balance(owner, None, version) + .expect("expected to read the balance") + .expect("the beneficiary exists"); + + delete(&drive, &contract, id, Identifier::new(owner), 5_000); + let fee = erase(&drive, &contract, id, 6_000); + let refunded: u64 = fee + .fee_refunds + .0 + .get(&owner) + .map(|per_epoch| per_epoch.values().sum()) + .expect("the writer must be credited"); + + drive + .add_to_identity_balance(owner, refunded, &BlockInfo::default(), true, None, version) + .expect("expected to credit the refund"); + let after = drive + .fetch_identity_balance(owner, None, version) + .expect("expected to read the balance") + .expect("the beneficiary exists"); + assert_eq!( + after - before, + refunded, + "the credited amount is exactly what the fee result named" + ); +} + +/// The estimate an erase is admitted against is sized for a full chunk, so a +/// one-revision erase needs the same balance as a hundred-revision one. That is +/// the point of the bound: it is what an erase can cost, not what this one will. +#[test] +fn should_admit_a_one_revision_erase_only_against_the_full_chunk_estimate() { + let owner = [43u8; 32]; + let (drive, contract, id) = setup_history(1, owner); + delete(&drive, &contract, id, Identifier::new(owner), 5_000); + + let estimated = estimated_erase_fee(&drive, &contract, id); + let actual = erase(&drive, &contract, id, 6_000); + assert!( + estimated.processing_fee > actual.processing_fee * 4, + "the estimate must be sized for the bound, not for the one revision this document holds: {} against {}", + estimated.processing_fee, + actual.processing_fee + ); +} + +/// GroveDB's batch consistency checking is off by default and on in some +/// deployments. A lifecycle batch has to produce the same state and the same +/// fees either way, or the two disagree about consensus. +#[test] +fn should_erase_identically_with_batch_consistency_checking_on() { + let chunk = chunk_size(); + let mut roots = vec![]; + let mut fees = vec![]; + for verify in [false, true] { + let owner = [44u8; 32]; + let version = latest(); + let directory = tempfile::TempDir::new().unwrap(); + let (drive, _) = Drive::open( + directory.path(), + Some(crate::config::DriveConfig { + batching_consistency_verification: verify, + ..Default::default() + }), + ) + .unwrap(); + drive.create_initial_state_structure(None, version).unwrap(); + let contract = json_document_to_contract(FAMILY_HISTORY_CONTRACT, false, version).unwrap(); + drive + .apply_contract(&contract, BlockInfo::default(), true, None, None, version) + .expect("expected to apply the contract"); + let document_type = document_type_of(&contract); + let mut document = + json_document_to_document(PERSON, Some(owner.into()), document_type, version).unwrap(); + let flags = Some(Cow::Owned(StorageFlags::new_single_epoch(0, Some(owner)))); + for revision in 1..=chunk { + document.set_revision(Some(revision)); + drive + .add_document_for_contract( + DocumentAndContractInfo { + owned_document_info: OwnedDocumentInfo { + document_info: DocumentInfo::DocumentRefInfo(( + &document, + flags.clone(), + )), + owner_id: None, + }, + contract: &contract, + document_type, + }, + revision > 1, + BlockInfo::default_with_time(1_000 + revision), + true, + None, + version, + None, + ) + .expect("expected to write a revision"); + } + let id = document.id(); + delete(&drive, &contract, id, Identifier::new(owner), 5_000); + fees.push(erase(&drive, &contract, id, 6_000).processing_fee); + roots.push( + drive + .grove + .root_hash(None, &version.drive.grove_version) + .value + .unwrap(), + ); + assert!(matches!( + lifecycle_of(&drive, &contract, id), + DocumentLifecycleState::Absent + )); + } + assert_eq!( + roots[0], roots[1], + "consistency checking must not change the state a lifecycle batch commits" + ); + assert_eq!(fees[0], fees[1], "nor what it costs"); +} + +/// Two deletes of different documents of one type, combined into one batch +/// before either is applied and before the type has a lifecycle container, each +/// emit the insert of the same shared key. Every document operation is +/// converted on its own and cannot see its siblings, so neither can tell that +/// the other is already creating it. +/// +/// GroveDB refuses the batch rather than committing one of the two, so nothing +/// is written and the caller is told. That fail-closed behaviour is what this +/// pins. Consensus cannot reach it: a batch state transition carries exactly +/// one document transition at every protocol version, so two document +/// operations never share a batch by that route, and no other caller combines +/// keep-history deletes. +#[test] +fn should_refuse_rather_than_half_apply_two_deletes_sharing_a_new_container() { + assert_eq!( + latest().system_limits.max_transitions_in_documents_batch, + 1, + "the cap is what keeps consensus away from this shape" + ); + use crate::util::batch::{DocumentOperationType, DriveOperation}; + use crate::util::object_size_info::{DataContractInfo, DocumentTypeInfo}; + + let owner = [45u8; 32]; + let version = latest(); + let (drive, contract, first_id) = setup_history(1, owner); + let document_type = document_type_of(&contract); + let mut second = + json_document_to_document(PERSON, Some(owner.into()), document_type, version).unwrap(); + second.set_id(Identifier::new([46u8; 32])); + second.set_revision(Some(1)); + drive + .add_document_for_contract( + DocumentAndContractInfo { + owned_document_info: OwnedDocumentInfo { + document_info: DocumentInfo::DocumentRefInfo(( + &second, + Some(Cow::Owned(StorageFlags::new_single_epoch(0, Some(owner)))), + )), + owner_id: None, + }, + contract: &contract, + document_type, + }, + false, + BlockInfo::default_with_time(2_000), + true, + None, + version, + None, + ) + .expect("expected a second document"); + + let delete_operation = |id: Identifier| { + DriveOperation::DocumentOperation(DocumentOperationType::DeleteDocument { + document_id: id, + deleter_id: Some(Identifier::new(owner)), + contract_info: DataContractInfo::BorrowedDataContract(&contract), + document_type_info: DocumentTypeInfo::DocumentTypeName("person".to_string()), + }) + }; + drive + .apply_drive_operations( + vec![delete_operation(first_id), delete_operation(second.id())], + true, + &BlockInfo::default_with_time(5_000), + None, + version, + None, + ) + .expect_err("the duplicated container insert must be refused, not half applied"); + + for id in [first_id, second.id()] { + assert!( + matches!( + lifecycle_of(&drive, &contract, id), + DocumentLifecycleState::Active(_) + ), + "a refused batch must leave both documents exactly as they were" + ); + } + + // One at a time is the supported shape, and the second finds the container + // the first created. + for id in [first_id, second.id()] { + drive + .apply_drive_operations( + vec![delete_operation(id)], + true, + &BlockInfo::default_with_time(6_000), + None, + version, + None, + ) + .expect("expected one delete at a time to succeed"); + } + for id in [first_id, second.id()] { + assert!(matches!( + lifecycle_of(&drive, &contract, id), + DocumentLifecycleState::Deleted(_) + )); + } +} + +/// Every state the lifecycle can be in has to survive the proof round trip, and +/// corrupting the lifecycle proof must fail verification rather than return +/// unauthenticated metadata. +#[test] +fn should_prove_the_erasing_state_and_reject_a_tampered_proof() { + let chunk = chunk_size(); + let owner = [47u8; 32]; + let (drive, contract, id) = setup_history(chunk + 2, owner); + delete(&drive, &contract, id, Identifier::new(owner), 5_000); + erase(&drive, &contract, id, 6_000); + + let query = |filter| DocumentHistoryDriveQuery { + contract_id: contract.id().to_buffer(), + document_type_name: "person".into(), + document_id: id.to_buffer(), + // A single-revision read is capped at one entry by the selector's own + // rule; the page selectors take the full page. + limit: Some(match filter { + DocumentHistoryFilter::Revision(_) => 1, + _ => 10, + }), + filter, + }; + + // A populated page, and a page whose lower bound is past the end of what + // survives. Both must authenticate the same erasing metadata. + for selector in [ + DocumentHistoryFilter::StartAtTime(0), + DocumentHistoryFilter::StartAtTime(u64::MAX / 2), + DocumentHistoryFilter::Revision(1), + ] { + let query = query(selector); + let page = drive + .fetch_document_history(&query, document_type_of(&contract), None, latest()) + .expect("expected to fetch the page"); + let proof = drive + .prove_document_history(&query, document_type_of(&contract), None, latest()) + .expect("expected to prove the page"); + let (_, verified) = + Drive::verify_document_history(&query, &proof, document_type_of(&contract), latest()) + .expect("expected the proof to verify"); + assert_eq!(verified, page); + let lifecycle = verified + .lifecycle + .expect("the latest protocol authenticates lifecycle metadata"); + assert_eq!(lifecycle.state, DocumentHistoryState::Erasing); + assert_eq!(lifecycle.remaining_revisions, 2); + assert_eq!(lifecycle.times.deleted_at_ms, 5_000); + assert_eq!(lifecycle.times.erasing_started_at_ms, 6_000); + assert_eq!(lifecycle.times.erasing_from_revision, chunk + 2); + + let mut tampered = + DocumentHistoryProof::from_bytes(&proof).expect("expected the history proof envelope"); + tampered.metadata_proof.push(0xff); + let tampered = tampered.to_bytes().expect("expected to encode the proof"); + Drive::verify_document_history(&query, &tampered, document_type_of(&contract), latest()) + .expect_err("tampering with the lifecycle proof must fail verification"); + } + + // And the terminal state after the erasure finishes. + erase(&drive, &contract, id, 7_000); + let query = query(DocumentHistoryFilter::StartAtTime(0)); + let page = drive + .fetch_document_history(&query, document_type_of(&contract), None, latest()) + .expect("expected to fetch the absent id"); + let proof = drive + .prove_document_history(&query, document_type_of(&contract), None, latest()) + .expect("expected to prove the absent id"); + let (_, verified) = + Drive::verify_document_history(&query, &proof, document_type_of(&contract), latest()) + .expect("expected the absence proof to verify"); + assert_eq!(verified, page); + let lifecycle = verified + .lifecycle + .expect("the latest protocol authenticates lifecycle metadata"); + assert_eq!(lifecycle.state, DocumentHistoryState::Absent); + assert_eq!(lifecycle.remaining_revisions, 0); + assert_eq!(lifecycle.times, Default::default()); + let proof = + DocumentHistoryProof::from_bytes(&proof).expect("expected the history proof envelope"); + assert!( + proof.entries_proof.is_none(), + "there is no history tree left to page over" + ); +} + +/// After a partial erasure the surviving revisions are the oldest ones and +/// still number from one, so a by-revision read lands on the revision it asks +/// for rather than on whatever is now at that offset. +#[test] +fn should_read_revisions_by_position_after_a_partial_erasure() { + let chunk = chunk_size(); + let owner = [48u8; 32]; + let (drive, contract, id) = setup_history(chunk + 3, owner); + delete(&drive, &contract, id, Identifier::new(owner), 5_000); + erase(&drive, &contract, id, 6_000); + + for revision in 1..=3u64 { + let query = DocumentHistoryDriveQuery { + contract_id: contract.id().to_buffer(), + document_type_name: "person".into(), + document_id: id.to_buffer(), + filter: DocumentHistoryFilter::Revision(revision), + limit: Some(1), + }; + let page = drive + .fetch_document_history(&query, document_type_of(&contract), None, latest()) + .expect("expected to read a surviving revision"); + assert_eq!( + page.entries.len(), + 1, + "revision {revision} survives a partial erasure" + ); + assert_eq!(page.entries[0].revision, revision); + } + + // The revisions the chunk removed are gone, not silently answered with a + // neighbour. + let query = DocumentHistoryDriveQuery { + contract_id: contract.id().to_buffer(), + document_type_name: "person".into(), + document_id: id.to_buffer(), + filter: DocumentHistoryFilter::Revision(4), + limit: Some(1), + }; + let page = drive + .fetch_document_history(&query, document_type_of(&contract), None, latest()) + .expect("expected the read to run"); + assert!( + page.entries.is_empty(), + "a removed revision must not be answered with the one that took its place" + ); +} + +/// A history that the legacy layout left gapped keeps refusing by-revision +/// reads after the document is deleted: a revision no longer maps onto a +/// position, and an empty page for a revision that exists would otherwise +/// verify as an authenticated absence. +#[test] +fn should_keep_refusing_by_revision_reads_of_a_gapped_history_after_deletion() { + let old = PlatformVersion::get(14).unwrap(); + let new = PlatformVersion::get(15).unwrap(); + let drive = setup_drive_with_initial_state_structure(None); + let contract = json_document_to_contract( + "tests/supporting_files/contract/dashpay/dashpay-contract-with-profile-history.json", + false, + old, + ) + .unwrap(); + drive + .apply_contract(&contract, BlockInfo::default(), true, None, None, old) + .unwrap(); + let document_type = contract.document_type_for_name("profile").unwrap(); + let owner = [9u8; 32]; + let mut document = json_document_to_document( + "tests/supporting_files/contract/dashpay/profile0.json", + Some(owner.into()), + document_type, + old, + ) + .unwrap(); + // Two writes in one block under the legacy layout overwrite each other's + // revision, so revision 2 is lost and the retained history is [1, 3]. + for (revision, time) in [(1, 1_000), (2, 2_000), (3, 2_000)] { + document.set_revision(Some(revision)); + drive + .add_document_for_contract( + DocumentAndContractInfo { + owned_document_info: OwnedDocumentInfo { + document_info: DocumentInfo::DocumentRefInfo(( + &document, + Some(Cow::Owned(StorageFlags::new_single_epoch(0, Some(owner)))), + )), + owner_id: None, + }, + contract: &contract, + document_type, + }, + true, + BlockInfo::default_with_time(time), + true, + None, + old, + None, + ) + .unwrap(); + } + let transaction = drive.grove.start_transaction(); + drive + .migrate_document_history_storage(&transaction, new) + .expect("expected the migration to run"); + drive + .grove + .commit_transaction(transaction) + .unwrap() + .expect("expected the migration to commit"); + let id = document.id(); + + let query = |filter| DocumentHistoryDriveQuery { + contract_id: contract.id().to_buffer(), + document_type_name: "profile".into(), + document_id: id.to_buffer(), + // A single-revision read is capped at one entry by the selector's + // own rule; the page selectors take the full page. + limit: Some(match filter { + DocumentHistoryFilter::Revision(_) => 1, + _ => 10, + }), + filter, + }; + let by_revision = [ + DocumentHistoryFilter::Revision(3), + DocumentHistoryFilter::StartAtRevision(3), + DocumentHistoryFilter::Revision(2), + ]; + let is_gap_refusal = |error: Error| { + let message = error.to_string(); + assert!( + message.contains("revision gap"), + "expected the gap refusal, got {message}" + ); + }; + + // While the document is current the gap already refuses both selectors. + for selector in by_revision.clone() { + is_gap_refusal( + drive + .fetch_document_history(&query(selector), document_type, None, new) + .expect_err("a gapped history refuses by-revision reads"), + ); + } + + let apply = |batch: Vec| { + drive + .apply_batch_low_level_drive_operations(None, None, batch, &mut vec![], &new.drive) + .expect("expected to apply the operations"); + }; + apply( + drive + .delete_document_for_contract_operations( + id, + &contract, + document_type, + &BlockInfo::default_with_time(5_000), + Some(Identifier::new(owner)), + None, + &mut None, + None, + new, + ) + .expect("expected to build the delete"), + ); + let page = drive + .fetch_document_history( + &query(DocumentHistoryFilter::StartAtTime(0)), + document_type, + None, + new, + ) + .expect("time pagination still reads a deleted gapped history"); + let lifecycle = page + .lifecycle + .expect("protocol 15 authenticates lifecycle metadata"); + assert_eq!(lifecycle.state, DocumentHistoryState::Deleted); + assert_eq!(lifecycle.remaining_revisions, 2); + assert_eq!( + page.entries + .iter() + .map(|entry| entry.revision) + .collect::>(), + vec![1, 3] + ); + for selector in by_revision.clone() { + let query = query(selector.clone()); + is_gap_refusal( + drive + .fetch_document_history(&query, document_type, None, new) + .expect_err("a deleted gapped history refuses by-revision reads"), + ); + is_gap_refusal( + drive + .prove_document_history(&query, document_type, None, new) + .expect_err("a deleted gapped history cannot be proved by revision"), + ); + } + + // Two revisions fit in one erase chunk, so the erasure finishes at once + // and the id is unused again: a by-revision read of an unused id is an + // ordinary empty page, authenticated as an absence rather than refused. + apply( + drive + .erase_document_for_contract_operations( + id, + &contract, + document_type, + &BlockInfo::default_with_time(6_000), + &mut None, + None, + new, + ) + .expect("expected to build the erase"), + ); + for selector in by_revision { + let page = drive + .fetch_document_history(&query(selector), document_type, None, new) + .expect("an unused id reads as an authenticated absence"); + assert_eq!( + page.lifecycle + .expect("protocol 15 authenticates lifecycle metadata") + .state, + DocumentHistoryState::Absent + ); + assert!(page.entries.is_empty()); + } +} + +/// A lifecycle delete given only the contract's id has to fetch the contract, +/// and that fetch is billed: its operation must come back with the delete's +/// operations rather than be dropped on the way out, as it is for an erase. +#[test] +fn should_keep_the_contract_fetch_cost_of_a_lifecycle_delete_by_contract_id() { + use crate::fees::op::LowLevelDriveOperation; + use crate::util::batch::drive_op_batch::DriveLowLevelOperationConverter; + use crate::util::batch::DocumentOperationType; + use crate::util::object_size_info::{DataContractInfo, DocumentTypeInfo}; + + let owner = [48u8; 32]; + let version = latest(); + let (drive, contract, id) = setup_history(1, owner); + let block_info = BlockInfo::default_with_time(5_000); + + let delete_with = |contract_info: DataContractInfo| { + DocumentOperationType::DeleteDocument { + document_id: id, + deleter_id: Some(Identifier::new(owner)), + contract_info, + document_type_info: DocumentTypeInfo::DocumentTypeName("person".to_string()), + } + .into_low_level_drive_operations(&drive, &mut None, &block_info, None, version) + .expect("expected the delete to convert") + }; + + // The delete bills its own reads too, so the comparison is against the + // borrowed-contract path, which performs no fetch. + let borrowed = delete_with(DataContractInfo::BorrowedDataContract(&contract)); + assert!(!borrowed.is_empty(), "the delete itself emits operations"); + + let by_id = delete_with(DataContractInfo::DataContractId(contract.id())); + assert!( + matches!( + by_id.first(), + Some(LowLevelDriveOperation::PreCalculatedFeeResult(_)) + ), + "the contract fetch is billed first, got {:?}", + by_id.first() + ); + assert_eq!( + by_id[1..], + borrowed[..], + "after the fetch cost, the by-id path emits exactly the delete's operations" + ); +} + +/// The erase operation is a public Drive method, reachable without the +/// transition validation that checks the lifecycle. It has to refuse a +/// document that is not deleted on its own: a document that was never deleted, +/// and one whose id was reused after an earlier erasure finished, both keep +/// every revision. +#[test] +fn should_refuse_to_erase_a_document_that_is_not_deleted() { + let owner = [49u8; 32]; + let version = latest(); + let (drive, contract, id) = setup_history(2, owner); + let document_type = document_type_of(&contract); + let block_info = BlockInfo::default_with_time(5_000); + let attempt = |drive: &Drive| { + drive.erase_document_for_contract_operations( + id, + &contract, + document_type, + &block_info, + &mut None, + None, + version, + ) + }; + let revisions_retained = |drive: &Drive| { + drive + .fetch_document_history( + &DocumentHistoryDriveQuery { + contract_id: contract.id().to_buffer(), + document_type_name: "person".into(), + document_id: id.to_buffer(), + filter: DocumentHistoryFilter::StartAtTime(0), + limit: Some(10), + }, + document_type, + None, + version, + ) + .expect("expected to read the history") + .entries + .len() + }; + + // Never deleted. + let error = attempt(&drive).expect_err("an active document must not be erasable"); + assert!( + matches!(error, Error::Drive(DriveError::InvalidInput(_))), + "got {error:?}" + ); + assert!(matches!( + lifecycle_of(&drive, &contract, id), + DocumentLifecycleState::Active(_) + )); + assert_eq!(revisions_retained(&drive), 2); + + // Deleted, fully erased, then the id reused by a fresh document: the + // earlier erasure left the per-type lifecycle container behind, and that + // must not stand in for a record of this document. + delete(&drive, &contract, id, Identifier::new(owner), 6_000); + erase(&drive, &contract, id, 7_000); + assert!(matches!( + lifecycle_of(&drive, &contract, id), + DocumentLifecycleState::Absent + )); + let mut recreated = + json_document_to_document(PERSON, Some(owner.into()), document_type, version) + .expect("expected a person document"); + recreated.set_revision(Some(1)); + assert_eq!(recreated.id(), id, "the fixture reproduces the same id"); + drive + .add_document_for_contract( + DocumentAndContractInfo { + owned_document_info: OwnedDocumentInfo { + document_info: DocumentInfo::DocumentRefInfo(( + &recreated, + Some(Cow::Owned(StorageFlags::new_single_epoch(0, Some(owner)))), + )), + owner_id: None, + }, + contract: &contract, + document_type, + }, + false, + BlockInfo::default_with_time(8_000), + true, + None, + version, + None, + ) + .expect("expected the id to be free again"); + + let error = attempt(&drive).expect_err("a recreated document must not be erasable"); + assert!( + matches!(error, Error::Drive(DriveError::InvalidInput(_))), + "got {error:?}" + ); + assert!(matches!( + lifecycle_of(&drive, &contract, id), + DocumentLifecycleState::Active(_) + )); + assert_eq!(revisions_retained(&drive), 1); +} diff --git a/packages/rs-drive/src/drive/document/migration/tests.rs b/packages/rs-drive/src/drive/document/migration/tests.rs index 3513d0075ce..bb791d23741 100644 --- a/packages/rs-drive/src/drive/document/migration/tests.rs +++ b/packages/rs-drive/src/drive/document/migration/tests.rs @@ -638,3 +638,90 @@ fn should_halt_migration_when_an_inventoried_index_reference_was_not_rewritten() .unwrap(); assert_eq!(stats.rewritten_index_entries, references.len() as u64); } + +/// A keep-history type with a contested index could be registered before +/// protocol 15. Activation loads every stored contract through the structural +/// parse, which must keep reading that combination, or the migration would +/// abort on the first such contract before touching any history. +#[test] +fn should_migrate_a_legacy_contested_keep_history_contract() { + use dpp::data_contract::DataContractFactory; + use dpp::document::document_factory::DocumentFactory; + use dpp::platform_value::platform_value; + + let old = PlatformVersion::get(14).unwrap(); + let new = PlatformVersion::get(15).unwrap(); + let schema = platform_value!({ + "type": "object", + "documentsKeepHistory": true, + "canBeDeleted": false, + "documentsMutable": false, + "properties": { + "label": {"type": "string", "maxLength": 50, "position": 0}, + }, + "required": ["label"], + "additionalProperties": false, + "indices": [{ + "name": "byLabel", + "properties": [{"label": "asc"}], + "unique": true, + "contested": { + "fieldMatches": [{"field": "label", "regexPattern": "^[a-z]{3,10}$"}], + "resolution": 0, + }, + }], + }); + let contract = DataContractFactory::new(14) + .unwrap() + .create_with_value_config( + [7; 32].into(), + 0, + platform_value!({"name": schema}), + None, + None, + ) + .expect("protocol 14 admits a contested keep-history type") + .data_contract_owned(); + let drive = setup_drive_with_initial_state_structure(None); + drive + .apply_contract(&contract, BlockInfo::default(), true, None, None, old) + .unwrap(); + let document_type = contract.document_type_for_name("name").unwrap(); + let mut document = DocumentFactory::new(14) + .unwrap() + .create_document( + &contract, + [7; 32].into(), + "name".into(), + platform_value!({"label": "alice"}), + ) + .unwrap(); + // An immutable type carries no revision; its history is keyed by time. + document.set_id([9; 32].into()); + drive + .add_document_for_contract( + DocumentAndContractInfo { + owned_document_info: OwnedDocumentInfo { + document_info: DocumentInfo::DocumentRefInfo((&document, None)), + owner_id: None, + }, + contract: &contract, + document_type, + }, + true, + BlockInfo::default_with_time(1_000), + true, + None, + old, + None, + ) + .unwrap(); + + let transaction = drive.grove.start_transaction(); + let stats = drive + .migrate_document_history_storage(&transaction, new) + .expect("a stored contested keep-history contract must load and migrate"); + assert_eq!(stats.contracts, 1); + assert_eq!(stats.types, 1); + assert_eq!(stats.migrated_documents, 1); +} diff --git a/packages/rs-drive/src/drive/document/mod.rs b/packages/rs-drive/src/drive/document/mod.rs index 5be0714d739..b808b80ecbf 100644 --- a/packages/rs-drive/src/drive/document/mod.rs +++ b/packages/rs-drive/src/drive/document/mod.rs @@ -34,6 +34,9 @@ mod index_uniqueness; mod insert; #[cfg(any(feature = "server", feature = "fixtures-and-mocks"))] mod insert_contested; +/// The lifecycle record of a deleted keep-history document. +#[cfg(any(feature = "server", feature = "verify"))] +pub mod lifecycle; /// Activation migration and its inventory. #[cfg(feature = "server")] pub mod migration; diff --git a/packages/rs-drive/src/drive/document/paths.rs b/packages/rs-drive/src/drive/document/paths.rs index 893fd219d85..8ec6e70e7a9 100644 --- a/packages/rs-drive/src/drive/document/paths.rs +++ b/packages/rs-drive/src/drive/document/paths.rs @@ -12,9 +12,24 @@ use grovedb::batch::key_info::KeyInfo; #[cfg(feature = "server")] use grovedb::batch::KeyInfoPath; +/// Reserved document-type key containing the lifecycle records of documents +/// that have been deleted but whose revisions are still retained. +/// +/// Sits between the primary-key tree at 0 and the history tree at 2. Index +/// trees are keyed by property names, which cannot start below `0x30`, so no +/// index can collide with a reserved single-byte key. +pub const DOCUMENT_LIFECYCLE_TREE_KEY: u8 = 1; + /// Reserved document-type key containing the revision trees. pub const DOCUMENT_HISTORY_TREE_KEY: u8 = 2; +/// Path to the lifecycle records of one document type. +pub fn document_lifecycle_path(contract_id: &[u8], document_type_name: &str) -> Vec> { + let mut path = contract_document_type_path_vec(contract_id, document_type_name); + path.push(vec![DOCUMENT_LIFECYCLE_TREE_KEY]); + path +} + /// Path to all retained revisions of one document. pub fn document_history_path( contract_id: &[u8], diff --git a/packages/rs-drive/src/query/document_history_drive_query.rs b/packages/rs-drive/src/query/document_history_drive_query.rs index ed2cccf35ad..ccf0c132e65 100644 --- a/packages/rs-drive/src/query/document_history_drive_query.rs +++ b/packages/rs-drive/src/query/document_history_drive_query.rs @@ -1,7 +1,9 @@ //! A page of a document's retained revisions with its lifecycle, read from //! whichever history layout the protocol version stores. -use crate::drive::document::paths::{contract_document_type_path_vec, DOCUMENT_HISTORY_TREE_KEY}; +use crate::drive::document::paths::{ + contract_document_type_path_vec, DOCUMENT_HISTORY_TREE_KEY, DOCUMENT_LIFECYCLE_TREE_KEY, +}; use crate::drive::document::MAX_DOCUMENT_HISTORY_FETCH_LIMIT; use crate::drive::Drive; use crate::error::drive::DriveError; @@ -9,8 +11,10 @@ use crate::error::query::QuerySyntaxError; use crate::error::Error; use crate::verify::RootHash; use dpp::data_contract::document_type::{DocumentPropertyType, DocumentTypeRef}; +use dpp::document::lifecycle::DocumentLifecycleRecord; use dpp::document::serialization_traits::DocumentPlatformConversionMethodsV0; use dpp::document::{Document, DocumentV0Getters}; +use dpp::serialization::PlatformDeserializableUntrusted; use dpp::version::PlatformVersion; #[cfg(feature = "server")] use grovedb::TransactionArg; @@ -58,6 +62,24 @@ pub enum DocumentHistoryState { Active, /// Neither a current document nor a retained history exists. Absent, + /// The document has been deleted; its revisions are still retained. + Deleted, + /// An authorized erasure has started and has not finished. + Erasing, +} + +/// Authenticated lifecycle timestamps independent of the requested page. +#[derive(Debug, Clone, PartialEq, Eq, Default)] +pub struct DocumentHistoryLifecycleTimes { + /// Block time the document was deleted at, zero while it is active. + pub deleted_at_ms: u64, + /// Block time an authorized erasure started at, zero while none has. + pub erasing_started_at_ms: u64, + /// Timestamp of the newest revision retained when the erasure started. + pub erasing_from_time_ms: u64, + /// History sequence of the newest revision retained when the erasure + /// started. + pub erasing_from_revision: u64, } /// Authenticated history metadata independent of the requested page. @@ -67,6 +89,8 @@ pub struct DocumentHistoryLifecycle { pub state: DocumentHistoryState, /// Count authenticated by the per-document count-tree element. pub remaining_revisions: u64, + /// Times recorded by the lifecycle record, all zero unless one exists. + pub times: DocumentHistoryLifecycleTimes, } /// One retained edit, including its complete pagination cursor. @@ -212,7 +236,7 @@ impl DocumentHistoryDriveQuery { /// Queries the pointer, lifecycle reservation, and raw history tree separately. pub fn metadata_path_query(&self, version: &PlatformVersion) -> Result { self.validate()?; - let queries = [0, 1, DOCUMENT_HISTORY_TREE_KEY].map(|branch| { + let queries = [0, DOCUMENT_LIFECYCLE_TREE_KEY, DOCUMENT_HISTORY_TREE_KEY].map(|branch| { let mut path = contract_document_type_path_vec(&self.contract_id, &self.document_type_name); path.push(vec![branch]); @@ -238,9 +262,12 @@ impl DocumentHistoryDriveQuery { primary.push(vec![0]); let mut history = primary.clone(); history[4] = vec![DOCUMENT_HISTORY_TREE_KEY]; + let mut lifecycle = primary.clone(); + lifecycle[4] = vec![DOCUMENT_LIFECYCLE_TREE_KEY]; let mut active = false; let mut latest_revision = None; let mut count = None; + let mut record = None; for (path, key, element) in metadata { let Some(element) = element else { continue; @@ -248,7 +275,19 @@ impl DocumentHistoryDriveQuery { if key != self.document_id { return Err(corrupt("history metadata key does not match the document")); } - if path == primary { + if path == lifecycle { + let Element::Item(bytes, _) = element else { + return Err(corrupt("a lifecycle record is not an item")); + }; + if record + .replace(DocumentLifecycleRecord::deserialize_from_bytes_untrusted( + &bytes, + )?) + .is_some() + { + return Err(corrupt("duplicate lifecycle record")); + } + } else if path == primary { let bytes = match element { Element::Item(bytes, _) | Element::ItemWithSumItem(bytes, _, _) => bytes, _ => { @@ -281,27 +320,64 @@ impl DocumentHistoryDriveQuery { )); } } - if active && count.unwrap_or_default() == 0 || !active && count.unwrap_or_default() > 0 { + if active && record.is_some() { + return Err(corrupt( + "a current document cannot also carry a lifecycle record", + )); + } + if count == Some(0) { + return Err(corrupt("an empty history tree must not remain in storage")); + } + if active && count.unwrap_or_default() == 0 + || !active && record.is_none() && count.unwrap_or_default() > 0 + { return Err(corrupt("current document and retained history disagree")); } + if record.is_some() && count.unwrap_or_default() == 0 { + return Err(corrupt("a lifecycle record survives its retained history")); + } + // A by-revision read maps the revision onto a position, which only + // holds while the retained revisions number one through the latest + // without a gap. A current document says so through its revision; a + // deleted one through what its record kept from the moment of + // deletion, since an erase only ever removes the newest revisions and + // so cannot open a gap afterwards. if matches!( self.filter, DocumentHistoryFilter::Revision(_) | DocumentHistoryFilter::StartAtRevision(_) - ) && active - && latest_revision != count - { - return Err(invalid( - "retained history contains a revision gap; use time pagination", - )); + ) { + let gapped = match (active, &record) { + (true, _) => latest_revision != count, + (false, Some(record)) => !record.revisions_are_contiguous(), + (false, None) => false, + }; + if gapped { + return Err(invalid( + "retained history contains a revision gap; use time pagination", + )); + } } + let state = match (active, &record) { + (true, _) => DocumentHistoryState::Active, + (false, Some(record)) if record.is_erasing() => DocumentHistoryState::Erasing, + (false, Some(_)) => DocumentHistoryState::Deleted, + // Retained history with no record and no pointer was rejected as + // inconsistent above, so nothing is left here but an unused id. + (false, None) => DocumentHistoryState::Absent, + }; + let times = record + .map(|record| DocumentHistoryLifecycleTimes { + deleted_at_ms: record.deleted_at_ms(), + erasing_started_at_ms: record.erasing_started_at_ms(), + erasing_from_time_ms: record.erasing_from_time_ms(), + erasing_from_revision: record.erasing_from_revision(), + }) + .unwrap_or_default(); Ok(( DocumentHistoryLifecycle { - state: if active { - DocumentHistoryState::Active - } else { - DocumentHistoryState::Absent - }, + state, remaining_revisions: count.unwrap_or_default(), + times, }, count.is_some(), )) diff --git a/packages/rs-drive/src/query/filter.rs b/packages/rs-drive/src/query/filter.rs index 5913805cf02..8fc9bd6fa8c 100644 --- a/packages/rs-drive/src/query/filter.rs +++ b/packages/rs-drive/src/query/filter.rs @@ -324,6 +324,10 @@ impl DriveDocumentQueryFilter<'_> { TransitionCheckResult::Fail } } + // An erase carries no document values and acts on a document that + // is already invisible to every document query, so no + // content-based subscription filter can describe it. + DocumentTransition::Erase(_) => TransitionCheckResult::Fail, DocumentTransition::IndexOnlyDelete(index_only_delete) => { if let DocumentActionMatchClauses::Delete { original_document_clauses, diff --git a/packages/rs-drive/src/query/mod.rs b/packages/rs-drive/src/query/mod.rs index d4c2f405f68..4be93d86dd9 100644 --- a/packages/rs-drive/src/query/mod.rs +++ b/packages/rs-drive/src/query/mod.rs @@ -202,7 +202,8 @@ pub mod document_history_drive_query; #[cfg(any(feature = "server", feature = "verify"))] pub use document_history_drive_query::{ DocumentHistoryDriveQuery, DocumentHistoryDriveQueryExecutionResult, DocumentHistoryEntry, - DocumentHistoryFilter, DocumentHistoryLifecycle, DocumentHistoryState, + DocumentHistoryFilter, DocumentHistoryLifecycle, DocumentHistoryLifecycleTimes, + DocumentHistoryState, }; #[cfg(any(feature = "server", feature = "verify"))] diff --git a/packages/rs-drive/src/state_transition_action/action_convert_to_operations/batch/document/document_delete_transition.rs b/packages/rs-drive/src/state_transition_action/action_convert_to_operations/batch/document/document_delete_transition.rs index e0bef2d3979..d0d42d34423 100644 --- a/packages/rs-drive/src/state_transition_action/action_convert_to_operations/batch/document/document_delete_transition.rs +++ b/packages/rs-drive/src/state_transition_action/action_convert_to_operations/batch/document/document_delete_transition.rs @@ -49,6 +49,7 @@ impl DriveHighLevelBatchOperationConverter for DocumentDeleteTransitionAction { }), DocumentOperation(DocumentOperationType::DeleteDocument { document_id: base.id(), + deleter_id: None, contract_info: DataContractInfo::DataContractFetchInfo( base.data_contract_fetch_info(), ), @@ -83,10 +84,59 @@ impl DriveHighLevelBatchOperationConverter for DocumentDeleteTransitionAction { Ok(ops) } + 1 => { + let base = self.base_owned(); + let contract_fetch_info = base.data_contract_fetch_info(); + let data_contract_id = base.data_contract_id(); + let identity_contract_nonce = base.identity_contract_nonce(); + let document_deletion_token_cost = base.token_cost(); + + let mut ops = vec![ + IdentityOperation(IdentityOperationType::UpdateIdentityContractNonce { + identity_id: owner_id.into_buffer(), + contract_id: data_contract_id.into_buffer(), + nonce: identity_contract_nonce, + }), + DocumentOperation(DocumentOperationType::DeleteDocument { + document_id: base.id(), + deleter_id: Some(owner_id), + contract_info: DataContractInfo::DataContractFetchInfo( + base.data_contract_fetch_info(), + ), + document_type_info: DocumentTypeInfo::DocumentTypeName( + base.document_type_name_owned(), + ), + }), + ]; + + if let Some((token_id, effect, cost)) = document_deletion_token_cost { + match effect { + DocumentActionTokenEffect::TransferTokenToContractOwner => { + if owner_id != contract_fetch_info.contract.owner_id() { + ops.push(TokenOperation(TokenOperationType::TokenTransfer { + token_id, + sender_id: owner_id, + recipient_id: contract_fetch_info.contract.owner_id(), + amount: cost, + })); + } + } + DocumentActionTokenEffect::BurnToken => { + ops.push(TokenOperation(TokenOperationType::TokenBurn { + token_id, + identity_balance_holder_id: owner_id, + burn_amount: cost, + })); + } + } + } + + Ok(ops) + } version => Err(Error::Drive(DriveError::UnknownVersionMismatch { method: "DocumentDeleteTransitionAction::into_high_level_document_drive_operations" .to_string(), - known_versions: vec![0], + known_versions: vec![0, 1], received: version, })), } diff --git a/packages/rs-drive/src/state_transition_action/action_convert_to_operations/batch/document/document_erase_transition.rs b/packages/rs-drive/src/state_transition_action/action_convert_to_operations/batch/document/document_erase_transition.rs new file mode 100644 index 00000000000..0ef738b3131 --- /dev/null +++ b/packages/rs-drive/src/state_transition_action/action_convert_to_operations/batch/document/document_erase_transition.rs @@ -0,0 +1,68 @@ +use crate::state_transition_action::action_convert_to_operations::batch::DriveHighLevelBatchOperationConverter; + +use crate::util::batch::DriveOperation::{DocumentOperation, IdentityOperation}; +use crate::util::batch::{DocumentOperationType, DriveOperation, IdentityOperationType}; + +use crate::error::drive::DriveError; +use crate::error::Error; +use crate::state_transition_action::batch::batched_transition::document_transition::document_base_transition_action::DocumentBaseTransitionActionAccessorsV0; +use crate::state_transition_action::batch::batched_transition::document_transition::document_erase_transition_action::v0::DocumentEraseTransitionActionAccessorsV0; +use crate::state_transition_action::batch::batched_transition::document_transition::document_erase_transition_action::DocumentEraseTransitionAction; +use crate::util::object_size_info::{DataContractInfo, DocumentTypeInfo}; +use dpp::block::epoch::Epoch; +use dpp::identifier::Identifier; +use dpp::version::PlatformVersion; + +impl DriveHighLevelBatchOperationConverter for DocumentEraseTransitionAction { + fn into_high_level_batch_drive_operations<'b>( + self, + _epoch: &Epoch, + owner_id: Identifier, + platform_version: &PlatformVersion, + ) -> Result>, Error> { + match platform_version + .drive + .methods + .state_transitions + .convert_to_high_level_operations + .document_erase_transition + { + Some(0) => { + let base = self.base_owned(); + let data_contract_id = base.data_contract_id(); + let identity_contract_nonce = base.identity_contract_nonce(); + + // No token operation: erase carries no token cost, and a + // transition that offers to pay one is refused where the offer + // is still visible, when the transition becomes an action. + Ok(vec![ + IdentityOperation(IdentityOperationType::UpdateIdentityContractNonce { + identity_id: owner_id.into_buffer(), + contract_id: data_contract_id.into_buffer(), + nonce: identity_contract_nonce, + }), + DocumentOperation(DocumentOperationType::EraseDocument { + document_id: base.id(), + contract_info: DataContractInfo::DataContractFetchInfo( + base.data_contract_fetch_info(), + ), + document_type_info: DocumentTypeInfo::DocumentTypeName( + base.document_type_name_owned(), + ), + }), + ]) + } + Some(version) => Err(Error::Drive(DriveError::UnknownVersionMismatch { + method: "DocumentEraseTransitionAction::into_high_level_document_drive_operations" + .to_string(), + known_versions: vec![0], + received: version, + })), + None => Err(Error::Drive(DriveError::VersionNotActive { + method: "DocumentEraseTransitionAction::into_high_level_document_drive_operations" + .to_string(), + known_versions: vec![0], + })), + } + } +} diff --git a/packages/rs-drive/src/state_transition_action/action_convert_to_operations/batch/document/document_transition.rs b/packages/rs-drive/src/state_transition_action/action_convert_to_operations/batch/document/document_transition.rs index 6a6deca3152..2178cdb16a3 100644 --- a/packages/rs-drive/src/state_transition_action/action_convert_to_operations/batch/document/document_transition.rs +++ b/packages/rs-drive/src/state_transition_action/action_convert_to_operations/batch/document/document_transition.rs @@ -56,6 +56,13 @@ impl DriveHighLevelBatchOperationConverter for DocumentTransitionAction { platform_version, ) } + DocumentTransitionAction::EraseAction(document_erase_transition) => { + document_erase_transition.into_high_level_batch_drive_operations( + epoch, + owner_id, + platform_version, + ) + } DocumentTransitionAction::IndexOnlyDeleteAction( document_index_only_delete_transition, ) => document_index_only_delete_transition.into_high_level_batch_drive_operations( diff --git a/packages/rs-drive/src/state_transition_action/action_convert_to_operations/batch/document/mod.rs b/packages/rs-drive/src/state_transition_action/action_convert_to_operations/batch/document/mod.rs index e18cf20ed34..b8c70bd80c2 100644 --- a/packages/rs-drive/src/state_transition_action/action_convert_to_operations/batch/document/mod.rs +++ b/packages/rs-drive/src/state_transition_action/action_convert_to_operations/batch/document/mod.rs @@ -1,5 +1,6 @@ mod document_create_transition; mod document_delete_transition; +mod document_erase_transition; mod document_index_only_delete_transition; mod document_purchase_transition; mod document_replace_transition; diff --git a/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_erase_transition_action/mod.rs b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_erase_transition_action/mod.rs new file mode 100644 index 00000000000..31de99fc39b --- /dev/null +++ b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_erase_transition_action/mod.rs @@ -0,0 +1,31 @@ +use derive_more::From; + +use crate::state_transition_action::batch::batched_transition::document_transition::document_erase_transition_action::v0::{DocumentEraseTransitionActionAccessorsV0, DocumentEraseTransitionActionV0}; + +/// transformer +pub mod transformer; +/// v0 +pub mod v0; + +use crate::state_transition_action::batch::batched_transition::document_transition::document_base_transition_action::DocumentBaseTransitionAction; + +/// document erase transition action +#[derive(Debug, Clone, From)] +pub enum DocumentEraseTransitionAction { + /// v0 + V0(DocumentEraseTransitionActionV0), +} + +impl DocumentEraseTransitionActionAccessorsV0 for DocumentEraseTransitionAction { + fn base(&self) -> &DocumentBaseTransitionAction { + match self { + DocumentEraseTransitionAction::V0(v0) => &v0.base, + } + } + + fn base_owned(self) -> DocumentBaseTransitionAction { + match self { + DocumentEraseTransitionAction::V0(v0) => v0.base, + } + } +} diff --git a/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_erase_transition_action/transformer.rs b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_erase_transition_action/transformer.rs new file mode 100644 index 00000000000..d603791944a --- /dev/null +++ b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_erase_transition_action/transformer.rs @@ -0,0 +1,37 @@ +use crate::drive::contract::DataContractFetchInfo; +use crate::error::Error; +use crate::state_transition_action::batch::batched_transition::document_transition::document_erase_transition_action::{DocumentEraseTransitionAction, DocumentEraseTransitionActionV0}; +use crate::state_transition_action::batch::batched_transition::BatchedTransitionAction; +use dpp::fee::fee_result::FeeResult; +use dpp::platform_value::Identifier; +use dpp::prelude::{ConsensusValidationResult, UserFeeIncrease}; +use dpp::state_transition::batch_transition::batched_transition::DocumentEraseTransition; +use dpp::ProtocolError; +use std::sync::Arc; + +impl DocumentEraseTransitionAction { + /// from borrowed + pub fn try_from_document_borrowed_erase_transition_with_contract_lookup( + value: &DocumentEraseTransition, + owner_id: Identifier, + user_fee_increase: UserFeeIncrease, + get_data_contract: impl Fn(Identifier) -> Result, ProtocolError>, + ) -> Result< + ( + ConsensusValidationResult, + FeeResult, + ), + Error, + > { + match value { + DocumentEraseTransition::V0(v0) => { + DocumentEraseTransitionActionV0::try_from_borrowed_document_erase_transition_with_contract_lookup( + v0, + owner_id, + user_fee_increase, + get_data_contract, + ) + } + } + } +} diff --git a/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_erase_transition_action/v0/mod.rs b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_erase_transition_action/v0/mod.rs new file mode 100644 index 00000000000..6b9805f40c1 --- /dev/null +++ b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_erase_transition_action/v0/mod.rs @@ -0,0 +1,19 @@ +/// transformer +pub mod transformer; + +use crate::state_transition_action::batch::batched_transition::document_transition::document_base_transition_action::DocumentBaseTransitionAction; + +#[derive(Debug, Clone)] +/// document erase transition action v0 +pub struct DocumentEraseTransitionActionV0 { + /// base + pub base: DocumentBaseTransitionAction, +} + +/// document erase transition action accessors v0 +pub trait DocumentEraseTransitionActionAccessorsV0 { + /// base + fn base(&self) -> &DocumentBaseTransitionAction; + /// base owned + fn base_owned(self) -> DocumentBaseTransitionAction; +} diff --git a/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_erase_transition_action/v0/transformer.rs b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_erase_transition_action/v0/transformer.rs new file mode 100644 index 00000000000..5f1262561ae --- /dev/null +++ b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_erase_transition_action/v0/transformer.rs @@ -0,0 +1,105 @@ +use crate::drive::contract::DataContractFetchInfo; +use crate::error::Error; +use crate::state_transition_action::batch::batched_transition::document_transition::document_base_transition_action::DocumentBaseTransitionAction; +use crate::state_transition_action::batch::batched_transition::document_transition::document_erase_transition_action::v0::DocumentEraseTransitionActionV0; +use crate::state_transition_action::batch::batched_transition::document_transition::DocumentTransitionAction; +use crate::state_transition_action::batch::batched_transition::BatchedTransitionAction; +use crate::state_transition_action::system::bump_identity_data_contract_nonce_action::BumpIdentityDataContractNonceAction; +use dpp::consensus::basic::document::InvalidDocumentTransitionActionError; +use dpp::consensus::basic::BasicError; +use dpp::consensus::ConsensusError; +use dpp::fee::fee_result::FeeResult; +use dpp::state_transition::batch_transition::document_base_transition::v0::v0_methods::DocumentBaseTransitionV0Methods; +use dpp::state_transition::batch_transition::document_base_transition::v1::v1_methods::DocumentBaseTransitionV1Methods; +use dpp::platform_value::Identifier; +use dpp::prelude::{ConsensusValidationResult, UserFeeIncrease}; +use dpp::state_transition::batch_transition::batched_transition::document_erase_transition::DocumentEraseTransitionV0; +use dpp::ProtocolError; +use std::sync::Arc; + +impl DocumentEraseTransitionActionV0 { + /// try from borrowed + pub fn try_from_borrowed_document_erase_transition_with_contract_lookup( + value: &DocumentEraseTransitionV0, + owner_id: Identifier, + user_fee_increase: UserFeeIncrease, + get_data_contract: impl Fn(Identifier) -> Result, ProtocolError>, + ) -> Result< + ( + ConsensusValidationResult, + FeeResult, + ), + Error, + > { + let DocumentEraseTransitionV0 { base, .. } = value; + + // Erase carries no token cost, so an offer to pay one has nothing to + // buy. Refused here rather than in structure validation because the + // base action keeps only the cost the contract sets, not the payment + // the submitter offered: by then there is nothing left to see. + // Accepting it would let a continuation, which any identity may submit, + // move somebody else's tokens. + if base.token_payment_info_ref().is_some() { + let bump_action = + BumpIdentityDataContractNonceAction::from_borrowed_document_base_transition( + base, + owner_id, + user_fee_increase, + ); + return Ok(( + ConsensusValidationResult::new_with_data_and_errors( + BatchedTransitionAction::BumpIdentityDataContractNonce(bump_action), + vec![ConsensusError::BasicError( + BasicError::InvalidDocumentTransitionActionError( + InvalidDocumentTransitionActionError::new(format!( + "an erase of document {} must not carry token payment information", + base.id() + )), + ), + )], + ), + FeeResult::default(), + )); + } + + let base_action_validation_result = + DocumentBaseTransitionAction::try_from_borrowed_base_transition_with_contract_lookup( + base, + get_data_contract, + // Erase carries no token cost: the deletion it follows was + // charged when the document was deleted. + |_document_type| None, + "erase", + )?; + + let base = match base_action_validation_result.is_valid() { + true => base_action_validation_result.into_data()?, + false => { + let bump_action = + BumpIdentityDataContractNonceAction::from_borrowed_document_base_transition( + base, + owner_id, + user_fee_increase, + ); + let batched_action = + BatchedTransitionAction::BumpIdentityDataContractNonce(bump_action); + + return Ok(( + ConsensusValidationResult::new_with_data_and_errors( + batched_action, + base_action_validation_result.errors, + ), + FeeResult::default(), + )); + } + }; + + Ok(( + BatchedTransitionAction::DocumentAction(DocumentTransitionAction::EraseAction( + DocumentEraseTransitionActionV0 { base }.into(), + )) + .into(), + FeeResult::default(), + )) + } +} diff --git a/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_transition_action_type.rs b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_transition_action_type.rs index 4a36b30372a..3a88a2ab849 100644 --- a/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_transition_action_type.rs +++ b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_transition_action_type.rs @@ -8,6 +8,7 @@ impl DocumentTransitionActionTypeGetter for DocumentTransitionAction { match self { DocumentTransitionAction::CreateAction(_) => DocumentTransitionActionType::Create, DocumentTransitionAction::DeleteAction(_) => DocumentTransitionActionType::Delete, + DocumentTransitionAction::EraseAction(_) => DocumentTransitionActionType::Erase, DocumentTransitionAction::ReplaceAction(_) => DocumentTransitionActionType::Replace, DocumentTransitionAction::TransferAction(_) => DocumentTransitionActionType::Transfer, DocumentTransitionAction::PurchaseAction(_) => DocumentTransitionActionType::Purchase, diff --git a/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/mod.rs b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/mod.rs index 3d696150e40..65cbdff1911 100644 --- a/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/mod.rs +++ b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/mod.rs @@ -4,6 +4,8 @@ pub mod document_base_transition_action; pub mod document_create_transition_action; /// document_delete_transition_action pub mod document_delete_transition_action; +/// document_erase_transition_action +pub mod document_erase_transition_action; /// document_index_only_delete_transition_action pub mod document_index_only_delete_transition_action; /// document_purchase_transition_action @@ -26,6 +28,8 @@ use crate::state_transition_action::batch::batched_transition::document_transiti use crate::state_transition_action::batch::batched_transition::document_transition::document_delete_transition_action::v0::DocumentDeleteTransitionActionAccessorsV0; use crate::state_transition_action::batch::batched_transition::document_transition::document_index_only_delete_transition_action::v0::DocumentIndexOnlyDeleteTransitionActionAccessorsV0; use crate::state_transition_action::batch::batched_transition::document_transition::document_index_only_delete_transition_action::DocumentIndexOnlyDeleteTransitionAction; +use crate::state_transition_action::batch::batched_transition::document_transition::document_erase_transition_action::v0::DocumentEraseTransitionActionAccessorsV0; +use crate::state_transition_action::batch::batched_transition::document_transition::document_erase_transition_action::DocumentEraseTransitionAction; use crate::state_transition_action::batch::batched_transition::document_transition::document_purchase_transition_action::{DocumentPurchaseTransitionAction, DocumentPurchaseTransitionActionAccessorsV0}; use crate::state_transition_action::batch::batched_transition::document_transition::document_transfer_transition_action::{DocumentTransferTransitionAction, DocumentTransferTransitionActionAccessorsV0}; use crate::state_transition_action::batch::batched_transition::document_transition::document_update_price_transition_action::{DocumentUpdatePriceTransitionAction, DocumentUpdatePriceTransitionActionAccessorsV0}; @@ -52,6 +56,9 @@ pub enum DocumentTransitionAction { /// indexOnly delete-by-values — carries the document's property /// values, since there is no primary-storage row to fetch them from IndexOnlyDeleteAction(DocumentIndexOnlyDeleteTransitionAction), + /// Removes the retained revisions of an already deleted keep-history + /// document, a chunk at a time. + EraseAction(DocumentEraseTransitionAction), } impl DocumentTransitionAction { @@ -65,6 +72,7 @@ impl DocumentTransitionAction { DocumentTransitionAction::PurchaseAction(d) => d.base(), DocumentTransitionAction::UpdatePriceAction(d) => d.base(), DocumentTransitionAction::IndexOnlyDeleteAction(d) => d.base(), + DocumentTransitionAction::EraseAction(d) => d.base(), } } @@ -78,6 +86,7 @@ impl DocumentTransitionAction { DocumentTransitionAction::PurchaseAction(d) => d.base_owned(), DocumentTransitionAction::UpdatePriceAction(d) => d.base_owned(), DocumentTransitionAction::IndexOnlyDeleteAction(d) => d.base_owned(), + DocumentTransitionAction::EraseAction(d) => d.base_owned(), } } } diff --git a/packages/rs-drive/src/state_transition_action/batch/tests.rs b/packages/rs-drive/src/state_transition_action/batch/tests.rs index 8ffacdeb10b..3038b804458 100644 --- a/packages/rs-drive/src/state_transition_action/batch/tests.rs +++ b/packages/rs-drive/src/state_transition_action/batch/tests.rs @@ -3039,3 +3039,110 @@ fn should_stamp_fetched_contract_version_on_replace_conversion() { .expect("owned replace conversion"); assert_eq!(owned.contract_version(), Some(STAMP_TEST_CONTRACT_VERSION)); } + +// ============================================================ +// Erase action inside the batch action +// ============================================================ + +mod erase_action { + use super::*; + use crate::state_transition_action::batch::batched_transition::document_transition::document_erase_transition_action::v0::DocumentEraseTransitionActionV0; + use crate::state_transition_action::batch::batched_transition::document_transition::document_erase_transition_action::DocumentEraseTransitionAction; + use crate::state_transition_action::batch::v0::BatchTransitionActionV0; + use dpp::data_contract::document_type::accessors::DocumentTypeV0Getters; + use dpp::identity::SecurityLevel; + + fn erase() -> BatchedTransitionAction { + BatchedTransitionAction::DocumentAction(DocumentTransitionAction::EraseAction( + DocumentEraseTransitionAction::V0(DocumentEraseTransitionActionV0 { + base: test_document_base(), + }), + )) + } + + fn delete() -> BatchedTransitionAction { + BatchedTransitionAction::DocumentAction(DocumentTransitionAction::DeleteAction( + DocumentDeleteTransitionAction::V0(DocumentDeleteTransitionActionV0 { + base: test_document_base(), + }), + )) + } + + fn batch(transitions: Vec) -> BatchTransitionAction { + BatchTransitionAction::V0(BatchTransitionActionV0 { + owner_id: Identifier::from([0x11; 32]), + transitions, + user_fee_increase: 10, + }) + } + + /// An erase is signed like any other action on its document type: the + /// key it demands is the one the type demands, exactly as for a delete. + #[test] + fn should_require_the_erased_types_security_level() { + let domain_level = test_dpns_contract_info() + .contract + .document_type_for_name("domain") + .expect("the DPNS contract has a domain type") + .security_level_requirement(); + assert_ne!( + domain_level, + SecurityLevel::MASTER, + "the test needs a type whose requirement spans more than one level" + ); + + let expected: Vec = (SecurityLevel::CRITICAL as u8..=domain_level as u8) + .map(|level| SecurityLevel::try_from(level).unwrap()) + .collect(); + assert_eq!( + batch(vec![erase()]) + .combined_security_level_requirement() + .unwrap(), + expected + ); + assert_eq!( + batch(vec![erase()]) + .combined_security_level_requirement() + .unwrap(), + batch(vec![delete()]) + .combined_security_level_requirement() + .unwrap(), + "an erase and a delete of the same type demand the same key" + ); + } + + /// The credits a batch commits up front decide whether the signer can + /// afford it; an erase commits none, so its presence changes no sum. + #[test] + fn should_commit_no_credits_up_front() { + let erase_only = batch(vec![erase()]); + assert_eq!(erase_only.all_purchases_amount().unwrap(), None); + assert_eq!( + erase_only + .all_conflicting_index_collateral_voting_funds() + .unwrap(), + None + ); + assert_eq!(erase_only.all_used_balances().unwrap(), None); + + let purchases = || { + vec![ + BatchedTransitionAction::DocumentAction(make_purchase().into()), + BatchedTransitionAction::TokenAction(make_direct_purchase().into()), + ] + }; + let without_erase = batch(purchases()); + let mut with_erase_transitions = vec![erase()]; + with_erase_transitions.extend(purchases()); + let with_erase = batch(with_erase_transitions); + assert_eq!( + with_erase.all_purchases_amount().unwrap(), + without_erase.all_purchases_amount().unwrap() + ); + assert_eq!(with_erase.all_purchases_amount().unwrap(), Some(15_000)); + assert_eq!( + with_erase.all_used_balances().unwrap(), + without_erase.all_used_balances().unwrap() + ); + } +} diff --git a/packages/rs-drive/src/util/batch/drive_op_batch/document.rs b/packages/rs-drive/src/util/batch/drive_op_batch/document.rs index 07d8e37fa0f..6633d182b8d 100644 --- a/packages/rs-drive/src/util/batch/drive_op_batch/document.rs +++ b/packages/rs-drive/src/util/batch/drive_op_batch/document.rs @@ -96,6 +96,22 @@ pub enum DocumentOperationType<'a> { }, /// Deletes a document DeleteDocument { + /// The document id + document_id: Identifier, + /// The identity credited with the lifecycle record a keep-history + /// delete writes; `None` credits nobody. + deleter_id: Option, + /// Data Contract info to potentially be resolved if needed + contract_info: DataContractInfo<'a>, + /// Document type + document_type_info: DocumentTypeInfo<'a>, + }, + /// Removes a bounded chunk of the retained revisions of an already deleted + /// keep-history document, dropping the history subtree and the lifecycle + /// record with the terminal chunk. Whether the type allows erasure and who + /// may start one are decided in transition validation, like a delete's + /// ownership; this operation assumes both. + EraseDocument { /// The document id document_id: Identifier, /// Data Contract info to potentially be resolved if needed @@ -251,6 +267,9 @@ impl DocumentOperationType<'_> { } // These write to system contracts, which have no TTL indexes. Self::AddWithdrawalDocument { .. } | Self::DocumentHistory { .. } => Ok(()), + // An erase acts on a document whose delete already removed every + // index reference, so there is nothing left to drain. + Self::EraseDocument { .. } => Ok(()), } } @@ -420,6 +439,41 @@ impl DocumentOperationType<'_> { Ok(batch_operations) } DocumentOperationType::DeleteDocument { + document_id, + deleter_id, + contract_info, + document_type_info, + } => { + let mut drive_operations: Vec = vec![]; + let contract_resolved_info = contract_info.resolve( + drive, + block_info, + transaction, + &mut drive_operations, + platform_version, + )?; + let contract = contract_resolved_info.as_ref(); + let document_type = document_type_info.resolve(contract)?; + + // The contract resolution above may have billed a fetch; keep + // its operations ahead of the delete's so the caller pays for + // both. + let mut operations = drive + .delete_document_for_contract_operations_without_ttl_drain( + document_id, + contract, + document_type, + block_info, + deleter_id, + None, + estimated_costs_only_with_layer_info, + transaction, + platform_version, + )?; + drive_operations.append(&mut operations); + Ok(drive_operations) + } + DocumentOperationType::EraseDocument { document_id, contract_info, document_type_info, @@ -435,16 +489,17 @@ impl DocumentOperationType<'_> { let contract = contract_resolved_info.as_ref(); let document_type = document_type_info.resolve(contract)?; - drive.delete_document_for_contract_operations_without_ttl_drain( + let mut operations = drive.erase_document_for_contract_operations( document_id, contract, document_type, - None, + block_info, estimated_costs_only_with_layer_info, - block_info.time_ms, transaction, platform_version, - ) + )?; + drive_operations.append(&mut operations); + Ok(drive_operations) } DocumentOperationType::DeleteIndexOnlyDocument { document_id, diff --git a/packages/rs-drive/src/verify/document/verify_document_history/tests.rs b/packages/rs-drive/src/verify/document/verify_document_history/tests.rs index e9db91b5ae8..6df417b9187 100644 --- a/packages/rs-drive/src/verify/document/verify_document_history/tests.rs +++ b/packages/rs-drive/src/verify/document/verify_document_history/tests.rs @@ -232,16 +232,36 @@ fn should_authenticate_history_pages_metadata_and_absence() { .value .unwrap(); query.filter = DocumentHistoryFilter::StartAtTime(0); - let (page, proof) = prove(&drive, &query, document_type, version); - assert!( - proof.entries_proof.is_some(), - "an empty but present tree still needs an entries proof" - ); - let (_, verified) = verify(&query, &proof, document_type, version).unwrap(); - assert_eq!(page, verified); - let mut missing = proof; - missing.entries_proof = None; - assert!(verify(&query, &missing, document_type, version).is_err()); + assert!(drive + .fetch_document_history(&query, document_type, None, version) + .is_err()); + assert!(drive + .prove_document_history(&query, document_type, None, version) + .is_err()); + + // A hostile peer can still assemble the two individually valid GroveDB + // proofs, so the verifier must reject the corrupt empty-present tree too. + let proof = DocumentHistoryProof { + metadata_proof: drive + .grove_get_proved_path_query( + &query.metadata_path_query(version).unwrap(), + None, + &mut vec![], + &version.drive, + ) + .unwrap(), + entries_proof: Some( + drive + .grove_get_proved_path_query( + &query.construct_path_query(version).unwrap(), + None, + &mut vec![], + &version.drive, + ) + .unwrap(), + ), + }; + assert!(verify(&query, &proof, document_type, version).is_err()); } #[test] diff --git a/packages/rs-drive/src/verify/state_transition/verify_state_transition_was_executed_with_proof/v0/mod.rs b/packages/rs-drive/src/verify/state_transition/verify_state_transition_was_executed_with_proof/v0/mod.rs index 47a467910e8..6151389904b 100644 --- a/packages/rs-drive/src/verify/state_transition/verify_state_transition_was_executed_with_proof/v0/mod.rs +++ b/packages/rs-drive/src/verify/state_transition/verify_state_transition_was_executed_with_proof/v0/mod.rs @@ -503,6 +503,24 @@ impl Drive { .to_string(), ))) } + DocumentTransition::Erase(erase_transition) => { + // An erase acts on a document that was already + // deleted, so the by-id proof shows it absent + // both before and after. The absence is the + // state the transition affected, not evidence + // that this erase ran; the classifier below + // reports it as such. + if document.is_some() { + return Err(Error::Proof(ProofError::IncorrectProof(format!("proof of state transition execution contained a current document for erased id {}", erase_transition.base().id())))); + } + Ok(( + root_hash, + VerifiedDocuments(BTreeMap::from([( + erase_transition.base().id(), + None, + )])), + )) + } } } BatchedTransitionRef::Token(token_transition) => { @@ -2339,10 +2357,17 @@ impl Drive { data_contract_id ))), )?; - !contract - .document_type_for_name(document_transition.document_type_name()) - .map_err(|e| Error::Proof(ProofError::UnknownContract(e.to_string())))? - .index_only() + // EXCEPT an erase: the document it acts on was already + // deleted, so the by-id proof shows the same absence + // whether or not this erase ran. + if matches!(document_transition, DocumentTransition::Erase(_)) { + false + } else { + !contract + .document_type_for_name(document_transition.document_type_name()) + .map_err(|e| Error::Proof(ProofError::UnknownContract(e.to_string())))? + .index_only() + } } Some(BatchedTransitionRef::Token(token_transition)) => { let data_contract_id = token_transition.data_contract_id(); @@ -3417,6 +3442,160 @@ mod tests { } } + /// An erase acts on a document that was already deleted, so the by-id + /// proof shows the same absence whether or not it ran. The classifier must + /// report that as affected state, while a delete over the very same proof + /// stays execution-proved: a document's absence by id is read as the + /// delete's outcome, which is what the SDK's strict wait relies on. + #[test] + fn verify_batch_document_erase_is_affected_state_not_execution_proved() { + use crate::query::{SingleDocumentDriveQuery, SingleDocumentDriveQueryContestedStatus}; + use dpp::document::DocumentV0Setters; + use dpp::state_transition::batch_transition::batched_transition::document_erase_transition::{ + DocumentEraseTransition, DocumentEraseTransitionV0, + }; + use dpp::state_transition::batch_transition::batched_transition::document_transition::DocumentTransition; + use dpp::state_transition::batch_transition::document_base_transition::v0::DocumentBaseTransitionV0; + use dpp::state_transition::batch_transition::document_base_transition::DocumentBaseTransition; + use dpp::state_transition::batch_transition::document_delete_transition::{ + DocumentDeleteTransition, DocumentDeleteTransitionV0, + }; + use dpp::state_transition::batch_transition::{BatchTransition, BatchTransitionV0}; + use dpp::tests::json_document::{json_document_to_contract, json_document_to_document}; + + let platform_version = PlatformVersion::latest(); + let drive = setup_drive_with_initial_state_structure(None); + let contract = json_document_to_contract( + "tests/supporting_files/contract/family/family-contract-with-history.json", + false, + platform_version, + ) + .expect("expected the family history contract"); + drive + .apply_contract( + &contract, + BlockInfo::default(), + true, + None, + None, + platform_version, + ) + .expect("expected to apply the contract"); + let document_type = contract + .document_type_for_name("person") + .expect("expected the person type"); + let mut document = json_document_to_document( + "tests/supporting_files/contract/family/person0.json", + Some([4u8; 32].into()), + document_type, + platform_version, + ) + .expect("expected a person"); + for revision in 1..=2u64 { + document.set_revision(Some(revision)); + drive + .add_document_for_contract( + DocumentAndContractInfo { + owned_document_info: OwnedDocumentInfo { + document_info: DocumentRefInfo((&document, None)), + owner_id: None, + }, + contract: &contract, + document_type, + }, + revision > 1, + BlockInfo::default_with_time(1_000 + revision), + true, + None, + platform_version, + None, + ) + .expect("expected to write a revision"); + } + let doc_id = document.id(); + drive + .delete_document_for_contract( + doc_id, + &contract, + "person", + BlockInfo::default_with_time(5_000), + true, + None, + platform_version, + None, + ) + .expect("expected the delete to succeed"); + + let path_query = SingleDocumentDriveQuery { + contract_id: contract.id().to_buffer(), + document_type_name: "person".to_string(), + document_type_keeps_history: true, + document_id: doc_id.to_buffer(), + block_time_ms: None, + contested_status: SingleDocumentDriveQueryContestedStatus::NotContested, + } + .construct_path_query(platform_version) + .expect("expected a by-id path query"); + let proof = drive + .grove_get_proved_path_query(&path_query, None, &mut vec![], &platform_version.drive) + .expect("expected an absence proof"); + + let base = || { + DocumentBaseTransition::V0(DocumentBaseTransitionV0 { + id: doc_id, + identity_contract_nonce: 1, + document_type_name: "person".to_string(), + data_contract_id: contract.id(), + }) + }; + let batch = |transition: DocumentTransition| { + StateTransition::Batch(BatchTransition::V0(BatchTransitionV0 { + owner_id: Default::default(), + transitions: vec![transition], + ..Default::default() + })) + }; + let contract_arc = Arc::new(contract.clone()); + let known_contracts_provider_fn: &ContractLookupFn = &|_id| Ok(Some(contract_arc.clone())); + let verify = |state_transition: StateTransition| { + Drive::verify_state_transition_was_executed_with_proof( + &state_transition, + &BlockInfo::default(), + &proof, + known_contracts_provider_fn, + platform_version, + ) + .expect("expected verification to succeed") + .1 + }; + + let erase_outcome = verify(batch(DocumentTransition::Erase( + DocumentEraseTransition::V0(DocumentEraseTransitionV0 { base: base() }), + ))); + match erase_outcome { + StateTransitionProofOutcome::AffectedState( + StateTransitionProofResult::VerifiedDocuments(documents), + ) => { + assert_eq!(documents.len(), 1); + let (id, found) = documents.into_iter().next().unwrap(); + assert_eq!(id, doc_id); + assert!(found.is_none(), "the document is absent by id"); + } + other => panic!("an erase must be classified as affected state, got {other:?}"), + } + + let delete_outcome = verify(batch(DocumentTransition::Delete( + DocumentDeleteTransition::V0(DocumentDeleteTransitionV0 { base: base() }), + ))); + assert!( + matches!( + delete_outcome, + StateTransitionProofOutcome::ExecutionProved(_) + ), + "a delete over the same proof stays execution-proved, got {delete_outcome:?}" + ); + } + // ----------------------------------------------------------------------- // Batch: document create happy path // ----------------------------------------------------------------------- diff --git a/packages/rs-drive/tests/drive_storage_ops_coverage.rs b/packages/rs-drive/tests/drive_storage_ops_coverage.rs index 6403e728fec..08d95bb4a92 100644 --- a/packages/rs-drive/tests/drive_storage_ops_coverage.rs +++ b/packages/rs-drive/tests/drive_storage_ops_coverage.rs @@ -871,6 +871,7 @@ mod document_operation_tests { let id = dpp::prelude::Identifier::new([2u8; 32]); let _op = DocumentOperationType::DeleteDocument { document_id: id, + deleter_id: None, contract_info: drive::util::object_size_info::DataContractInfo::DataContractId( dpp::prelude::Identifier::new([3u8; 32]), ), diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/mod.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/mod.rs index ec80ba92ca8..9f061b41434 100644 --- a/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/mod.rs +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/mod.rs @@ -5,6 +5,7 @@ pub mod v3; pub mod v4; pub mod v5; pub mod v6; +pub mod v7; #[derive(Clone, Debug, Default)] pub struct DPPContractVersions { diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v7.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v7.rs new file mode 100644 index 00000000000..c0f20249551 --- /dev/null +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v7.rs @@ -0,0 +1,36 @@ +use crate::version::dpp_versions::dpp_contract_versions::v6::CONTRACT_VERSIONS_V6; +use crate::version::dpp_versions::dpp_contract_versions::{ + DPPContractVersions, DocumentTypeClassMethodVersions, DocumentTypeSchemaVersions, + DocumentTypeVersions, +}; + +// Introduced in protocol version 15 for the delete and erase lifecycle of +// keep-history documents. Uses the v4 document meta-schema, which is v3 plus +// the `canBeErased` keyword. +// +// `try_from_schema` moves to 4, selecting a new document-type parser +// generation (`try_from_schema/v4`). Generation 4 admits `documentsKeepHistory` +// together with `canBeDeleted` (a delete leaves the retained revisions +// readable), parses `canBeErased` (a deleted document's revisions may be +// purged), and refuses a keep-history type that carries a contested index. +// Generation 3 keeps rejecting deletable keep-history types, so replaying a +// protocol 14 block validates contracts exactly as it did. +// +// `document_type_schema` moves to 4 in the same step: generation 4 and +// meta-schema v4 are introduced together and pair by construction. +pub const CONTRACT_VERSIONS_V7: DPPContractVersions = DPPContractVersions { + document_type_versions: DocumentTypeVersions { + class_method_versions: DocumentTypeClassMethodVersions { + try_from_schema: 4, + ..CONTRACT_VERSIONS_V6 + .document_type_versions + .class_method_versions + }, + schema: DocumentTypeSchemaVersions { + document_type_schema: 4, + ..CONTRACT_VERSIONS_V6.document_type_versions.schema + }, + ..CONTRACT_VERSIONS_V6.document_type_versions + }, + ..CONTRACT_VERSIONS_V6 +}; diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_state_transition_serialization_versions/mod.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_state_transition_serialization_versions/mod.rs index 8e5996f3ba2..e3819244196 100644 --- a/packages/rs-platform-version/src/version/dpp_versions/dpp_state_transition_serialization_versions/mod.rs +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_state_transition_serialization_versions/mod.rs @@ -3,6 +3,7 @@ use versioned_feature_core::FeatureVersionBounds; pub mod v1; pub mod v2; pub mod v3; +pub mod v4; #[derive(Clone, Debug, Default)] pub struct DPPStateTransitionSerializationVersions { @@ -28,6 +29,11 @@ pub struct DPPStateTransitionSerializationVersions { /// as `OptionalFeatureVersion`), and the batch basic-structure wire /// gate rejects the variant wherever this is `None`. pub document_index_only_delete_state_transition: Option, + /// The erase kind, which purges the retained revisions of a deleted + /// keep-history document. `None` below protocol version 15 — the kind does + /// not exist on the wire there, and the batch basic-structure wire gate + /// rejects the variant wherever this is `None`. + pub document_erase_state_transition: Option, pub document_transfer_state_transition: DocumentFeatureVersionBounds, pub document_update_price_state_transition: DocumentFeatureVersionBounds, pub document_purchase_state_transition: DocumentFeatureVersionBounds, diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_state_transition_serialization_versions/v1.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_state_transition_serialization_versions/v1.rs index e6d53d23513..db17cbcd7b5 100644 --- a/packages/rs-platform-version/src/version/dpp_versions/dpp_state_transition_serialization_versions/v1.rs +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_state_transition_serialization_versions/v1.rs @@ -98,6 +98,8 @@ pub const STATE_TRANSITION_SERIALIZATION_VERSIONS_V1: DPPStateTransitionSerializ }, // The indexOnly delete kind joins the wire at PV14. document_index_only_delete_state_transition: None, + // The erase kind joins the wire at protocol version 15. + document_erase_state_transition: None, document_transfer_state_transition: DocumentFeatureVersionBounds { bounds: FeatureVersionBounds { min_version: 0, diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_state_transition_serialization_versions/v2.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_state_transition_serialization_versions/v2.rs index c2f5c219fa2..373db1f0c13 100644 --- a/packages/rs-platform-version/src/version/dpp_versions/dpp_state_transition_serialization_versions/v2.rs +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_state_transition_serialization_versions/v2.rs @@ -98,6 +98,8 @@ pub const STATE_TRANSITION_SERIALIZATION_VERSIONS_V2: DPPStateTransitionSerializ }, // The indexOnly delete kind joins the wire at PV14. document_index_only_delete_state_transition: None, + // The erase kind joins the wire at protocol version 15. + document_erase_state_transition: None, document_transfer_state_transition: DocumentFeatureVersionBounds { bounds: FeatureVersionBounds { min_version: 0, diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_state_transition_serialization_versions/v3.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_state_transition_serialization_versions/v3.rs index d1e59b2dae3..383723f26eb 100644 --- a/packages/rs-platform-version/src/version/dpp_versions/dpp_state_transition_serialization_versions/v3.rs +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_state_transition_serialization_versions/v3.rs @@ -110,6 +110,8 @@ pub const STATE_TRANSITION_SERIALIZATION_VERSIONS_V3: DPPStateTransitionSerializ default_current_version: 0, }, }), + // The erase kind joins the wire at protocol version 15. + document_erase_state_transition: None, document_transfer_state_transition: DocumentFeatureVersionBounds { bounds: FeatureVersionBounds { min_version: 0, diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_state_transition_serialization_versions/v4.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_state_transition_serialization_versions/v4.rs new file mode 100644 index 00000000000..51392436e0d --- /dev/null +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_state_transition_serialization_versions/v4.rs @@ -0,0 +1,25 @@ +//! V4 (PV15): V3 plus the erase kind. +//! +//! `DocumentEraseTransition` purges the retained revisions of a deleted +//! keep-history document. The kind is appended to the batch's document +//! transition enum; `document_erase_state_transition` bounds its generation +//! and stays `None` in every earlier table, where the batch basic-structure +//! gate refuses the kind. + +use crate::version::dpp_versions::dpp_state_transition_serialization_versions::v3::STATE_TRANSITION_SERIALIZATION_VERSIONS_V3; +use crate::version::dpp_versions::dpp_state_transition_serialization_versions::{ + DPPStateTransitionSerializationVersions, DocumentFeatureVersionBounds, +}; +use versioned_feature_core::FeatureVersionBounds; + +pub const STATE_TRANSITION_SERIALIZATION_VERSIONS_V4: DPPStateTransitionSerializationVersions = + DPPStateTransitionSerializationVersions { + document_erase_state_transition: Some(DocumentFeatureVersionBounds { + bounds: FeatureVersionBounds { + min_version: 0, + max_version: 0, + default_current_version: 0, + }, + }), + ..STATE_TRANSITION_SERIALIZATION_VERSIONS_V3 + }; diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_validation_versions/mod.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_validation_versions/mod.rs index 1149cdaa32d..39de89da284 100644 --- a/packages/rs-platform-version/src/version/dpp_versions/dpp_validation_versions/mod.rs +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_validation_versions/mod.rs @@ -5,6 +5,7 @@ pub mod v2; pub mod v3; pub mod v4; pub mod v5; +pub mod v6; #[derive(Clone, Debug, Default)] pub struct DPPValidationVersions { diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_validation_versions/v6.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_validation_versions/v6.rs new file mode 100644 index 00000000000..dbb8ce60616 --- /dev/null +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_validation_versions/v6.rs @@ -0,0 +1,18 @@ +use crate::version::dpp_versions::dpp_validation_versions::{ + DPPValidationVersions, DocumentTypeValidationVersions, +}; + +use super::v5::DPP_VALIDATION_VERSIONS_V5; + +/// Protocol v15 validation versions. +/// +/// v2 document-type update validation knows the `canBeErased` keyword: it +/// keeps erasability immutable in both directions and lets a keep-history +/// type withdraw deletion only while it is not erasable. +pub const DPP_VALIDATION_VERSIONS_V6: DPPValidationVersions = DPPValidationVersions { + document_type: DocumentTypeValidationVersions { + validate_update: 2, + ..DPP_VALIDATION_VERSIONS_V5.document_type + }, + ..DPP_VALIDATION_VERSIONS_V5 +}; diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/mod.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/mod.rs index 6617537c899..a83d4a53486 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/mod.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/mod.rs @@ -1,5 +1,6 @@ pub mod v1; pub mod v10; +pub mod v11; pub mod v2; pub mod v3; pub mod v4; @@ -225,6 +226,8 @@ pub struct DriveAbciDocumentsStateTransitionValidationVersions { /// The indexOnly delete-by-values kind (PV14+); 0 in every earlier /// version table, where the kind cannot appear. pub document_index_only_delete_transition_structure_validation: FeatureVersion, + /// The erase kind, active from protocol version 15. + pub document_erase_transition_structure_validation: OptionalFeatureVersion, pub document_replace_transition_structure_validation: FeatureVersion, pub document_transfer_transition_structure_validation: FeatureVersion, pub document_purchase_transition_structure_validation: FeatureVersion, @@ -235,6 +238,12 @@ pub struct DriveAbciDocumentsStateTransitionValidationVersions { /// The indexOnly delete-by-values kind (PV14+); 0 in every earlier /// version table, where the kind cannot appear. pub document_index_only_delete_transition_state_validation: FeatureVersion, + /// The erase kind, active from protocol version 15. + pub document_erase_transition_state_validation: OptionalFeatureVersion, + /// Versions `fetch_keep_history_document_lifecycle`, the stateful read + /// that classifies a keep-history document as active, deleted, erasing or + /// absent. The helper is active from protocol version 15. + pub fetch_keep_history_document_lifecycle: OptionalFeatureVersion, pub document_replace_transition_state_validation: FeatureVersion, pub document_transfer_transition_state_validation: FeatureVersion, pub document_purchase_transition_state_validation: FeatureVersion, diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v1.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v1.rs index 3a23c94c751..a5faa5a3acb 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v1.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v1.rs @@ -127,6 +127,7 @@ pub const DRIVE_ABCI_VALIDATION_VERSIONS_V1: DriveAbciValidationVersions = document_create_transition_structure_validation: 0, document_delete_transition_structure_validation: 0, document_index_only_delete_transition_structure_validation: 0, + document_erase_transition_structure_validation: None, document_replace_transition_structure_validation: 0, document_transfer_transition_structure_validation: 0, document_purchase_transition_structure_validation: 0, @@ -135,6 +136,8 @@ pub const DRIVE_ABCI_VALIDATION_VERSIONS_V1: DriveAbciValidationVersions = document_create_transition_state_validation: 0, document_delete_transition_state_validation: 0, document_index_only_delete_transition_state_validation: 0, + document_erase_transition_state_validation: None, + fetch_keep_history_document_lifecycle: None, document_replace_transition_state_validation: 0, document_transfer_transition_state_validation: 0, document_purchase_transition_state_validation: 0, diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v10.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v10.rs index cc1fd27797b..d568945f5ff 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v10.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v10.rs @@ -189,6 +189,7 @@ pub const DRIVE_ABCI_VALIDATION_VERSIONS_V10: DriveAbciValidationVersions = // Protocols through 13 retain the original internal-error outcome. document_delete_transition_structure_validation: 1, document_index_only_delete_transition_structure_validation: 0, + document_erase_transition_structure_validation: None, document_replace_transition_structure_validation: 0, document_transfer_transition_structure_validation: 0, document_purchase_transition_structure_validation: 0, @@ -197,6 +198,8 @@ pub const DRIVE_ABCI_VALIDATION_VERSIONS_V10: DriveAbciValidationVersions = document_create_transition_state_validation: 2, document_delete_transition_state_validation: 0, document_index_only_delete_transition_state_validation: 0, + document_erase_transition_state_validation: None, + fetch_keep_history_document_lifecycle: None, document_replace_transition_state_validation: 1, document_transfer_transition_state_validation: 0, document_purchase_transition_state_validation: 0, diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v11.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v11.rs new file mode 100644 index 00000000000..a31ff1fbbe2 --- /dev/null +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v11.rs @@ -0,0 +1,38 @@ +use crate::version::drive_abci_versions::drive_abci_validation_versions::v10::DRIVE_ABCI_VALIDATION_VERSIONS_V10; +use crate::version::drive_abci_versions::drive_abci_validation_versions::{ + DriveAbciDocumentsStateTransitionValidationVersions, + DriveAbciStateTransitionValidationVersions, DriveAbciValidationVersions, +}; + +/// Protocol v15 validation versions: the delete and erase lifecycle of +/// keep-history documents. +/// +/// The batch generations selected by earlier protocol versions stay selected; +/// the erase kind is carried by the shipped batch and reaches them through the +/// arms appended for it, which the basic-structure gate keeps unreachable +/// below this version. +/// +/// * delete structure v2 admits a delete of a keep-history document and +/// refuses one whose type carries a contested index; delete state v1 refuses +/// a delete of a document that is already deleted or erasing as a paid +/// consensus error; create state v3 refuses the id of such a document; +/// * the erase generations validate the erase itself and read the lifecycle +/// it acts on. +pub const DRIVE_ABCI_VALIDATION_VERSIONS_V11: DriveAbciValidationVersions = + DriveAbciValidationVersions { + state_transitions: DriveAbciStateTransitionValidationVersions { + batch_state_transition: DriveAbciDocumentsStateTransitionValidationVersions { + document_delete_transition_structure_validation: 2, + document_create_transition_state_validation: 3, + document_erase_transition_structure_validation: Some(0), + document_delete_transition_state_validation: 1, + document_erase_transition_state_validation: Some(0), + fetch_keep_history_document_lifecycle: Some(0), + ..DRIVE_ABCI_VALIDATION_VERSIONS_V10 + .state_transitions + .batch_state_transition + }, + ..DRIVE_ABCI_VALIDATION_VERSIONS_V10.state_transitions + }, + ..DRIVE_ABCI_VALIDATION_VERSIONS_V10 + }; diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v2.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v2.rs index c395ded0e36..46edb3178c1 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v2.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v2.rs @@ -127,6 +127,7 @@ pub const DRIVE_ABCI_VALIDATION_VERSIONS_V2: DriveAbciValidationVersions = document_create_transition_structure_validation: 0, document_delete_transition_structure_validation: 0, document_index_only_delete_transition_structure_validation: 0, + document_erase_transition_structure_validation: None, document_replace_transition_structure_validation: 0, document_transfer_transition_structure_validation: 0, document_purchase_transition_structure_validation: 0, @@ -135,6 +136,8 @@ pub const DRIVE_ABCI_VALIDATION_VERSIONS_V2: DriveAbciValidationVersions = document_create_transition_state_validation: 1, document_delete_transition_state_validation: 0, document_index_only_delete_transition_state_validation: 0, + document_erase_transition_state_validation: None, + fetch_keep_history_document_lifecycle: None, document_replace_transition_state_validation: 0, document_transfer_transition_state_validation: 0, document_purchase_transition_state_validation: 0, diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v3.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v3.rs index 41d57f57298..95ef1fcccda 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v3.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v3.rs @@ -127,6 +127,7 @@ pub const DRIVE_ABCI_VALIDATION_VERSIONS_V3: DriveAbciValidationVersions = document_create_transition_structure_validation: 0, document_delete_transition_structure_validation: 0, document_index_only_delete_transition_structure_validation: 0, + document_erase_transition_structure_validation: None, document_replace_transition_structure_validation: 0, document_transfer_transition_structure_validation: 0, document_purchase_transition_structure_validation: 0, @@ -135,6 +136,8 @@ pub const DRIVE_ABCI_VALIDATION_VERSIONS_V3: DriveAbciValidationVersions = document_create_transition_state_validation: 1, document_delete_transition_state_validation: 0, document_index_only_delete_transition_state_validation: 0, + document_erase_transition_state_validation: None, + fetch_keep_history_document_lifecycle: None, document_replace_transition_state_validation: 0, document_transfer_transition_state_validation: 0, document_purchase_transition_state_validation: 0, diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v4.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v4.rs index 528f2d3594c..8f962d14a87 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v4.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v4.rs @@ -130,6 +130,7 @@ pub const DRIVE_ABCI_VALIDATION_VERSIONS_V4: DriveAbciValidationVersions = document_create_transition_structure_validation: 0, document_delete_transition_structure_validation: 0, document_index_only_delete_transition_structure_validation: 0, + document_erase_transition_structure_validation: None, document_replace_transition_structure_validation: 0, document_transfer_transition_structure_validation: 0, document_purchase_transition_structure_validation: 0, @@ -138,6 +139,8 @@ pub const DRIVE_ABCI_VALIDATION_VERSIONS_V4: DriveAbciValidationVersions = document_create_transition_state_validation: 1, document_delete_transition_state_validation: 0, document_index_only_delete_transition_state_validation: 0, + document_erase_transition_state_validation: None, + fetch_keep_history_document_lifecycle: None, document_replace_transition_state_validation: 0, document_transfer_transition_state_validation: 0, document_purchase_transition_state_validation: 0, diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v5.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v5.rs index 4547fed686b..dcbf2db5503 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v5.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v5.rs @@ -131,6 +131,7 @@ pub const DRIVE_ABCI_VALIDATION_VERSIONS_V5: DriveAbciValidationVersions = document_create_transition_structure_validation: 0, document_delete_transition_structure_validation: 0, document_index_only_delete_transition_structure_validation: 0, + document_erase_transition_structure_validation: None, document_replace_transition_structure_validation: 0, document_transfer_transition_structure_validation: 0, document_purchase_transition_structure_validation: 0, @@ -139,6 +140,8 @@ pub const DRIVE_ABCI_VALIDATION_VERSIONS_V5: DriveAbciValidationVersions = document_create_transition_state_validation: 1, document_delete_transition_state_validation: 0, document_index_only_delete_transition_state_validation: 0, + document_erase_transition_state_validation: None, + fetch_keep_history_document_lifecycle: None, document_replace_transition_state_validation: 0, document_transfer_transition_state_validation: 0, document_purchase_transition_state_validation: 0, diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v6.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v6.rs index f0e7276bc20..33ead66ce64 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v6.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v6.rs @@ -134,6 +134,7 @@ pub const DRIVE_ABCI_VALIDATION_VERSIONS_V6: DriveAbciValidationVersions = document_create_transition_structure_validation: 0, document_delete_transition_structure_validation: 0, document_index_only_delete_transition_structure_validation: 0, + document_erase_transition_structure_validation: None, document_replace_transition_structure_validation: 0, document_transfer_transition_structure_validation: 0, document_purchase_transition_structure_validation: 0, @@ -142,6 +143,8 @@ pub const DRIVE_ABCI_VALIDATION_VERSIONS_V6: DriveAbciValidationVersions = document_create_transition_state_validation: 1, document_delete_transition_state_validation: 0, document_index_only_delete_transition_state_validation: 0, + document_erase_transition_state_validation: None, + fetch_keep_history_document_lifecycle: None, document_replace_transition_state_validation: 0, document_transfer_transition_state_validation: 0, document_purchase_transition_state_validation: 0, diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v7.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v7.rs index 67017f621d1..c9befa9af48 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v7.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v7.rs @@ -128,6 +128,7 @@ pub const DRIVE_ABCI_VALIDATION_VERSIONS_V7: DriveAbciValidationVersions = document_create_transition_structure_validation: 0, document_delete_transition_structure_validation: 0, document_index_only_delete_transition_structure_validation: 0, + document_erase_transition_structure_validation: None, document_replace_transition_structure_validation: 0, document_transfer_transition_structure_validation: 0, document_purchase_transition_structure_validation: 0, @@ -136,6 +137,8 @@ pub const DRIVE_ABCI_VALIDATION_VERSIONS_V7: DriveAbciValidationVersions = document_create_transition_state_validation: 1, document_delete_transition_state_validation: 0, document_index_only_delete_transition_state_validation: 0, + document_erase_transition_state_validation: None, + fetch_keep_history_document_lifecycle: None, document_replace_transition_state_validation: 0, document_transfer_transition_state_validation: 0, document_purchase_transition_state_validation: 0, diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v8.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v8.rs index 01f465dac4c..83a0b29efc2 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v8.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v8.rs @@ -182,6 +182,7 @@ pub const DRIVE_ABCI_VALIDATION_VERSIONS_V8: DriveAbciValidationVersions = document_create_transition_structure_validation: 0, document_delete_transition_structure_validation: 0, document_index_only_delete_transition_structure_validation: 0, + document_erase_transition_structure_validation: None, document_replace_transition_structure_validation: 0, document_transfer_transition_structure_validation: 0, document_purchase_transition_structure_validation: 0, @@ -190,6 +191,8 @@ pub const DRIVE_ABCI_VALIDATION_VERSIONS_V8: DriveAbciValidationVersions = document_create_transition_state_validation: 1, document_delete_transition_state_validation: 0, document_index_only_delete_transition_state_validation: 0, + document_erase_transition_state_validation: None, + fetch_keep_history_document_lifecycle: None, document_replace_transition_state_validation: 0, document_transfer_transition_state_validation: 0, document_purchase_transition_state_validation: 0, diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v9.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v9.rs index df2c16c4f4b..1de62df75b3 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v9.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v9.rs @@ -178,6 +178,7 @@ pub const DRIVE_ABCI_VALIDATION_VERSIONS_V9: DriveAbciValidationVersions = document_create_transition_structure_validation: 0, document_delete_transition_structure_validation: 0, document_index_only_delete_transition_structure_validation: 0, + document_erase_transition_structure_validation: None, document_replace_transition_structure_validation: 0, document_transfer_transition_structure_validation: 0, document_purchase_transition_structure_validation: 0, @@ -186,6 +187,8 @@ pub const DRIVE_ABCI_VALIDATION_VERSIONS_V9: DriveAbciValidationVersions = document_create_transition_state_validation: 1, document_delete_transition_state_validation: 0, document_index_only_delete_transition_state_validation: 0, + document_erase_transition_state_validation: None, + fetch_keep_history_document_lifecycle: None, document_replace_transition_state_validation: 0, document_transfer_transition_state_validation: 0, document_purchase_transition_state_validation: 0, diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/mod.rs b/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/mod.rs index 7dd700ef08d..68f5d935f6a 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/mod.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/mod.rs @@ -1,4 +1,4 @@ -use versioned_feature_core::FeatureVersion; +use versioned_feature_core::{FeatureVersion, OptionalFeatureVersion}; pub mod v1; pub mod v2; @@ -38,6 +38,10 @@ pub struct DriveDocumentQueryMethodVersions { /// Version 1 selects the protocol-15 keep-history layout, where the /// primary-key entry is the current document rather than a history tree. pub primary_key_path_query: FeatureVersion, + /// Reads the lifecycle record of one keep-history document and, when the + /// record says the document is deleted, its newest retained revision. + /// Absent before protocol version 15, which introduces the lifecycle tree. + pub fetch_document_lifecycle: OptionalFeatureVersion, /// Mode-detection routing table for `SELECT COUNT` queries. /// Versioned because the routing table is consensus-relevant on /// the query surface — a future protocol version that changes @@ -155,6 +159,17 @@ pub struct DriveDocumentDeleteMethodVersions { /// The fee-applying indexOnly deletion wrapper (dormant slot, 0 in /// every table; only reachable for indexOnly document types). pub delete_index_only_document_for_contract: FeatureVersion, + /// Removes a bounded chunk of the retained revisions of a deleted + /// keep-history document, dropping the history subtree and the lifecycle + /// record with the terminal chunk. Absent before protocol version 15. + pub erase_document_for_contract_operations: OptionalFeatureVersion, + /// Estimation layers for the erase chunk. Same dormancy as + /// `erase_document_for_contract_operations`. + pub add_estimation_costs_for_erase_document: OptionalFeatureVersion, + /// Estimation layers for the lifecycle record a delete writes and an + /// erase removes, shared by both. Same dormancy as + /// `erase_document_for_contract_operations`. + pub add_estimation_costs_for_lifecycle_record: OptionalFeatureVersion, } #[derive(Clone, Debug, Default)] diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v1.rs b/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v1.rs index cea6d64165c..c096be8aa91 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v1.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v1.rs @@ -18,6 +18,7 @@ pub const DRIVE_DOCUMENT_METHOD_VERSIONS_V1: DriveDocumentMethodVersions = fetch_document_history: 0, prove_document_history: 0, primary_key_path_query: 0, + fetch_document_lifecycle: None, detect_count_mode: 0, detect_sum_mode: 0, detect_ranked_mode: 0, @@ -41,6 +42,9 @@ pub const DRIVE_DOCUMENT_METHOD_VERSIONS_V1: DriveDocumentMethodVersions = delete_document_for_contract_operations: 0, delete_index_only_document_for_contract_operations: 0, delete_index_only_document_for_contract: 0, + erase_document_for_contract_operations: None, + add_estimation_costs_for_erase_document: None, + add_estimation_costs_for_lifecycle_record: None, }, insert: DriveDocumentInsertMethodVersions { add_document: 0, diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v2.rs b/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v2.rs index cd87455ef9c..8dadedddfae 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v2.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v2.rs @@ -20,6 +20,7 @@ pub const DRIVE_DOCUMENT_METHOD_VERSIONS_V2: DriveDocumentMethodVersions = fetch_document_history: 0, prove_document_history: 0, primary_key_path_query: 0, + fetch_document_lifecycle: None, detect_count_mode: 0, detect_sum_mode: 0, detect_ranked_mode: 0, @@ -43,6 +44,9 @@ pub const DRIVE_DOCUMENT_METHOD_VERSIONS_V2: DriveDocumentMethodVersions = delete_document_for_contract_operations: 0, delete_index_only_document_for_contract_operations: 0, delete_index_only_document_for_contract: 0, + erase_document_for_contract_operations: None, + add_estimation_costs_for_erase_document: None, + add_estimation_costs_for_lifecycle_record: None, }, insert: DriveDocumentInsertMethodVersions { add_document: 0, diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v3.rs b/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v3.rs index d8d3cf07b1a..00dd6076149 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v3.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v3.rs @@ -30,6 +30,7 @@ pub const DRIVE_DOCUMENT_METHOD_VERSIONS_V3: DriveDocumentMethodVersions = fetch_document_history: 0, prove_document_history: 0, primary_key_path_query: 0, + fetch_document_lifecycle: None, detect_count_mode: 0, detect_sum_mode: 0, detect_ranked_mode: 0, @@ -58,6 +59,9 @@ pub const DRIVE_DOCUMENT_METHOD_VERSIONS_V3: DriveDocumentMethodVersions = delete_document_for_contract_operations: 0, delete_index_only_document_for_contract_operations: 0, delete_index_only_document_for_contract: 0, + erase_document_for_contract_operations: None, + add_estimation_costs_for_erase_document: None, + add_estimation_costs_for_lifecycle_record: None, }, insert: DriveDocumentInsertMethodVersions { add_document: 0, diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v4.rs b/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v4.rs index b4404c89b60..9d448c789ab 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v4.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v4.rs @@ -82,6 +82,7 @@ pub const DRIVE_DOCUMENT_METHOD_VERSIONS_V4: DriveDocumentMethodVersions = fetch_document_history: 0, prove_document_history: 0, primary_key_path_query: 0, + fetch_document_lifecycle: None, detect_count_mode: 0, detect_sum_mode: 0, detect_ranked_mode: 0, @@ -108,6 +109,9 @@ pub const DRIVE_DOCUMENT_METHOD_VERSIONS_V4: DriveDocumentMethodVersions = delete_document_for_contract_operations: 0, delete_index_only_document_for_contract_operations: 0, delete_index_only_document_for_contract: 0, + erase_document_for_contract_operations: None, + add_estimation_costs_for_erase_document: None, + add_estimation_costs_for_lifecycle_record: None, }, insert: DriveDocumentInsertMethodVersions { add_document: 0, diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v5.rs b/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v5.rs index faec462bd69..66c4e384550 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v5.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_document_method_versions/v5.rs @@ -5,7 +5,15 @@ use super::{ DriveDocumentQueryMethodVersions, DriveDocumentUpdateMethodVersions, }; -/// Protocol v15 document methods for per-type keep-history storage. +/// Protocol v15 document methods for per-type keep-history storage and the +/// delete and erase lifecycle of keep-history documents. +/// +/// A delete of a keep-history document removes its current pointer and index +/// references and records a lifecycle entry while the retained revisions stay +/// readable; an erase removes those revisions a chunk at a time. The delete +/// wrappers, the primary-storage removal estimate and the lifecycle read are +/// the generations that know that layout, and the insert generation never lets +/// a caller's override write over a deleted document's retained revisions. pub const DRIVE_DOCUMENT_METHOD_VERSIONS_V5: DriveDocumentMethodVersions = DriveDocumentMethodVersions { query: DriveDocumentQueryMethodVersions { @@ -13,13 +21,21 @@ pub const DRIVE_DOCUMENT_METHOD_VERSIONS_V5: DriveDocumentMethodVersions = fetch_document_history: 1, prove_document_history: 1, primary_key_path_query: 1, + fetch_document_lifecycle: Some(0), ..DRIVE_DOCUMENT_METHOD_VERSIONS_V4.query }, delete: DriveDocumentDeleteMethodVersions { remove_reference_for_index_level_for_contract_operations: 2, + add_estimation_costs_for_remove_document_to_primary_storage: 1, + remove_document_from_primary_storage: 1, + delete_document_for_contract_operations: 1, + erase_document_for_contract_operations: Some(0), + add_estimation_costs_for_erase_document: Some(0), + add_estimation_costs_for_lifecycle_record: Some(0), ..DRIVE_DOCUMENT_METHOD_VERSIONS_V4.delete }, insert: DriveDocumentInsertMethodVersions { + add_document_for_contract_operations: 2, add_document_to_primary_storage: 1, add_reference_for_index_level_for_contract_operations: 1, ..DRIVE_DOCUMENT_METHOD_VERSIONS_V4.insert diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/mod.rs b/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/mod.rs index ed0b0326f34..ecb63864207 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/mod.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/mod.rs @@ -2,9 +2,10 @@ pub mod v1; pub mod v2; pub mod v3; pub mod v4; +pub mod v5; use crate::version::drive_versions::DriveDataContractOperationMethodVersions; -use versioned_feature_core::FeatureVersion; +use versioned_feature_core::{FeatureVersion, OptionalFeatureVersion}; #[derive(Clone, Debug, Default)] pub struct DriveStateTransitionMethodVersions { @@ -32,6 +33,8 @@ pub struct DriveStateTransitionActionConvertToHighLevelOperationsMethodVersions pub document_create_transition: FeatureVersion, pub document_delete_transition: FeatureVersion, pub document_index_only_delete_transition: FeatureVersion, + /// The erase kind, absent before protocol version 15. + pub document_erase_transition: OptionalFeatureVersion, pub document_purchase_transition: FeatureVersion, pub document_replace_transition: FeatureVersion, pub document_transfer_transition: FeatureVersion, diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v1.rs b/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v1.rs index 7955a70b8cc..da4b5aef71f 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v1.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v1.rs @@ -20,6 +20,7 @@ pub const DRIVE_STATE_TRANSITION_METHOD_VERSIONS_V1: DriveStateTransitionMethodV document_create_transition: 0, document_delete_transition: 0, document_index_only_delete_transition: 0, + document_erase_transition: None, document_purchase_transition: 0, document_replace_transition: 0, document_transfer_transition: 0, diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v2.rs b/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v2.rs index 8a34eebaf0d..83d23cf2080 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v2.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v2.rs @@ -21,6 +21,7 @@ pub const DRIVE_STATE_TRANSITION_METHOD_VERSIONS_V2: DriveStateTransitionMethodV document_create_transition: 0, document_delete_transition: 0, document_index_only_delete_transition: 0, + document_erase_transition: None, document_purchase_transition: 0, document_replace_transition: 0, document_transfer_transition: 0, diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v3.rs b/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v3.rs index 78288f4334d..12fc5663a3c 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v3.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v3.rs @@ -21,6 +21,7 @@ pub const DRIVE_STATE_TRANSITION_METHOD_VERSIONS_V3: DriveStateTransitionMethodV document_create_transition: 0, document_delete_transition: 0, document_index_only_delete_transition: 0, + document_erase_transition: None, // PROTOCOL_VERSION_13: v1 rewrites a transferred or purchased // DPNS domain document's `records.identity` to the new owner // so the username resolves to the buyer. v0 stays for diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v4.rs b/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v4.rs index c60cb68582a..547d6d450d7 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v4.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v4.rs @@ -21,6 +21,7 @@ pub const DRIVE_STATE_TRANSITION_METHOD_VERSIONS_V4: DriveStateTransitionMethodV document_create_transition: 0, document_delete_transition: 0, document_index_only_delete_transition: 0, + document_erase_transition: None, // PROTOCOL_VERSION_13: v1 rewrites a transferred or purchased // DPNS domain document's `records.identity` to the new owner // so the username resolves to the buyer. v0 stays for diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v5.rs b/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v5.rs new file mode 100644 index 00000000000..106066efa67 --- /dev/null +++ b/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v5.rs @@ -0,0 +1,22 @@ +use crate::version::drive_versions::drive_state_transition_method_versions::v4::DRIVE_STATE_TRANSITION_METHOD_VERSIONS_V4; +use crate::version::drive_versions::drive_state_transition_method_versions::{ + DriveStateTransitionActionConvertToHighLevelOperationsMethodVersions, + DriveStateTransitionMethodVersions, +}; + +/// Protocol v15 state transition methods for the keep-history document +/// lifecycle. +/// +/// Delete conversion generation 1 records a lifecycle entry for a keep-history +/// document instead of removing its retained revisions; erase conversion +/// removes them a chunk at a time. +pub const DRIVE_STATE_TRANSITION_METHOD_VERSIONS_V5: DriveStateTransitionMethodVersions = + DriveStateTransitionMethodVersions { + convert_to_high_level_operations: + DriveStateTransitionActionConvertToHighLevelOperationsMethodVersions { + document_delete_transition: 1, + document_erase_transition: Some(0), + ..DRIVE_STATE_TRANSITION_METHOD_VERSIONS_V4.convert_to_high_level_operations + }, + ..DRIVE_STATE_TRANSITION_METHOD_VERSIONS_V4 + }; diff --git a/packages/rs-platform-version/src/version/drive_versions/v10.rs b/packages/rs-platform-version/src/version/drive_versions/v10.rs index 9784199401f..2a11156336d 100644 --- a/packages/rs-platform-version/src/version/drive_versions/v10.rs +++ b/packages/rs-platform-version/src/version/drive_versions/v10.rs @@ -1,5 +1,6 @@ use super::drive_contract_method_versions::v5::DRIVE_CONTRACT_METHOD_VERSIONS_V5; use super::drive_document_method_versions::v5::DRIVE_DOCUMENT_METHOD_VERSIONS_V5; +use super::drive_state_transition_method_versions::v5::DRIVE_STATE_TRANSITION_METHOD_VERSIONS_V5; use super::drive_verify_method_versions::v3::DRIVE_VERIFY_METHOD_VERSIONS_V3; use super::v9::DRIVE_VERSION_V9; use super::{DriveMethodVersions, DriveVersion}; @@ -8,11 +9,15 @@ use super::{DriveMethodVersions, DriveVersion}; /// /// Moves keep-history documents to per-type history trees and selects the /// matching contract writers, document operations, queries, and proof verifier. +/// It also selects the delete and erase lifecycle of keep-history documents: +/// the delete wrappers and action conversion that record and consume a +/// lifecycle entry, and the erase conversion that removes retained revisions. pub const DRIVE_VERSION_V10: DriveVersion = DriveVersion { methods: DriveMethodVersions { document: DRIVE_DOCUMENT_METHOD_VERSIONS_V5, contract: DRIVE_CONTRACT_METHOD_VERSIONS_V5, verify: DRIVE_VERIFY_METHOD_VERSIONS_V3, + state_transitions: DRIVE_STATE_TRANSITION_METHOD_VERSIONS_V5, ..DRIVE_VERSION_V9.methods }, ..DRIVE_VERSION_V9 diff --git a/packages/rs-platform-version/src/version/mocks/v2_test.rs b/packages/rs-platform-version/src/version/mocks/v2_test.rs index 49e0cb4048f..44a45107bc1 100644 --- a/packages/rs-platform-version/src/version/mocks/v2_test.rs +++ b/packages/rs-platform-version/src/version/mocks/v2_test.rs @@ -534,6 +534,7 @@ pub const TEST_PLATFORM_V2: PlatformVersion = PlatformVersion { max_time_range_ttl_seconds: None, min_time_range_ttl_drop_operations_per_write: None, minimum_grovedb_proof_envelope_version: 0, + max_document_revisions_erased_per_transition: None, }, consensus: ConsensusVersions { tenderdash_consensus_version: 0, diff --git a/packages/rs-platform-version/src/version/system_limits/mod.rs b/packages/rs-platform-version/src/version/system_limits/mod.rs index a1f83892a36..7d75630d4fa 100644 --- a/packages/rs-platform-version/src/version/system_limits/mod.rs +++ b/packages/rs-platform-version/src/version/system_limits/mod.rs @@ -2,6 +2,7 @@ pub mod v1; pub mod v2; pub mod v3; pub mod v4; +pub mod v5; #[derive(Clone, Debug, Default)] pub struct SystemLimits { @@ -142,6 +143,23 @@ pub struct SystemLimits { /// version 3 (protocol version 13), so every live network already serves /// V1 by the time the floor applies. pub minimum_grovedb_proof_envelope_version: u32, + /// Maximum number of retained revisions one erase transition may remove + /// from a deleted keep-history document. + /// + /// An erase is chunked so that a single transition can never expand into + /// unbounded work: the chunk is bounded here, and the admission estimate + /// every erase must have in balance is sized for a full chunk whatever the + /// document's actual history length. A document with more retained + /// revisions than this stays in the erasing state and is finished by + /// further erase transitions, which any identity may submit. + /// + /// The bound applies per transition, not per block, so raising it raises + /// the work a maximum legal block of erases can demand as well as the + /// balance every single erase needs up front. + /// + /// `None` preserves the behavior of protocol versions that predate the + /// document lifecycle, where no erase transition can exist. + pub max_document_revisions_erased_per_transition: Option, } #[cfg(test)] diff --git a/packages/rs-platform-version/src/version/system_limits/v1.rs b/packages/rs-platform-version/src/version/system_limits/v1.rs index 63474654244..08323b579a5 100644 --- a/packages/rs-platform-version/src/version/system_limits/v1.rs +++ b/packages/rs-platform-version/src/version/system_limits/v1.rs @@ -54,4 +54,5 @@ pub const SYSTEM_LIMITS_V1: SystemLimits = SystemLimits { max_time_range_ttl_seconds: None, min_time_range_ttl_drop_operations_per_write: None, minimum_grovedb_proof_envelope_version: 0, // V0 envelopes stay accepted until v14 + max_document_revisions_erased_per_transition: None, }; diff --git a/packages/rs-platform-version/src/version/system_limits/v2.rs b/packages/rs-platform-version/src/version/system_limits/v2.rs index b83e8c951f1..1cec8aa2e01 100644 --- a/packages/rs-platform-version/src/version/system_limits/v2.rs +++ b/packages/rs-platform-version/src/version/system_limits/v2.rs @@ -35,4 +35,5 @@ pub const SYSTEM_LIMITS_V2: SystemLimits = SystemLimits { max_time_range_ttl_seconds: None, min_time_range_ttl_drop_operations_per_write: None, minimum_grovedb_proof_envelope_version: 0, // V0 envelopes stay accepted until v14 + max_document_revisions_erased_per_transition: None, }; diff --git a/packages/rs-platform-version/src/version/system_limits/v3.rs b/packages/rs-platform-version/src/version/system_limits/v3.rs index 6a78c885809..2f42b9162de 100644 --- a/packages/rs-platform-version/src/version/system_limits/v3.rs +++ b/packages/rs-platform-version/src/version/system_limits/v3.rs @@ -37,4 +37,5 @@ pub const SYSTEM_LIMITS_V3: SystemLimits = SystemLimits { max_time_range_ttl_seconds: None, min_time_range_ttl_drop_operations_per_write: None, minimum_grovedb_proof_envelope_version: 0, // V0 envelopes stay accepted until v14 + max_document_revisions_erased_per_transition: None, }; diff --git a/packages/rs-platform-version/src/version/system_limits/v4.rs b/packages/rs-platform-version/src/version/system_limits/v4.rs index 6e1083f2806..5f4afc741af 100644 --- a/packages/rs-platform-version/src/version/system_limits/v4.rs +++ b/packages/rs-platform-version/src/version/system_limits/v4.rs @@ -66,4 +66,5 @@ pub const SYSTEM_LIMITS_V4: SystemLimits = SystemLimits { max_time_range_ttl_seconds: Some(604_800), // one week min_time_range_ttl_drop_operations_per_write: Some(32), minimum_grovedb_proof_envelope_version: 1, // clients reject legacy V0 GroveDB proof envelopes from v14 + max_document_revisions_erased_per_transition: None, }; diff --git a/packages/rs-platform-version/src/version/system_limits/v5.rs b/packages/rs-platform-version/src/version/system_limits/v5.rs new file mode 100644 index 00000000000..f8f0350f723 --- /dev/null +++ b/packages/rs-platform-version/src/version/system_limits/v5.rs @@ -0,0 +1,11 @@ +use crate::version::system_limits::SystemLimits; + +/// System limits for protocol version 15 and above. Relative to V4 this adds +/// the erase chunk: `max_document_revisions_erased_per_transition` bounds how +/// many retained revisions of a deleted keep-history document one erase +/// transition removes, so the work of erasing a long history is spread over +/// several transitions of bounded cost. +pub const SYSTEM_LIMITS_V5: SystemLimits = SystemLimits { + max_document_revisions_erased_per_transition: Some(100), + ..super::v4::SYSTEM_LIMITS_V4 +}; diff --git a/packages/rs-platform-version/src/version/v15.rs b/packages/rs-platform-version/src/version/v15.rs index 787228facc9..5581c4ae167 100644 --- a/packages/rs-platform-version/src/version/v15.rs +++ b/packages/rs-platform-version/src/version/v15.rs @@ -1,25 +1,61 @@ +use crate::version::dpp_versions::dpp_contract_versions::v7::CONTRACT_VERSIONS_V7; +use crate::version::dpp_versions::dpp_state_transition_serialization_versions::v4::STATE_TRANSITION_SERIALIZATION_VERSIONS_V4; +use crate::version::dpp_versions::dpp_validation_versions::v6::DPP_VALIDATION_VERSIONS_V6; +use crate::version::dpp_versions::DPPVersion; use crate::version::drive_abci_versions::drive_abci_method_versions::v11::DRIVE_ABCI_METHOD_VERSIONS_V11; +use crate::version::drive_abci_versions::drive_abci_validation_versions::v11::DRIVE_ABCI_VALIDATION_VERSIONS_V11; use crate::version::drive_abci_versions::DriveAbciVersion; use crate::version::drive_versions::v10::DRIVE_VERSION_V10; use crate::version::protocol_version::PlatformVersion; +use crate::version::system_limits::v5::SYSTEM_LIMITS_V5; use crate::version::v14::PLATFORM_V14; use crate::version::ProtocolVersion; pub const PROTOCOL_VERSION_15: ProtocolVersion = 15; -/// Protocol v15 moves keep-history documents to per-type history trees. +/// Protocol v15 moves keep-history documents to per-type history trees and +/// gives them a delete and erase lifecycle. /// -/// The primary-key entry stores the current document while retained revisions -/// live in a separate provable count tree keyed by block time and revision. -/// The first v15 block migrates existing v14 histories and index references -/// before v15 state transitions execute. History queries and proofs select the -/// matching layout through Drive's version tables. +/// 1. **Per-type history trees.** The primary-key entry stores the current +/// document while retained revisions live in a separate provable count tree +/// keyed by block time and revision. The first v15 block migrates existing +/// v14 histories and index references before v15 state transitions execute. +/// History queries and proofs select the matching layout through Drive's +/// version tables. +/// 2. **Keep-history delete.** A keep-history document type may now allow +/// deletion. A delete removes the document's current pointer and index +/// references and records a lifecycle entry; the retained revisions stay +/// readable through the history query, which reports the deleted state and +/// its time. Creating a document under a deleted id is refused until its +/// history is gone. +/// 3. **Erase.** A document type that keeps history and can be deleted may +/// declare `canBeErased` (meta-schema v4, parser generation 4; the keyword +/// is immutable across contract updates). An erase transition removes the +/// retained revisions of a deleted document a chunk of +/// `max_document_revisions_erased_per_transition` at a time: the owner +/// authorizes the first chunk, any identity may submit a continuation, and +/// the last chunk removes the lifecycle entry, after which the id reads as +/// absent. The history query reports the erasing state, its start time and +/// the remaining revision count. +/// 4. **Wire.** The erase kind is appended to the batch's document transition +/// enum, so a batch of either shipped wire format may carry one. The batch +/// basic-structure gate admits the kind only where this protocol version +/// publishes bounds for it, which keeps new software agreeing with software +/// that cannot decode the kind while an earlier version is still active. pub const PLATFORM_V15: PlatformVersion = PlatformVersion { protocol_version: PROTOCOL_VERSION_15, - drive: DRIVE_VERSION_V10, // changed: document v5 and contract v5 write the per-type history tree; verify v3 authenticates history pages and lifecycle counts + drive: DRIVE_VERSION_V10, // changed: document v5 and contract v5 write the per-type history tree and the delete/erase lifecycle; verify v3 authenticates history pages and lifecycle counts; state transition methods v5 convert the lifecycle delete and the erase drive_abci: DriveAbciVersion { methods: DRIVE_ABCI_METHOD_VERSIONS_V11, // changed: the protocol-change hook v2 migrates retained histories on the first v15 block + validation_and_processing: DRIVE_ABCI_VALIDATION_VERSIONS_V11, // changed: delete consults the lifecycle; erase structure and state validation ..PLATFORM_V14.drive_abci }, + dpp: DPPVersion { + validation: DPP_VALIDATION_VERSIONS_V6, // changed: document type update validation v2 keeps `canBeErased` immutable + state_transition_serialization_versions: STATE_TRANSITION_SERIALIZATION_VERSIONS_V4, // changed: the erase kind joins the wire + contract_versions: CONTRACT_VERSIONS_V7, // changed: v4 document meta-schema and parser generation 4 admit deletable keep-history types and `canBeErased` + ..PLATFORM_V14.dpp + }, + system_limits: SYSTEM_LIMITS_V5, // changed: erase chunk of 100 revisions per transition ..PLATFORM_V14 }; diff --git a/packages/rs-platform-version/tests/keep_history_delete_versions.rs b/packages/rs-platform-version/tests/keep_history_delete_versions.rs index 81491b87f87..867581b70e1 100644 --- a/packages/rs-platform-version/tests/keep_history_delete_versions.rs +++ b/packages/rs-platform-version/tests/keep_history_delete_versions.rs @@ -1,9 +1,20 @@ +//! Semantic pins for the version slots the keep-history document lifecycle +//! rides on. +//! +//! A numeric pin alone would not say what the number means, so each assertion +//! below names the behaviour the slot selects and the released protocol +//! versions it must leave alone. + use platform_version::version::PlatformVersion; +/// The released protocol versions parse contracts with a generation that +/// predates the lifecycle grammar and refuse a keep-history delete before it +/// reaches storage. Both must replay identically forever. #[test] fn should_preserve_released_keep_history_validation_versions() { - for protocol in [12, 13] { + for protocol in [12, 13, 14] { let version = PlatformVersion::get(protocol).unwrap(); + let released_parser_generation = if protocol == 14 { 3 } else { 2 }; assert_eq!( version .dpp @@ -11,9 +22,13 @@ fn should_preserve_released_keep_history_validation_versions() { .document_type_versions .class_method_versions .try_from_schema, - 2, + released_parser_generation, "parser at protocol {protocol}" ); + // Protocol 14 refuses a delete of a keep-history document as a paid + // consensus error instead of the earlier internal error; neither + // generation carries one out. + let released_delete_structure_generation = if protocol == 14 { 1 } else { 0 }; assert_eq!( version .drive_abci @@ -21,15 +36,138 @@ fn should_preserve_released_keep_history_validation_versions() { .state_transitions .batch_state_transition .document_delete_transition_structure_validation, + released_delete_structure_generation, + "delete structure validation at protocol {protocol}" + ); + assert_eq!( + version + .drive_abci + .validation_and_processing + .state_transitions + .batch_state_transition + .document_delete_transition_state_validation, 0, - "delete validation at protocol {protocol}" + "delete state validation at protocol {protocol}" + ); + assert!( + version + .dpp + .state_transition_serialization_versions + .document_erase_state_transition + .is_none(), + "the erase kind must not exist on the wire at protocol {protocol}" + ); + assert!( + version + .system_limits + .max_document_revisions_erased_per_transition + .is_none(), + "no erase can exist at protocol {protocol}, so no chunk bounds one" + ); + assert_eq!( + version + .drive + .methods + .document + .delete + .delete_document_for_contract_operations, + 0, + "the keep-history delete branch must not be selected at protocol {protocol}" + ); + let released_insert_generation = if protocol == 14 { 1 } else { 0 }; + assert_eq!( + version + .drive + .methods + .document + .insert + .add_document_for_contract_operations, + released_insert_generation, + "document insert at protocol {protocol}" + ); + assert_eq!( + version + .drive + .methods + .state_transitions + .convert_to_high_level_operations + .document_delete_transition, + 0, + "delete action conversion at protocol {protocol}" + ); + // The batch-level slots the released versions replay with. Pinned + // literally so that a change to both protocol 14 and 15 at once still + // fails the protocol 15 comparison test below. + let batch_bounds = &version + .dpp + .state_transition_serialization_versions + .batch_state_transition; + assert_eq!( + ( + batch_bounds.min_version, + batch_bounds.max_version, + batch_bounds.default_current_version + ), + (0, 1, 1), + "batch wire bounds at protocol {protocol}" + ); + assert_eq!( + version + .dpp + .state_transitions + .documents + .documents_batch_transition + .validation + .validate_base_structure, + 0, + "batch structure validation at protocol {protocol}" + ); + let batch = &version + .drive_abci + .validation_and_processing + .state_transitions + .batch_state_transition; + assert_eq!( + ( + batch.basic_structure, + batch.advanced_structure, + batch.state, + batch.revision, + batch.transform_into_action, + batch.failed_per_transition_action, + batch.fetch_documents_for_transitions_knowing_contract_and_document_type, + batch.fetch_document_with_id, + batch.is_allowed, + batch.document_reference_validation, + batch.document_base_transition_state_validation, + ), + (0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0), + "batch structure, state, nonce, transform, failure, fetch and admission \ + generations at protocol {protocol}" + ); + assert_eq!( + version.drive.methods.prove.prove_state_transition, 0, + "state transition prover at protocol {protocol}" + ); + assert_eq!( + version + .drive + .methods + .verify + .state_transition + .verify_state_transition_was_executed_with_proof, + 0, + "state transition proof verification at protocol {protocol}" ); } } +/// Protocol 15 selects the lifecycle: the parser generation that admits the new +/// keywords, the delete that consults the lifecycle, and the erase kind with +/// the bound on how much one transition may remove. #[test] -fn should_activate_keep_history_validation_at_protocol_14() { - let version = PlatformVersion::get(14).unwrap(); +fn should_activate_keep_history_validation_at_protocol_15() { + let version = PlatformVersion::get(15).unwrap(); assert_eq!( version .dpp @@ -37,7 +175,7 @@ fn should_activate_keep_history_validation_at_protocol_14() { .document_type_versions .class_method_versions .try_from_schema, - 3 + 4 ); assert_eq!( version @@ -46,6 +184,276 @@ fn should_activate_keep_history_validation_at_protocol_14() { .state_transitions .batch_state_transition .document_delete_transition_structure_validation, + 2 + ); + assert_eq!( + version + .drive_abci + .validation_and_processing + .state_transitions + .batch_state_transition + .document_delete_transition_state_validation, 1 ); + assert_eq!( + version + .drive_abci + .validation_and_processing + .state_transitions + .batch_state_transition + .document_create_transition_state_validation, + 3, + "protocol 15 refuses to create over a deleted or erasing keep-history document" + ); + assert_eq!( + version + .drive + .methods + .document + .delete + .delete_document_for_contract_operations, + 1, + "protocol 15 selects the keep-history delete branch" + ); + assert_eq!( + version + .drive + .methods + .document + .insert + .add_document_for_contract_operations, + 2, + "protocol 15 selects the insert that never writes over retained revisions" + ); + assert_eq!( + version + .drive + .methods + .state_transitions + .convert_to_high_level_operations + .document_delete_transition, + 1, + "protocol 15 selects the lifecycle-specific delete operation" + ); + + let bounds = version + .dpp + .state_transition_serialization_versions + .document_erase_state_transition + .as_ref() + .expect("the erase kind joins the wire at protocol 15"); + assert_eq!(bounds.bounds.min_version, 0); + assert_eq!(bounds.bounds.max_version, 0); + assert_eq!(bounds.bounds.default_current_version, 0); + + let chunk = version + .system_limits + .max_document_revisions_erased_per_transition + .expect("protocol 15 bounds the erase chunk"); + assert_eq!( + chunk, 100, + "the chunk bounds the work one erase can demand and the balance every \ + erase needs up front; changing it changes both" + ); +} + +/// The erase kind is appended to the batch's document transition enum and +/// carried by the shipped batch wire formats, so no batch-level generation +/// changes with it: the batch wire bounds and every generation that reads a +/// batch as a whole are the same at protocol 15 as at protocol 14. Only the +/// per-kind erase slots turn on. +#[test] +fn should_carry_the_erase_kind_without_a_new_batch_generation() { + let released = PlatformVersion::get(14).unwrap(); + let current = PlatformVersion::get(15).unwrap(); + + let batch_bounds = |version: &PlatformVersion| { + let bounds = &version + .dpp + .state_transition_serialization_versions + .batch_state_transition; + ( + bounds.min_version, + bounds.max_version, + bounds.default_current_version, + ) + }; + assert_eq!( + batch_bounds(current), + batch_bounds(released), + "the batch wire formats are unchanged; the erase kind rides inside them" + ); + assert_eq!( + batch_bounds(current), + (0, 1, 1), + "the batch default wire format stays 1" + ); + + let batch_validation = |version: &PlatformVersion| { + version + .dpp + .state_transitions + .documents + .documents_batch_transition + .validation + .validate_base_structure + }; + assert_eq!( + batch_validation(current), + batch_validation(released), + "the batch basic-structure generation gates the erase kind by its bounds slot" + ); + + let batch_generations = |version: &PlatformVersion| { + let batch = &version + .drive_abci + .validation_and_processing + .state_transitions + .batch_state_transition; + ( + batch.basic_structure, + batch.transform_into_action, + batch.advanced_structure, + batch.state, + batch.revision, + batch.failed_per_transition_action, + batch.is_allowed, + batch.fetch_documents_for_transitions_knowing_contract_and_document_type, + batch.fetch_document_with_id, + batch.data_triggers.bindings, + batch.document_reference_validation, + batch.document_base_transition_state_validation, + ) + }; + assert_eq!( + batch_generations(current), + batch_generations(released), + "the batch transformer, structure, state, nonce, failure, fetch, trigger and \ + admission generations are unchanged" + ); + + let batch_drive = |version: &PlatformVersion| { + ( + version + .drive + .methods + .state_transitions + .convert_to_high_level_operations + .documents_batch_transition, + version.drive.methods.prove.prove_state_transition, + version + .drive + .methods + .verify + .state_transition + .verify_state_transition_was_executed_with_proof, + ) + }; + assert_eq!( + batch_drive(current), + batch_drive(released), + "the batch conversion, prover and execution-proof verifier are unchanged" + ); +} + +/// The lifecycle read and the erase storage operation exist at exactly one +/// version each, so no table can select an implementation that is not there. +#[test] +fn should_expose_one_implementation_of_each_new_lifecycle_slot() { + for protocol in 1..=PlatformVersion::latest().protocol_version { + let Ok(version) = PlatformVersion::get(protocol) else { + continue; + }; + let expected_erase_version = (protocol >= 15).then_some(0); + assert_eq!( + version + .drive + .methods + .document + .query + .fetch_document_lifecycle, + expected_erase_version, + "lifecycle read at protocol {protocol}" + ); + assert_eq!( + version + .drive + .methods + .document + .delete + .erase_document_for_contract_operations, + expected_erase_version, + "erase operation at protocol {protocol}" + ); + assert_eq!( + version + .drive + .methods + .document + .delete + .add_estimation_costs_for_erase_document, + expected_erase_version, + "erase estimation at protocol {protocol}" + ); + assert_eq!( + version + .drive + .methods + .document + .delete + .add_estimation_costs_for_lifecycle_record, + expected_erase_version, + "lifecycle record estimation at protocol {protocol}" + ); + assert_eq!( + version + .drive + .methods + .state_transitions + .convert_to_high_level_operations + .document_erase_transition, + expected_erase_version, + "erase action conversion at protocol {protocol}" + ); + assert_eq!( + version + .drive_abci + .validation_and_processing + .state_transitions + .batch_state_transition + .document_erase_transition_structure_validation, + expected_erase_version, + "erase structure validation at protocol {protocol}" + ); + assert_eq!( + version + .drive_abci + .validation_and_processing + .state_transitions + .batch_state_transition + .document_erase_transition_state_validation, + expected_erase_version, + "erase state validation at protocol {protocol}" + ); + assert_eq!( + version + .drive_abci + .validation_and_processing + .state_transitions + .batch_state_transition + .fetch_keep_history_document_lifecycle, + expected_erase_version, + "lifecycle state read at protocol {protocol}" + ); + assert_eq!( + version + .dpp + .state_transition_serialization_versions + .document_erase_state_transition + .as_ref() + .map(|bounds| bounds.bounds.default_current_version), + expected_erase_version, + "erase wire bounds at protocol {protocol}" + ); + } } diff --git a/packages/rs-sdk/src/mock/requests.rs b/packages/rs-sdk/src/mock/requests.rs index 1bbf0fe6c15..64beccb5b45 100644 --- a/packages/rs-sdk/src/mock/requests.rs +++ b/packages/rs-sdk/src/mock/requests.rs @@ -234,7 +234,13 @@ impl MockResponse for Document { } } -type MockDocumentHistory = (Vec<(u64, u64, Vec)>, Option<(bool, u64)>); +/// Entries as (time, revision, document bytes); the lifecycle as its state +/// discriminant, the remaining revision count and the four lifecycle times, +/// so a mocked deleted or erasing history round-trips with its metadata. +type MockDocumentHistory = ( + Vec<(u64, u64, Vec)>, + Option<(u8, u64, u64, u64, u64, u64)>, +); impl MockResponse for drive_proof_verifier::types::DocumentHistoryProofInfo { fn mock_serialize(&self, sdk: &MockDashPlatformSdk) -> Vec { @@ -277,9 +283,19 @@ impl MockResponse for drive_proof_verifier::types::DocumentHistory { }) .collect::>(); let lifecycle = self.lifecycle.as_ref().map(|lifecycle| { + let state = match lifecycle.state { + DocumentHistoryState::Active => 0u8, + DocumentHistoryState::Deleted => 1, + DocumentHistoryState::Erasing => 2, + DocumentHistoryState::Absent => 3, + }; ( - lifecycle.state == DocumentHistoryState::Active, + state, lifecycle.remaining_revisions, + lifecycle.times.deleted_at_ms, + lifecycle.times.erasing_started_at_ms, + lifecycle.times.erasing_from_time_ms, + lifecycle.times.erasing_from_revision, ) }); bincode::encode_to_vec((entries, lifecycle), BINCODE_CONFIG) @@ -288,7 +304,8 @@ impl MockResponse for drive_proof_verifier::types::DocumentHistory { fn mock_deserialize(sdk: &MockDashPlatformSdk, buf: &[u8]) -> Self { use drive_proof_verifier::types::{ - DocumentHistoryEntry, DocumentHistoryLifecycle, DocumentHistoryState, + DocumentHistoryEntry, DocumentHistoryLifecycle, DocumentHistoryLifecycleTimes, + DocumentHistoryState, }; let ((entries, lifecycle), _): (MockDocumentHistory, _) = bincode::decode_from_slice(buf, BINCODE_CONFIG).expect("decode document history"); @@ -301,18 +318,86 @@ impl MockResponse for drive_proof_verifier::types::DocumentHistory { document: Document::mock_deserialize(sdk, &bytes), }) .collect(), - lifecycle: lifecycle.map(|(active, remaining_revisions)| DocumentHistoryLifecycle { - state: if active { - DocumentHistoryState::Active - } else { - DocumentHistoryState::Absent + lifecycle: lifecycle.map( + |( + state, + remaining_revisions, + deleted_at_ms, + erasing_started_at_ms, + erasing_from_time_ms, + erasing_from_revision, + )| DocumentHistoryLifecycle { + state: match state { + 0 => DocumentHistoryState::Active, + 1 => DocumentHistoryState::Deleted, + 2 => DocumentHistoryState::Erasing, + _ => DocumentHistoryState::Absent, + }, + remaining_revisions, + times: DocumentHistoryLifecycleTimes { + deleted_at_ms, + erasing_started_at_ms, + erasing_from_time_ms, + erasing_from_revision, + }, }, - remaining_revisions, - }), + ), } } } +#[cfg(test)] +mod document_history_mock_tests { + use super::*; + use drive_proof_verifier::types::{ + DocumentHistory, DocumentHistoryLifecycle, DocumentHistoryLifecycleTimes, + DocumentHistoryState, + }; + + /// Every lifecycle state and every lifecycle time survives the mock + /// round trip, so a mocked deleted or erasing history looks to the caller + /// exactly as the verified one would. + #[test] + fn should_round_trip_every_lifecycle_state_with_its_times() { + let mut sdk = crate::SdkBuilder::default() + .with_version(dpp::version::PlatformVersion::latest()) + .build() + .unwrap(); + let mock = sdk.mock(); + let sdk: &MockDashPlatformSdk = &mock; + for (state, remaining_revisions) in [ + (DocumentHistoryState::Active, 3), + (DocumentHistoryState::Deleted, 3), + (DocumentHistoryState::Erasing, 1), + (DocumentHistoryState::Absent, 0), + ] { + let history = DocumentHistory { + entries: vec![], + lifecycle: Some(DocumentHistoryLifecycle { + state, + remaining_revisions, + times: DocumentHistoryLifecycleTimes { + deleted_at_ms: 5_000, + erasing_started_at_ms: 6_000, + erasing_from_time_ms: 4_000, + erasing_from_revision: 3, + }, + }), + }; + let recovered = DocumentHistory::mock_deserialize(sdk, &history.mock_serialize(sdk)); + assert_eq!(recovered, history, "{state:?} did not round-trip"); + } + let without_lifecycle = DocumentHistory { + entries: vec![], + lifecycle: None, + }; + assert_eq!( + DocumentHistory::mock_deserialize(sdk, &without_lifecycle.mock_serialize(sdk)), + without_lifecycle + ); + } +} + impl MockResponse for Element { fn mock_serialize(&self, _sdk: &MockDashPlatformSdk) -> Vec { // Create a bincode configuration diff --git a/packages/rs-sdk/src/platform/documents/transitions/erase.rs b/packages/rs-sdk/src/platform/documents/transitions/erase.rs new file mode 100644 index 00000000000..bb35a4a96c3 --- /dev/null +++ b/packages/rs-sdk/src/platform/documents/transitions/erase.rs @@ -0,0 +1,488 @@ +use crate::platform::transition::broadcast::BroadcastStateTransition; +use crate::platform::transition::put_settings::PutSettings; +use crate::platform::{Fetch, Identifier}; +use crate::{Error, Sdk}; +use dpp::data_contract::accessors::v0::DataContractV0Getters; +use dpp::data_contract::DataContract; +use dpp::document::{Document, INITIAL_REVISION}; +use dpp::identity::signer::Signer; +use dpp::identity::IdentityPublicKey; +use dpp::prelude::UserFeeIncrease; +use dpp::state_transition::batch_transition::methods::v0::DocumentsBatchTransitionMethodsV0; +use dpp::state_transition::batch_transition::methods::StateTransitionCreationOptions; +use dpp::state_transition::batch_transition::BatchTransition; +use dpp::state_transition::proof_result::StateTransitionProofResult; +use dpp::state_transition::StateTransition; +use dpp::version::PlatformVersion; +use dpp::ProtocolError; +use drive::query::document_history_drive_query::{DocumentHistoryFilter, DocumentHistoryLifecycle}; +use std::sync::Arc; + +/// A builder to configure and broadcast document erase transitions. +/// +/// Erasing purges the retained revisions of a document that has already been +/// deleted. The first erase must come from the document's owner and commits the +/// document to erasure; every erase after that may come from any identity, +/// because the committed state already carries the authorization. +pub struct DocumentEraseTransitionBuilder { + /// The data contract. + pub data_contract: Arc, + /// The name of the document type whose revisions are being erased. + pub document_type_name: String, + /// The document whose revisions are being erased. + pub document_id: Identifier, + /// The identity submitting and paying for this erase, which need not own + /// the document once its erasure has begun. + pub owner_id: Identifier, + /// Settings for broadcasting. + pub settings: Option, + /// A user fee increase. + pub user_fee_increase: Option, + /// State transition creation options. + pub state_transition_creation_options: Option, +} + +impl DocumentEraseTransitionBuilder { + /// Start building an erase request for the provided data contract. + /// + /// There is no token payment: the deletion this erase follows was charged + /// when the document was deleted, and an erase that carried one would let a + /// continuation, which anyone may submit, move tokens. + pub fn new( + data_contract: Arc, + document_type_name: String, + document_id: Identifier, + owner_id: Identifier, + ) -> Self { + Self { + data_contract, + document_type_name, + document_id, + owner_id, + settings: None, + user_fee_increase: None, + state_transition_creation_options: None, + } + } + + /// Adds a user fee increase to the erase transition. + pub fn with_user_fee_increase(mut self, user_fee_increase: UserFeeIncrease) -> Self { + self.user_fee_increase = Some(user_fee_increase); + self + } + + /// Adds settings to the erase transition. + pub fn with_settings(mut self, settings: PutSettings) -> Self { + self.settings = Some(settings); + self + } + + /// Adds creation options to the erase transition. + pub fn with_state_transition_creation_options( + mut self, + creation_options: StateTransitionCreationOptions, + ) -> Self { + self.state_transition_creation_options = Some(creation_options); + self + } + + /// Signs the erase transition. + pub async fn sign( + &self, + sdk: &Sdk, + identity_public_key: &IdentityPublicKey, + signer: &impl Signer, + platform_version: &PlatformVersion, + ) -> Result { + // Validate the target before the nonce fetch below bumps the SDK's + // cached contract nonce: no transition is broadcast on an error path, + // so a rejection after it would leak an increment per failed call. + let document_type = self + .data_contract + .document_type_for_name(&self.document_type_name) + .map_err(|e| Error::Protocol(e.into()))?; + let (user_fee_increase, creation_options) = self.signing_parameters(); + Self::check_erase_is_constructible(creation_options.as_ref(), platform_version)?; + + // The transition carries only the base, so an id is all the builder + // needs; the values of a document that is no longer visible are not + // available to a client anyway. + let document = Document::V0(dpp::document::DocumentV0 { + contract_version: None, + id: self.document_id, + owner_id: self.owner_id, + properties: Default::default(), + revision: Some(INITIAL_REVISION), + created_at: None, + updated_at: None, + transferred_at: None, + created_at_block_height: None, + updated_at_block_height: None, + transferred_at_block_height: None, + created_at_core_block_height: None, + updated_at_core_block_height: None, + transferred_at_core_block_height: None, + creator_id: None, + }); + + let identity_contract_nonce = sdk + .get_identity_contract_nonce( + self.owner_id, + self.data_contract.id(), + true, + self.settings, + ) + .await?; + + let state_transition = BatchTransition::new_document_erase_transition_from_document( + document, + document_type, + identity_public_key, + identity_contract_nonce, + user_fee_increase, + signer, + platform_version, + creation_options, + ) + .await?; + + Ok(state_transition) + } + + /// The fee increase and creation options the transition is signed with. + /// + /// An explicitly set value wins; otherwise the ones carried by the put + /// settings apply, so a caller that only hands over settings (the wasm and + /// FFI wrappers do) still signs with what it asked for. + fn signing_parameters(&self) -> (UserFeeIncrease, Option) { + let settings = self.settings.as_ref(); + ( + self.user_fee_increase + .or(settings.and_then(|settings| settings.user_fee_increase)) + .unwrap_or_default(), + self.state_transition_creation_options + .or(settings.and_then(|settings| settings.state_transition_creation_options)), + ) + } + + /// Refuses, before any nonce is reserved, an erase the platform version + /// cannot construct: the transition kind joined the wire at protocol + /// version 15, and the batch may only be built at a version it knows. + /// + /// The same rejection happens inside the transition constructor, but by + /// then the SDK has already advanced its cached contract nonce for a + /// transition that is never broadcast. + fn check_erase_is_constructible( + creation_options: Option<&StateTransitionCreationOptions>, + platform_version: &PlatformVersion, + ) -> Result<(), Error> { + let serialization = &platform_version.dpp.state_transition_serialization_versions; + let Some(erase_bounds) = serialization.document_erase_state_transition.as_ref() else { + return Err(Error::Protocol(ProtocolError::Generic( + "erase transitions do not exist at this platform version".to_string(), + ))); + }; + let batch_feature_version = creation_options + .and_then(|options| options.batch_feature_version) + .unwrap_or(serialization.batch_state_transition.default_current_version); + if !matches!(batch_feature_version, 0 | 1) { + return Err(Error::Protocol(ProtocolError::UnknownVersionMismatch { + method: "DocumentEraseTransitionBuilder::sign".to_string(), + known_versions: vec![0, 1], + received: batch_feature_version, + })); + } + let method_feature_version = creation_options + .and_then(|options| options.method_feature_version) + .unwrap_or(erase_bounds.bounds.default_current_version); + if method_feature_version != 0 { + return Err(Error::Protocol(ProtocolError::UnknownVersionMismatch { + method: "DocumentEraseTransitionBuilder::sign".to_string(), + known_versions: vec![0], + received: method_feature_version, + })); + } + let base_feature_version = creation_options + .and_then(|options| options.base_feature_version) + .unwrap_or( + serialization + .document_base_state_transition + .default_current_version, + ); + if !matches!(base_feature_version, 0 | 1) { + return Err(Error::Protocol(ProtocolError::UnknownVersionMismatch { + method: "DocumentEraseTransitionBuilder::sign".to_string(), + known_versions: vec![0, 1], + received: base_feature_version, + })); + } + Ok(()) + } +} + +/// What one erase transition left behind. +#[derive(Debug)] +pub enum DocumentEraseResult { + /// The document is absent by id as of the proof's block. It already was + /// before the erase ran, so this is an observation of the state the erase + /// affected, not evidence that this erase executed — read the current + /// lifecycle with [`Sdk::document_current_lifecycle`] to find out how much + /// history is left. + AbsentAsOfProof(Identifier), +} + +/// Reads the observation an erase leaves behind out of the verified result. +/// +/// Its own function so a test can drive it with the outcome an erase actually +/// produces: the proof classifier reports an erase as affected state, which the +/// strict wait refuses, so the erase has to take the affected-state wait and +/// this has to accept what that wait returns. +fn erase_observation(result: StateTransitionProofResult) -> Result { + match result { + StateTransitionProofResult::VerifiedDocuments(documents) => { + if let Some((erased_id, None)) = documents.into_iter().next() { + Ok(DocumentEraseResult::AbsentAsOfProof(erased_id)) + } else { + Err(Error::DriveProofError( + drive::error::proof::ProofError::UnexpectedResultProof( + "expected an absent document in the VerifiedDocuments result for an \ + erase transition" + .to_string(), + ), + vec![], + Default::default(), + )) + } + } + _ => Err(Error::DriveProofError( + drive::error::proof::ProofError::UnexpectedResultProof( + "expected VerifiedDocuments for a document erase transition".to_string(), + ), + vec![], + Default::default(), + )), + } +} + +fn lifecycle_from_history( + history: Option, +) -> Option { + history.and_then(|history| history.lifecycle) +} + +impl Sdk { + /// Erases a chunk of the retained revisions of an already deleted document. + /// + /// The first erase must be signed by the document's owner; any identity may + /// submit the ones after it. A document with more retained revisions than + /// one transition may remove needs several, and this method broadcasts one. + /// + /// The proof this returns authenticates that the document is absent by id, + /// which it already was before the erase ran, so it is an observation of + /// the state the erase affected rather than evidence that this erase + /// executed. That is why it takes the affected-state wait: the strict wait + /// refuses exactly this classification. Use + /// [`Sdk::document_current_lifecycle`] to observe how much history is left. + pub async fn document_erase>( + &self, + erase_document_transition_builder: DocumentEraseTransitionBuilder, + signing_key: &IdentityPublicKey, + signer: &S, + ) -> Result { + let platform_version = self.version(); + let put_settings = erase_document_transition_builder.settings; + + let state_transition = erase_document_transition_builder + .sign(self, signing_key, signer, platform_version) + .await?; + + let proof_result = state_transition + .broadcast_and_wait_for_affected_state::(self, put_settings) + .await?; + + erase_observation(proof_result) + } + + /// Reads where a document stands in its lifecycle right now. + /// + /// Deliberately named apart from the erase and delete calls: the answer + /// describes committed state at the moment of the read, not the outcome of + /// any particular transition. Another erase, or a re-create of the same id, + /// may land between a transition and this read. `None` means the queried + /// storage/proof generation does not authenticate lifecycle metadata; it + /// must not be interpreted as an authenticated absent state. + pub async fn document_current_lifecycle( + &self, + data_contract_id: Identifier, + document_type_name: String, + document_id: Identifier, + ) -> Result, Error> { + use dash_platform_queries::documents::document_history_query::DocumentHistoryQuery; + use drive_proof_verifier::types::DocumentHistory; + + let history = DocumentHistory::fetch( + self, + DocumentHistoryQuery { + data_contract_id, + document_type_name, + document_id, + // The oldest retained revision, if any: the page is not the + // point, the metadata alongside it is. + filter: DocumentHistoryFilter::StartAtTime(0), + limit: Some(1), + }, + ) + .await?; + + Ok(lifecycle_from_history(history)) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use drive::query::document_history_drive_query::DocumentHistoryState; + use std::collections::BTreeMap; + + /// The erase reads its observation out of exactly the result the + /// affected-state wait hands back, and refuses anything else. + #[test] + fn should_read_an_absent_document_as_the_erases_observation() { + let id = Identifier::from([9u8; 32]); + let observation = erase_observation(StateTransitionProofResult::VerifiedDocuments( + BTreeMap::from([(id, None)]), + )) + .expect("an absent document is what an erase leaves behind"); + assert!(matches!( + observation, + DocumentEraseResult::AbsentAsOfProof(seen) if seen == id + )); + + let document = dpp::document::Document::V0(Default::default()); + erase_observation(StateTransitionProofResult::VerifiedDocuments( + BTreeMap::from([(id, Some(document))]), + )) + .expect_err("a document that is still there did not survive an erase"); + + erase_observation(StateTransitionProofResult::VerifiedTokenBalanceAbsence(id)) + .expect_err("only a document result can describe an erase"); + } + + #[test] + fn should_not_turn_unavailable_lifecycle_into_absent() { + use drive_proof_verifier::types::DocumentHistory; + + assert_eq!(lifecycle_from_history(None), None); + assert_eq!( + lifecycle_from_history(Some(DocumentHistory { + entries: vec![], + lifecycle: None, + })), + None + ); + let absent = DocumentHistoryLifecycle { + state: DocumentHistoryState::Absent, + remaining_revisions: 0, + times: Default::default(), + }; + assert_eq!( + lifecycle_from_history(Some(DocumentHistory { + entries: vec![], + lifecycle: Some(absent.clone()), + })), + Some(absent) + ); + } + + #[cfg(feature = "mocks")] + fn builder_with(settings: Option) -> DocumentEraseTransitionBuilder { + let contract = dpp::tests::fixtures::get_dashpay_contract_fixture( + None, + 0, + PlatformVersion::latest().protocol_version, + ) + .data_contract_owned(); + let builder = DocumentEraseTransitionBuilder::new( + Arc::new(contract), + "profile".to_string(), + Identifier::from([1u8; 32]), + Identifier::from([2u8; 32]), + ); + match settings { + Some(settings) => builder.with_settings(settings), + None => builder, + } + } + + /// A fee increase or creation options handed over inside the put settings + /// reach the signing step, and an explicit builder value still wins. + #[cfg(feature = "mocks")] + #[test] + fn should_sign_with_the_settings_fee_increase_unless_set_explicitly() { + let options = StateTransitionCreationOptions { + batch_feature_version: Some(1), + ..Default::default() + }; + let settings = PutSettings { + user_fee_increase: Some(250), + state_transition_creation_options: Some(options), + ..Default::default() + }; + assert_eq!(builder_with(None).signing_parameters(), (0, None)); + assert_eq!( + builder_with(Some(settings)).signing_parameters(), + (250, Some(options)) + ); + assert_eq!( + builder_with(Some(settings)) + .with_user_fee_increase(7) + .with_state_transition_creation_options(Default::default()) + .signing_parameters(), + (7, Some(Default::default())) + ); + } + + /// The version gate runs before the nonce fetch, so a platform version + /// that cannot construct an erase is refused without reserving a nonce. + #[test] + fn should_refuse_an_erase_the_platform_version_cannot_construct() { + let too_old = PlatformVersion::get(14).unwrap(); + let error = DocumentEraseTransitionBuilder::check_erase_is_constructible(None, too_old) + .expect_err("protocol 14 has no erase transition"); + assert!(error + .to_string() + .contains("erase transitions do not exist at this platform version")); + let current = PlatformVersion::latest(); + DocumentEraseTransitionBuilder::check_erase_is_constructible(None, current) + .expect("protocol 15 constructs erases"); + for shipped in [0, 1] { + let options = StateTransitionCreationOptions { + batch_feature_version: Some(shipped), + ..Default::default() + }; + DocumentEraseTransitionBuilder::check_erase_is_constructible(Some(&options), current) + .expect("both shipped batch wire formats carry an erase"); + } + let unknown_batch = StateTransitionCreationOptions { + batch_feature_version: Some(9), + ..Default::default() + }; + DocumentEraseTransitionBuilder::check_erase_is_constructible(Some(&unknown_batch), current) + .expect_err("an unknown batch version is refused before any nonce is reserved"); + let unknown_method = StateTransitionCreationOptions { + method_feature_version: Some(1), + ..Default::default() + }; + DocumentEraseTransitionBuilder::check_erase_is_constructible( + Some(&unknown_method), + current, + ) + .expect_err("an unknown erase method version is refused before any nonce is reserved"); + let unknown_base = StateTransitionCreationOptions { + base_feature_version: Some(9), + ..Default::default() + }; + DocumentEraseTransitionBuilder::check_erase_is_constructible(Some(&unknown_base), current) + .expect_err("an unknown erase base version is refused before any nonce is reserved"); + } +} diff --git a/packages/rs-sdk/src/platform/documents/transitions/mod.rs b/packages/rs-sdk/src/platform/documents/transitions/mod.rs index 200a2164267..406ac82c0d9 100644 --- a/packages/rs-sdk/src/platform/documents/transitions/mod.rs +++ b/packages/rs-sdk/src/platform/documents/transitions/mod.rs @@ -1,5 +1,6 @@ pub mod create; pub mod delete; +pub mod erase; pub mod purchase; pub mod replace; pub mod set_price; @@ -7,6 +8,7 @@ pub mod transfer; pub use create::{DocumentCreateResult, DocumentCreateTransitionBuilder}; pub use delete::{DocumentDeleteResult, DocumentDeleteTransitionBuilder}; +pub use erase::{DocumentEraseResult, DocumentEraseTransitionBuilder}; pub use purchase::{DocumentPurchaseResult, DocumentPurchaseTransitionBuilder}; pub use replace::{DocumentReplaceResult, DocumentReplaceTransitionBuilder}; pub use set_price::{DocumentSetPriceResult, DocumentSetPriceTransitionBuilder}; diff --git a/packages/rs-sdk/src/platform/query.rs b/packages/rs-sdk/src/platform/query.rs index 1e5e1044656..48a25b23613 100644 --- a/packages/rs-sdk/src/platform/query.rs +++ b/packages/rs-sdk/src/platform/query.rs @@ -1484,6 +1484,7 @@ mod history_query_tests { lifecycle: Some(DocumentHistoryLifecycle { state: DocumentHistoryState::Active, remaining_revisions: 2, + times: Default::default(), }), }; sdk.mock() diff --git a/packages/rs-sdk/src/platform/transition/broadcast.rs b/packages/rs-sdk/src/platform/transition/broadcast.rs index 8dfc8152d61..8c2b1b83360 100644 --- a/packages/rs-sdk/src/platform/transition/broadcast.rs +++ b/packages/rs-sdk/src/platform/transition/broadcast.rs @@ -196,9 +196,7 @@ impl BroadcastStateTransition for StateTransition { settings: Option, ) -> Result<(T, ResponseMetadata), Error> { let (outcome, metadata) = self.wait_for_outcome_with_metadata(sdk, settings).await?; - // An execution-proved outcome carries a strictly stronger guarantee - // than the requested snapshot, so both tags are accepted here. - convert_proof_result::(outcome.into_result()).map(|converted| (converted, metadata)) + convert_proof_result::(accept_affected_state(outcome)).map(|c| (c, metadata)) } async fn broadcast_and_wait_for_affected_state< @@ -277,6 +275,12 @@ fn require_execution_proved( } } +/// Accept either tag for the affected-state wait APIs: an execution-proved +/// outcome carries a strictly stronger guarantee than the snapshot requested. +fn accept_affected_state(outcome: StateTransitionProofOutcome) -> StateTransitionProofResult { + outcome.into_result() +} + /// Convert the verified inner result into the caller's expected type. fn convert_proof_result>( result: StateTransitionProofResult, @@ -476,4 +480,29 @@ mod tests { require_execution_proved(StateTransitionProofOutcome::ExecutionProved(proved)) .expect("execution-proved outcomes must pass the strict wait"); } + + /// The outcome an erase produces, run through both gates. A transition + /// family the classifier reports as affected state cannot use the strict + /// wait: its own valid proof would come back as an error. + #[test] + fn affected_state_wait_accepts_what_the_strict_wait_refuses() { + use std::collections::BTreeMap; + + let erased = || { + StateTransitionProofResult::VerifiedDocuments(BTreeMap::from([( + Identifier::from([9u8; 32]), + None, + )])) + }; + + let err = require_execution_proved(StateTransitionProofOutcome::AffectedState(erased())) + .expect_err("the strict wait must refuse an erase's own outcome"); + assert!(matches!(err, Error::ExecutionNotProved(_))); + + assert_eq!( + accept_affected_state(StateTransitionProofOutcome::AffectedState(erased())), + erased(), + "the affected-state wait must return the erase's observation unchanged" + ); + } } diff --git a/packages/wasm-dpp/src/document/state_transition/batch_transition/document_transition/mod.rs b/packages/wasm-dpp/src/document/state_transition/batch_transition/document_transition/mod.rs index 89e0e5df571..1a5acf13908 100644 --- a/packages/wasm-dpp/src/document/state_transition/batch_transition/document_transition/mod.rs +++ b/packages/wasm-dpp/src/document/state_transition/batch_transition/document_transition/mod.rs @@ -68,6 +68,7 @@ impl DocumentTransitionWasm { DocumentTransition::Transfer(_) => JsValue::null(), DocumentTransition::UpdatePrice(_) => JsValue::null(), DocumentTransition::Purchase(_) => JsValue::null(), + DocumentTransition::Erase(_) => JsValue::null(), DocumentTransition::IndexOnlyDelete(index_only_delete) => { let json_value = index_only_delete.data().to_json_value().unwrap(); json_value @@ -118,6 +119,7 @@ impl DocumentTransitionWasm { DocumentTransition::UpdatePrice(update_price) => Some(update_price.price()), DocumentTransition::Purchase(purchase) => Some(purchase.price()), DocumentTransition::IndexOnlyDelete(_) => None, + DocumentTransition::Erase(_) => None, } } @@ -131,6 +133,7 @@ impl DocumentTransitionWasm { DocumentTransition::UpdatePrice(_) => None, DocumentTransition::Purchase(_) => None, DocumentTransition::IndexOnlyDelete(_) => None, + DocumentTransition::Erase(_) => None, } } diff --git a/packages/wasm-dpp2/src/enums/batch/batch_enum.rs b/packages/wasm-dpp2/src/enums/batch/batch_enum.rs index 130c0740906..58ccca1bc70 100644 --- a/packages/wasm-dpp2/src/enums/batch/batch_enum.rs +++ b/packages/wasm-dpp2/src/enums/batch/batch_enum.rs @@ -13,6 +13,7 @@ pub enum BatchTypeWasm { UpdatePrice, IgnoreWhileBumpingRevision, IndexOnlyDelete, + Erase, } impl TryFrom<&JsValue> for BatchTypeWasm { @@ -33,6 +34,7 @@ impl TryFrom<&JsValue> for BatchTypeWasm { "updateprice" => Ok(BatchTypeWasm::UpdatePrice), "ignorewhilebumpingrevision" => Ok(BatchTypeWasm::IgnoreWhileBumpingRevision), "indexonlydelete" => Ok(BatchTypeWasm::IndexOnlyDelete), + "erase" => Ok(BatchTypeWasm::Erase), _ => Err(WasmDppError::invalid_argument(format!( "unknown batch type value: {}", enum_val @@ -53,6 +55,7 @@ impl TryFrom<&JsValue> for BatchTypeWasm { 5 => Ok(BatchTypeWasm::UpdatePrice), 6 => Ok(BatchTypeWasm::IgnoreWhileBumpingRevision), 7 => Ok(BatchTypeWasm::IndexOnlyDelete), + 8 => Ok(BatchTypeWasm::Erase), _ => Err(WasmDppError::invalid_argument(format!( "unknown batch type value: {}", enum_val @@ -82,6 +85,7 @@ impl From for String { BatchTypeWasm::UpdatePrice => String::from("updatePrice"), BatchTypeWasm::IgnoreWhileBumpingRevision => String::from("ignoreWhileBumpingRevision"), BatchTypeWasm::IndexOnlyDelete => String::from("indexOnlyDelete"), + BatchTypeWasm::Erase => String::from("erase"), } } } @@ -99,6 +103,7 @@ impl From for BatchTypeWasm { BatchTypeWasm::IgnoreWhileBumpingRevision } DocumentTransitionActionType::IndexOnlyDelete => BatchTypeWasm::IndexOnlyDelete, + DocumentTransitionActionType::Erase => BatchTypeWasm::Erase, } } } diff --git a/packages/wasm-dpp2/src/state_transitions/batch/document_transition.rs b/packages/wasm-dpp2/src/state_transitions/batch/document_transition.rs index 5b022185001..1e5cce18f0b 100644 --- a/packages/wasm-dpp2/src/state_transitions/batch/document_transition.rs +++ b/packages/wasm-dpp2/src/state_transitions/batch/document_transition.rs @@ -53,6 +53,7 @@ impl DocumentTransitionWasm { DocumentTransitionActionType::UpdatePrice => 5, DocumentTransitionActionType::IgnoreWhileBumpingRevision => 6, DocumentTransitionActionType::IndexOnlyDelete => 7, + DocumentTransitionActionType::Erase => 8, } } diff --git a/packages/wasm-sdk/src/queries/document.rs b/packages/wasm-sdk/src/queries/document.rs index e1aa60f724f..37f3b4706bc 100644 --- a/packages/wasm-sdk/src/queries/document.rs +++ b/packages/wasm-sdk/src/queries/document.rs @@ -176,7 +176,13 @@ export interface DocumentHistoryQuery { */ documentId: IdentifierLike - /** Inclusive lower time bound. Supply exactly one selector. */ + /** + * Inclusive lower time bound. Supply exactly one selector. + * + * Every selector is an exact u64: a `number` is accepted only up to + * `Number.MAX_SAFE_INTEGER`, and anything larger must be a `bigint`, since + * JavaScript would have rounded it before the query is built. + */ startAtMs?: bigint | number; /** Complete exclusive cursor returned by a previous page. */ startAfter?: { timeMs: bigint | number; revision: bigint | number }; @@ -307,7 +313,6 @@ impl DocumentHistoryEntryWasm { pub fn time_ms(&self) -> BigInt { BigInt::from(self.time_ms) } - #[wasm_bindgen(getter)] pub fn revision(&self) -> BigInt { BigInt::from(self.revision) @@ -340,6 +345,10 @@ impl DocumentHistoryEntryWasm { pub struct DocumentHistoryLifecycleWasm { state: String, remaining_revisions: u64, + deleted_at_ms: u64, + erasing_started_at_ms: u64, + erasing_from_time_ms: u64, + erasing_from_revision: u64, } #[derive(Serialize)] @@ -347,11 +356,21 @@ pub struct DocumentHistoryLifecycleWasm { struct DocumentHistoryLifecycleSerde { state: String, remaining_revisions: String, + deleted_at_ms: String, + erasing_started_at_ms: String, + erasing_from_time_ms: String, + erasing_from_revision: String, } #[wasm_bindgen(js_class = DocumentHistoryLifecycle)] impl DocumentHistoryLifecycleWasm { - #[wasm_bindgen(getter)] + /// `ACTIVE` while the document is visible to ordinary reads, `DELETED` + /// once it has been deleted and its revisions are retained, `ERASING` once + /// an authorized erasure has begun, `ABSENT` when nothing is left. + #[wasm_bindgen( + getter, + unchecked_return_type = "\"ACTIVE\" | \"DELETED\" | \"ERASING\" | \"ABSENT\"" + )] pub fn state(&self) -> String { self.state.clone() } @@ -361,6 +380,30 @@ impl DocumentHistoryLifecycleWasm { BigInt::from(self.remaining_revisions) } + /// Zero unless the document has been deleted. + #[wasm_bindgen(getter = "deletedAtMs")] + pub fn deleted_at_ms(&self) -> BigInt { + BigInt::from(self.deleted_at_ms) + } + + /// Zero unless an authorized erasure has begun. + #[wasm_bindgen(getter = "erasingStartedAtMs")] + pub fn erasing_started_at_ms(&self) -> BigInt { + BigInt::from(self.erasing_started_at_ms) + } + + /// Timestamp of the newest revision retained when the erasure began. + #[wasm_bindgen(getter = "erasingFromTimeMs")] + pub fn erasing_from_time_ms(&self) -> BigInt { + BigInt::from(self.erasing_from_time_ms) + } + + /// History sequence of the newest revision retained when the erasure began. + #[wasm_bindgen(getter = "erasingFromRevision")] + pub fn erasing_from_revision(&self) -> BigInt { + BigInt::from(self.erasing_from_revision) + } + #[wasm_bindgen(js_name = toJSON)] pub fn to_json(&self) -> Result { serialization::to_json(&self.to_serde()).map_err(WasmSdkError::from) @@ -372,6 +415,10 @@ impl DocumentHistoryLifecycleWasm { DocumentHistoryLifecycleSerde { state: self.state.clone(), remaining_revisions: self.remaining_revisions.to_string(), + deleted_at_ms: self.deleted_at_ms.to_string(), + erasing_started_at_ms: self.erasing_started_at_ms.to_string(), + erasing_from_time_ms: self.erasing_from_time_ms.to_string(), + erasing_from_revision: self.erasing_from_revision.to_string(), } } } @@ -398,6 +445,8 @@ impl DocumentHistoryResultWasm { self.entries.clone() } + /// Missing when the legacy storage generation cannot authenticate lifecycle + /// metadata. #[wasm_bindgen(getter)] pub fn lifecycle(&self) -> Option { self.lifecycle.clone() @@ -452,10 +501,16 @@ impl DocumentHistoryResultWasm { .map(|lifecycle| DocumentHistoryLifecycleWasm { state: match lifecycle.state { DocumentHistoryState::Active => "ACTIVE", + DocumentHistoryState::Deleted => "DELETED", + DocumentHistoryState::Erasing => "ERASING", DocumentHistoryState::Absent => "ABSENT", } .to_owned(), remaining_revisions: lifecycle.remaining_revisions, + deleted_at_ms: lifecycle.times.deleted_at_ms, + erasing_started_at_ms: lifecycle.times.erasing_started_at_ms, + erasing_from_time_ms: lifecycle.times.erasing_from_time_ms, + erasing_from_revision: lifecycle.times.erasing_from_revision, }), }) } @@ -1556,7 +1611,8 @@ mod tests { mod history_wasm_tests { use super::*; use drive_proof_verifier::types::{ - DocumentHistoryEntry, DocumentHistoryLifecycle, DocumentHistoryState, + DocumentHistoryEntry, DocumentHistoryLifecycle, DocumentHistoryLifecycleTimes, + DocumentHistoryState, }; use js_sys::{Array, Reflect}; use wasm_bindgen::JsCast; @@ -1581,6 +1637,37 @@ mod history_wasm_tests { ); } + /// A `u64` selector crosses from JavaScript exactly or not at all: the + /// deserializer takes a `number` only while it is a safe integer, since a + /// larger one was rounded before it reached Rust, and takes a `bigint` at + /// any `u64`. Nothing in this crate adds to that; this pins that it holds. + #[wasm_bindgen_test] + fn should_take_history_selectors_as_exact_integers_only() { + let query = |start_at_ms: JsValue| { + let object = js_sys::Object::new(); + let id = IdentifierWasm::from([1u8; 32]).to_base58(); + Reflect::set(&object, &"dataContractId".into(), &id.clone().into()).unwrap(); + Reflect::set(&object, &"documentTypeName".into(), &"note".into()).unwrap(); + Reflect::set(&object, &"documentId".into(), &id.into()).unwrap(); + Reflect::set(&object, &"startAtMs".into(), &start_at_ms).unwrap(); + parse_document_history_query(JsValue::from(object).unchecked_into()) + }; + let past_safe = (1u64 << 53) + 1; + + let parsed = query(JsValue::from(js_sys::BigInt::from(past_safe))) + .expect("a bigint carries the exact value"); + assert_eq!( + parsed.filter, + DocumentHistoryFilter::StartAtTime(past_safe), + "the selector must be the value the caller passed" + ); + + query(JsValue::from(past_safe as f64)) + .expect_err("a number past Number.MAX_SAFE_INTEGER was already rounded and is refused"); + query(JsValue::from(-1.0)).expect_err("a negative number is refused"); + query(JsValue::from(1.5)).expect_err("a fractional number is refused"); + } + #[wasm_bindgen_test] fn should_preserve_same_time_revisions_and_exact_lifecycle_counts_in_javascript() { let count = (1u64 << 53) + 1; @@ -1596,6 +1683,7 @@ mod history_wasm_tests { lifecycle: Some(DocumentHistoryLifecycle { state: DocumentHistoryState::Active, remaining_revisions: count, + times: Default::default(), }), }; let result = JsValue::from( @@ -1620,6 +1708,47 @@ mod history_wasm_tests { ); } + /// Every lifecycle time crosses into JavaScript as an exact BigInt, like + /// the counts and revisions beside them: a millisecond timestamp does not + /// survive a JavaScript number. + #[wasm_bindgen_test] + fn should_report_the_erasing_state_and_its_times_exactly_in_javascript() { + let started_at = (1u64 << 53) + 3; + let history = DocumentHistory { + entries: vec![], + lifecycle: Some(DocumentHistoryLifecycle { + state: DocumentHistoryState::Erasing, + remaining_revisions: 7, + times: DocumentHistoryLifecycleTimes { + deleted_at_ms: 1_700_000_000_001, + erasing_started_at_ms: started_at, + erasing_from_time_ms: 1_700_000_000_002, + erasing_from_revision: 42, + }, + }), + }; + let result = JsValue::from( + DocumentHistoryResultWasm::from_history(history, [1; 32].into(), "note").unwrap(), + ); + let lifecycle = Reflect::get(&result, &"lifecycle".into()).unwrap(); + assert_eq!( + Reflect::get(&lifecycle, &"state".into()).unwrap(), + JsValue::from_str("ERASING") + ); + for (key, expected) in [ + ("deletedAtMs", 1_700_000_000_001u64), + ("erasingStartedAtMs", started_at), + ("erasingFromTimeMs", 1_700_000_000_002), + ("erasingFromRevision", 42), + ] { + assert_eq!( + Reflect::get(&lifecycle, &key.into()).unwrap(), + JsValue::from(expected), + "{key} must survive as an exact BigInt" + ); + } + } + #[wasm_bindgen_test] fn should_serialize_history_data_in_a_proof_metadata_response() { let history = DocumentHistory { @@ -1631,6 +1760,7 @@ mod history_wasm_tests { lifecycle: Some(DocumentHistoryLifecycle { state: DocumentHistoryState::Active, remaining_revisions: (1u64 << 53) + 1, + times: Default::default(), }), }; let response = ProofMetadataResponseWasm::from_sdk_parts( diff --git a/packages/wasm-sdk/src/state_transitions/document.rs b/packages/wasm-sdk/src/state_transitions/document.rs index fb75f79180d..4755e0aa319 100644 --- a/packages/wasm-sdk/src/state_transitions/document.rs +++ b/packages/wasm-sdk/src/state_transitions/document.rs @@ -12,7 +12,9 @@ use dash_sdk::dpp::fee::Credits; use dash_sdk::dpp::identity::IdentityPublicKey; use dash_sdk::dpp::platform_value::Identifier; use dash_sdk::dpp::tokens::token_payment_info::TokenPaymentInfo; -use dash_sdk::platform::documents::transitions::DocumentDeleteTransitionBuilder; +use dash_sdk::platform::documents::transitions::{ + DocumentDeleteTransitionBuilder, DocumentEraseTransitionBuilder, +}; use dash_sdk::platform::transition::purchase_document::PurchaseDocument; use dash_sdk::platform::transition::put_document::PutDocument; use dash_sdk::platform::transition::transfer_document::TransferDocument; @@ -506,6 +508,157 @@ impl WasmSdk { } } +// ============================================================================ +// Document Erase +// ============================================================================ + +/// TypeScript interface for document erase options +#[wasm_bindgen(typescript_custom_section)] +const DOCUMENT_ERASE_OPTIONS_TS: &'static str = r#" +/** + * Options for erasing the retained revisions of an already deleted document. + */ +export interface DocumentEraseOptions { + /** + * The document to erase, or the identifiers that name it. The document is + * already invisible to ordinary reads, so its identifiers are all that is + * needed. + */ + document: Document | { + id: IdentifierLike; + ownerId: IdentifierLike; + dataContractId: IdentifierLike; + documentTypeName: string; + }; + + /** + * The identity submitting and paying for this erase. It is the identity the + * key and signer below belong to, and its contract nonce is consumed. + * Defaults to the document's owner, which is who the first erase must come + * from; the erases after it may come from any identity, which then names + * itself here. + */ + identityId?: IdentifierLike; + + /** + * The identity public key to use for signing the transition. + * The first erase must be signed by the document's owner; any identity may + * sign the ones after it. + */ + identityKey: IdentityPublicKey; + + /** + * Signer containing the private key that corresponds to the identity key. + * Use IdentitySigner to add the private key before calling. + */ + signer: IdentitySigner; + + /** + * Optional settings for the broadcast operation. + * Includes retries, timeouts, userFeeIncrease, etc. + */ + settings?: PutSettings; +} +"#; + +#[wasm_bindgen] +extern "C" { + #[wasm_bindgen(typescript_type = "DocumentEraseOptions")] + pub type DocumentEraseOptionsJs; +} + +#[wasm_bindgen] +impl WasmSdk { + /// Erase a chunk of the retained revisions of an already deleted document. + /// + /// A document with more retained revisions than one transition may remove + /// needs several calls. The first must come from the document's owner and + /// commits the document to erasure; any identity may make the ones after + /// it. Read the document's history to see how much is left. + /// + /// @param options - Erase options including the document identifiers, identity key, and signer + /// @returns Promise that resolves after an affected-state proof authenticates + /// that the document is absent by id. This does not prove that this erase + /// executed or was accepted; query document history to observe the lifecycle + /// and remaining revisions. + #[wasm_bindgen(js_name = "documentErase")] + pub async fn document_erase( + &self, + options: DocumentEraseOptionsJs, + ) -> Result<(), WasmSdkError> { + let document_js = js_sys::Reflect::get(&options, &JsValue::from_str("document")) + .map_err(|_| WasmSdkError::invalid_argument("document is required"))?; + + if document_js.is_undefined() || document_js.is_null() { + return Err(WasmSdkError::invalid_argument("document is required")); + } + + // The transition carries only the base, so the values of a Document + // instance are not needed and a plain object of identifiers works for + // every document type. + let (document_id, owner_id, contract_id, document_type_name): ( + Identifier, + Identifier, + Identifier, + String, + ) = if get_class_type(&document_js).ok().as_deref() == Some("Document") { + let doc: DocumentWasm = document_js + .to_wasm::("Document") + .map(|boxed| (*boxed).clone())?; + let doc_inner: Document = doc.clone().into(); + ( + doc.id().into(), + doc_inner.owner_id(), + doc.data_contract_id().into(), + doc.document_type_name(), + ) + } else { + ( + IdentifierWasm::try_from_options(&document_js, "id")?.into(), + IdentifierWasm::try_from_options(&document_js, "ownerId")?.into(), + IdentifierWasm::try_from_options(&document_js, "dataContractId")?.into(), + try_from_options_with(&document_js, "documentTypeName", |v| { + try_to_string(v, "documentTypeName") + })?, + ) + }; + + // The builder's owner is the identity that submits and pays; only the + // first erase has to be the document's owner, so a continuation names + // its own identity here and keeps the document owner as metadata. + let submitter_id: Identifier = + match try_from_options_optional::(&options, "identityId")? { + Some(identity_id) => identity_id.into(), + None => owner_id, + }; + + let identity_key_wasm = IdentityPublicKeyWasm::try_from_options(&options, "identityKey")?; + let identity_key: IdentityPublicKey = identity_key_wasm.into(); + let signer = IdentitySignerWasm::try_from_options(&options, "signer")?; + let data_contract = self.get_or_fetch_contract(contract_id).await?; + let settings = + try_from_options_optional::(&options, "settings")?.map(Into::into); + + let builder = DocumentEraseTransitionBuilder::new( + Arc::new(data_contract), + document_type_name, + document_id, + submitter_id, + ); + let builder = if let Some(s) = settings { + builder.with_settings(s) + } else { + builder + }; + + self.inner_sdk() + .document_erase(builder, &identity_key, &signer) + .await?; + + Ok(()) + } +} + // ============================================================================ // Document Transfer // ============================================================================