Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions book/src/drive/cost-tracking.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,14 @@ pub enum LowLevelDriveOperation {
FunctionOperation(FunctionOp),
CalculatedCostOperation(OperationCost),
PreCalculatedFeeResult(FeeResult),
CalculatedCostOperationWithRefundOwners {
cost: OperationCost,
refund_owners: RefundOwnersByIdentifier,
},
}
```

Four variants, each representing a different kind of cost:
Five variants, each representing a different kind of cost:

### GroveOperation

Expand Down Expand Up @@ -108,6 +112,10 @@ impl FunctionOp {

A fee result that was already computed elsewhere and just needs to be included in the total. This is a pass-through -- no further calculation needed.

### CalculatedCostOperationWithRefundOwners

A `CalculatedCostOperation` whose sectioned storage removal comes with the typed `RefundOwner` recorded for every carrier key when the removed bytes were split. It is pushed by the batch apply generations that split removed bytes with typed storage flags (`push_drive_operation_result_with_refund_owners`). `operation_cost()` rejects it on purpose: a fee decoder that predates typed owners reaches `operation_cost()` through its catch-all arm and therefore fails closed instead of pricing a refund whose owner it cannot route. `combine_cost_operations` leaves it out for the same reason (folding it into a plain `OperationCost` would erase the owners); `combine_cost_operations_with_refund_owners` sums plain and typed costs together and returns the owner of every sectioned removal key: a plain cost's keys are identities, a typed cost must have recorded an owner for each of its own keys, the system key is never an owner, and one key attributed to two owners is refused. See [Refunds](../fees/overview.md#refunds) for how the owners are recorded.

## BaseOp: Arithmetic Operation Costs

For simple computational operations (not storage-related), the `BaseOp` enum provides fixed costs:
Expand Down Expand Up @@ -320,7 +328,7 @@ These provide a cleaner API than constructing `QualifiedGroveDbOp` directly, and
- Let operations accumulate in the `drive_operations` vector throughout the call chain.

**Do not:**
- Call `operation_cost()` on a `GroveOperation` -- it will return an error. Grove operations must be executed first; only `CalculatedCostOperation` carries a usable cost.
- Call `operation_cost()` on a `GroveOperation` -- it will return an error. Grove operations must be executed first; only `CalculatedCostOperation` carries a usable cost. `CalculatedCostOperationWithRefundOwners` also returns an error there, by design: its owners must be read by a decoder that routes typed owners.
- Forget that storage fees and processing fees are calculated differently. Storage fees are proportional to bytes. Processing fees are a complex function of seeks, loads, hashes, and byte movements.
- Assume fee rates are constant. They are versioned through `FeeVersion` and can change between protocol versions.
- Ignore `removed_bytes_from_system`. This tracks bytes removed that belong to the system rather than a specific identity, affecting the refund calculation.
26 changes: 23 additions & 3 deletions book/src/fees/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,13 +203,33 @@ operations and measures the actual cost of each insert, delete, and query.

When data is removed from GroveDB (a document is deleted, a key is removed),
the system calculates a refund of the original storage fee. Refunds are tracked
per identity per epoch:
per owner per epoch:

```rust
pub struct FeeRefunds(pub CreditsPerEpochByIdentifier);
// BTreeMap<IdentifierBytes32, BTreeMap<EpochIndex, Credits>>
pub struct FeeRefunds(pub CreditsPerEpochByIdentifier, pub RefundOwnersByIdentifier);
// BTreeMap<[u8; 32], BTreeMap<EpochIndex, Credits>> plus BTreeMap<[u8; 32], RefundOwner>
```

The first field is the carrier GroveDB hands back: removed bytes keyed by a
32-byte identifier. The second field records the typed owner of every carrier
key. `RefundOwner` (`rs-dpp/src/fee/refund_owner`) names the owner
explicitly: `Identity(id)` for bytes an identity paid for, or
`ContractBucket { contract_id, position }` for bytes a contract credit bucket
paid for. An identity's carrier key is its id, so every historical record
keeps its key; a bucket's carrier key is a domain separated double SHA-256 of
the contract id and the bucket position. The kind is always read from the
record and never inferred from the shape of the key.

The owner is bound to the bytes when they are stored, in the element flags
(`rs-drive/src/util/storage_flags`). Flag type bytes 0 to 3 are the
historical unowned and identity-owned encodings, produced and parsed by the
pinned `grovedb-epoch-based-storage-flags` crate so they cannot drift. Type
bytes 4 and 5 carry a contract bucket owner. Only the batch apply generation
that knows typed owners splits removed bytes for type bytes 4 and 5 and it
records the owner of every sectioned removal on the cost operation it pushes
(`CalculatedCostOperationWithRefundOwners`); the earlier generation hands the
flags to the crate and fails closed on a type byte it does not know.

Refunds are not 1:1 with the original fee because storage fees are distributed
across future epochs (see below). The refund amount depends on how many epochs
have elapsed since the data was stored — the longer the data has been stored, the
Expand Down
35 changes: 31 additions & 4 deletions packages/rs-dpp/src/fee/fee_result/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ use crate::consensus::fee::fee_error::FeeError;

use crate::fee::fee_result::refunds::FeeRefunds;
use crate::fee::fee_result::BalanceChange::{AddToBalance, NoBalanceChange, RemoveFromBalance};
use crate::fee::refund_owner::RefundOwner;
use crate::fee::Credits;
use crate::prelude::UserFeeIncrease;
use crate::ProtocolError;
Expand Down Expand Up @@ -121,13 +122,29 @@ impl BalanceChangeForIdentity {
&self.change
}

/// Returns refund amount of credits for other identities
/// Returns refund amount of credits for other identities.
///
/// Identity keyed view kept for the shipped balance consumer, which runs
/// only under generations that predate typed owners. A path that may
/// hold bucket owned refunds uses [`Self::other_typed_refunds`], or
/// checks `FeeRefunds::ensure_identity_owners_only` first.
pub fn other_refunds(&self) -> BTreeMap<Identifier, Credits> {
self.fee_result
.fee_refunds
.calculate_all_refunds_except_identity(self.identity_id)
}

/// Returns the refund amount of credits for every recorded owner other
/// than the paying identity, keyed by the typed owner.
///
/// A carrier key without a recorded owner is an error: the refund cannot
/// be routed and must not be guessed at.
pub fn other_typed_refunds(&self) -> Result<BTreeMap<RefundOwner, Credits>, ProtocolError> {
self.fee_result
.fee_refunds
.calculate_all_refunds_except_owner(&RefundOwner::Identity(self.identity_id))
}

/// Convert into a fee result
pub fn into_fee_result(self) -> FeeResult {
self.fee_result
Expand Down Expand Up @@ -286,7 +303,9 @@ mod tests {
use super::*;
use crate::consensus::fee::fee_error::FeeError;
use crate::fee::epoch::CreditsPerEpoch;
use crate::fee::fee_result::refunds::{CreditsPerEpochByIdentifier, FeeRefunds};
use crate::fee::fee_result::refunds::{
CreditsPerEpochByIdentifier, FeeRefunds, RefundOwnersByIdentifier,
};

fn make_id(byte: u8) -> Identifier {
Identifier::from([byte; 32])
Expand All @@ -298,7 +317,11 @@ mod tests {
credits_per_epoch.insert(0, credits);
let mut map = CreditsPerEpochByIdentifier::new();
map.insert(*identity_id.as_bytes(), credits_per_epoch);
FeeRefunds(map)
let owners = RefundOwnersByIdentifier::from([(
*identity_id.as_bytes(),
RefundOwner::Identity(identity_id),
)]);
FeeRefunds(map, owners)
}

// --- BalanceChangeForIdentity::change() ---
Expand Down Expand Up @@ -344,7 +367,11 @@ mod tests {
let mut map = CreditsPerEpochByIdentifier::new();
map.insert(*id.as_bytes(), credits_per_epoch_self);
map.insert(*other_id.as_bytes(), credits_per_epoch_other);
let refunds = FeeRefunds(map);
let owners = RefundOwnersByIdentifier::from([
(*id.as_bytes(), RefundOwner::Identity(id)),
(*other_id.as_bytes(), RefundOwner::Identity(other_id)),
]);
let refunds = FeeRefunds(map, owners);

let fee_result = FeeResult {
storage_fee: 100,
Expand Down
Loading
Loading