Skip to content

feat(db): Blend v2 schema — entity-category enums, blend_* tables, data models - #658

Merged
aditya1702 merged 16 commits into
blend/pr1-framework-seamfrom
blend/pr2-schema-models
Aug 17, 2026
Merged

feat(db): Blend v2 schema — entity-category enums, blend_* tables, data models#658
aditya1702 merged 16 commits into
blend/pr1-framework-seamfrom
blend/pr2-schema-models

Conversation

@aditya1702

@aditya1702 aditya1702 commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Blend v2 schema + data models

Second of 5 stacked PRs adding Blend Capital v2 lending support (stacked on #657). This PR adds the database schema and the Go writer layer that Blend ingestion populates. The GraphQL readers that expose this data land in #661.

Migrations

state_changes CHECK constraints gain the Blend v2 values under the schema-wide convention that category names the on-chain object and reason names the action: eight categories (BLEND_SUPPLY, BLEND_COLLATERAL, BLEND_DEBT, BLEND_AUCTION, BLEND_EMISSIONS, BLEND_BACKSTOP_EMISSIONS, BLEND_BACKSTOP, BLEND_BACKSTOP_QUEUE). Amount-bearing categories reuse the generic CREDIT/DEBIT/ADD/REMOVE/BURN verbs; only BORROW, REPAY, FLASH_LOAN, BAD_DEBT, FILL, and CLAIM are added. The constraints restate the tightened base lists exactly and are recreated NOT VALID so no hypertable rewrite is needed. The rollback enforces its no-BLEND-rows precondition with a fail-loud guard (a DO block probing both the category and reason lists — served by the columnstore's bloom sparse indexes — since NOT VALID re-narrowed CHECKs cannot reject pre-existing rows), and all constraint drops are IF EXISTS so both directions are re-runnable.

11 new current-state tables — plain mutable tables, not hypertables, because they hold UPSERT-heavy snapshots of live on-chain state (same rationale as sep41_balances). Storage parameters follow the current conventions: fillfactor = 80 only where essentially every row turns over per active ledger (blend_reserves, blend_reserve_emissions, blend_oracle_prices, each with an in-file justification), fillfactor = 90 elsewhere, and no autovacuum_vacuum_cost_limit (cost_delay = 0 already exempts these tables from cross-worker balancing):

Table Holds
blend_pools per pool: oracle, status, backstop rate, max positions, min collateral, admin, reward-zone flag, name
blend_positions per pool/user/reserve: supplied / collateral / borrowed balances + net-supplied / net-borrowed cost basis
blend_reserves per pool/reserve: b/d rates & supplies, backstop credit, interest-rate-curve config, collateral/liability factors
blend_backstop_positions per pool/user: backstop shares + queued-for-withdrawal (Q4W) entries
blend_backstop_pools per pool: total backstop shares / tokens / q4w + emission state
blend_reserve_emissions per pool/reserve-token: emission rate, index, expiration
blend_emissions per user: reserve and backstop emission accrual
blend_oracle_prices per oracle/asset: latest price snapshot
blend_pool_claimed / blend_backstop_claimed lifetime claimed BLND (per pool/user) / Comet LP (per user)
blend_auctions active Dutch auctions per pool/user/type (bid, lot, start block)

Every user-keyed table is indexed on user_account_id for the per-account "my positions" read path. blend_reserves additionally enforces UNIQUE (pool_contract_id, asset_contract_id) — the fold SQL resolves reserve_index by joining on that pair, and UPDATE ... FROM would silently apply an arbitrary source row if it ever matched twice; the backing unique index also serves those joins. Also registers the BLEND protocol.

Go data layer (internal/data/blend, writer surface only)

  • PositionModel — snapshot upserts (last-write-wins), zeroing of exited reserves (rows kept so lifetime-earned survives a full exit), additive net-delta cost-basis folds (asset → reserve resolved via blend_reserves), bad-debt reset, and auction adjustments that convert protocol tokens to underlying at current rates (floored to match the contract; conversion happens once on the batch-summed delta, so folded fills round once, not per fill — pinned by test, skew ≤ n−1 stroops, below the documented rate-staleness bound).
  • PoolModel (partial upsert — a NULL field never clobbers known config; plus an absolute reward-zone set), ReserveModel (full-row upserts plus a ResData-only partial update, both rejecting duplicate (pool, asset) keys per batch — UPDATE ... FROM would silently pick one arbitrary source row), BackstopPositionModel / BackstopPoolModel, Emission / ReserveEmission models, PoolClaimed / BackstopClaimed (additive lifetime totals), and AuctionModel.
  • Wired as data.Models.Blend; adds the 8 Blend category + 6 reason Go constants (the locked cross-PR contract).
  • All models are covered by dbtest.

Notes for reviewers

  • Backstop user-emission rows share blend_emissions with reserve emissions, distinguished by token_id = -1 (backstop) vs >= 0 (reserve token); backstop rows carry the pool as source_contract_id.
  • Lifetime-claimed totals aren't stored on-chain (a claim zeroes on-chain accrual), so they're folded additively from claim events during ingestion.
  • Known limitation: the backstop's backfill-emissions gate (BACKFILL_STATUS / BACKFILL_EMISSIONS) is not captured — during that transitional state accrued backstop BLND isn't yet claimable, so a user's claimable-BLND figure can be overstated in that window. Deliberately out of scope.

🤖 Generated with Claude Code

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a32b0dcef0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/data/blend/positions.go
Comment thread internal/data/blend/positions.go Outdated
@aditya1702
aditya1702 force-pushed the blend/pr2-schema-models branch 3 times, most recently from 4a818a6 to c088f01 Compare July 9, 2026 20:34
@aditya1702
aditya1702 force-pushed the blend/pr2-schema-models branch 2 times, most recently from 4080676 to fb4858f Compare July 10, 2026 11:32
Comment thread internal/indexer/types/types.go Outdated
Comment thread internal/data/blend/pools.go
Comment thread internal/db/migrations/2026-07-08.3-blend_reserves.sql
Comment thread internal/data/blend/reserves.go
@aditya1702
aditya1702 force-pushed the blend/pr2-schema-models branch from 916748d to 8ea8219 Compare July 30, 2026 16:28
@aditya1702 aditya1702 changed the title feat(db): Blend v2 schema — LENDING enums, blend_* tables, data models feat(db): Blend v2 schema — entity-category enums, blend_* tables, data models Jul 30, 2026
@aditya1702
aditya1702 force-pushed the blend/pr2-schema-models branch from 8ea8219 to 0965d80 Compare July 30, 2026 21:39
@aristidesstaffieri

Copy link
Copy Markdown
Contributor

ApplyAuctionAdjustments: rounding order when multiple fills land in one batch

Not a blocker — the magnitude is below an error bound this function already documents and accepts. Raising it because the current rounding order is load-bearing but implicit, and no test pins it.

applyAuctionAdjustmentsSQL SUMs the raw token deltas in the subquery, then converts and truncates the total once. Flooring doesn't distribute over addition, so for n fills folded into one (pool, user, asset) row the result overstates magnitude by 0..n−1 stroops:

floor(a) + floor(b)  ≤  floor(a+b)  ≤  floor(a) + floor(b) + 1

With the existing test fixture (b_rate 1.1, d_rate 1.05), two fills of 5 lot b-tokens give trunc(10 × 1.1) = 11 today versus trunc(5.5) × 2 = 10 per-fill. Signs don't cancel it — the liquidated user's negative deltas skew more negative by the same 1 stroop.

Worth noting this is purely a composition issue, not a mistake in either change that produced it: f73ff53 added the SUM to fix genuine silent drops from UPDATE ... FROM applying one source row per target, and 34ec0f8 added trunc() to fix a genuine fractional tail. The SUM is also correctly described as lossless in the comment — that sentence is about aggregation safety, and it's right. The gap is just that trunc ended up outside a GROUP BY and nobody asked whether trunc-after-SUM equals sum-of-truncs.

Trigger is narrow: 2+ fills against the same (pool, user, asset) in one batch. Live ingestion runs window=1, so that means two partial fills of the same auction in one ledger (two fillers each taking a slice). The windowed migration path makes it easier.

Suggested test — no new fixture needed, and neither existing case can catch this (values protocol-token deltas at current rates and sums duplicate ... adjustments both use amounts that are exact at these rates; the floors the converted underlying case has only one row per user, so nothing aggregates):

t.Run("aggregates before flooring, so folded fills round once not per-fill", func(t *testing.T) {
	// Documents the current rounding order. Flooring doesn't distribute over
	// addition: at b_rate 1.1, two fills of 5 give trunc(10 * 1.1) = 11 here,
	// where per-fill flooring would give trunc(5.5) * 2 = 10. Same 1-stroop
	// skew on the negative (liquidated user) side. If the conversion ever moves
	// inside the GROUP BY, these expectations become 10 / 20 and -10 / -20.
	fillerAddr := keypair.MustRandom().Address()
	userAddr := keypair.MustRandom().Address()
	insertPosition(t, ctx, pool, poolAddr, fillerAddr, 4, "0", "0", "0", "0", "0")
	insertPosition(t, ctx, pool, poolAddr, userAddr, 4, "0", "0", "0", "0", "0")

	runInTx(t, ctx, pool, func(tx pgx.Tx) {
		require.NoError(t, m.ApplyAuctionAdjustments(ctx, tx, []blend.PositionAuctionAdjustment{
			{Pool: poolAddr, User: fillerAddr, Asset: assetAddr, LotBTokensDelta: "5", BidDTokensDelta: "10", LedgerNumber: 91},
			{Pool: poolAddr, User: fillerAddr, Asset: assetAddr, LotBTokensDelta: "5", BidDTokensDelta: "10", LedgerNumber: 92},
			{Pool: poolAddr, User: userAddr, Asset: assetAddr, LotBTokensDelta: "-5", BidDTokensDelta: "-10", LedgerNumber: 91},
			{Pool: poolAddr, User: userAddr, Asset: assetAddr, LotBTokensDelta: "-5", BidDTokensDelta: "-10", LedgerNumber: 92},
		}))
	})

	filler, ok := getPosition(t, ctx, pool, poolAddr, fillerAddr, 4)
	require.True(t, ok)
	assert.Equal(t, "11", filler.NetSupplied, "trunc((5+5) * 1.1) = 11, not trunc(5.5) * 2 = 10")
	assert.Equal(t, "21", filler.NetBorrowed, "trunc((10+10) * 1.05) = 21, not trunc(10.5) * 2 = 20")

	user, ok := getPosition(t, ctx, pool, poolAddr, userAddr, 4)
	require.True(t, ok)
	assert.Equal(t, "-11", user.NetSupplied, "same skew, more negative")
	assert.Equal(t, "-21", user.NetBorrowed)
})

Suggested doc note on applyAuctionAdjustmentsSQL, after the existing paragraph about pre-aggregation:

// Conversion happens after the GROUP BY, so a folded batch is floored once on
// the summed delta rather than per fill. Flooring doesn't distribute over
// addition, so n fills against one position overstate magnitude by at most
// n-1 stroops (at b_rate 1.1, two fills of 5 give trunc(11.0) = 11, not
// trunc(5.5) * 2 = 10). That is strictly smaller than the end-of-window rate
// approximation quantified on ApplyAuctionAdjustments below, and affects the
// same display-only cost-basis fields, so the cheaper single trunc is kept
// deliberately. Moving trunc inside the subquery would restore per-fill
// flooring if that ever matters.

Happy to be told this is over-documenting a stroop.

@aristidesstaffieri

Copy link
Copy Markdown
Contributor

2026-07-08.0 Down: the stated precondition can't be enforced by NOT VALID

Line 33 says the rollback "Requires no BLEND_* rows to exist (delete them first in dev)", but lines 35-41 and 44-49 re-add the narrowed CHECKs as NOT VALID — which is precisely the modifier that tells Postgres not to look for those rows. The rollback therefore succeeds on a database that still has them.

NOT VALID is asymmetric in a way that's easy to misread: existing rows go unchecked, but the constraint is still fully enforced on every new INSERT/UPDATE. So post-rollback you get a table whose BLEND_* rows read back fine but can no longer be written, and the only operations that actually surface the inconsistency are an explicit VALIDATE CONSTRAINT or a later attempt to add the constraint as VALID.

Why the precondition matters more than constraint hygiene. The reason the Down wants those rows gone is that a rollback should restore a state the pre-Blend code can serve — and the pre-Blend resolver hits the default: arm at internal/serve/graphql/resolvers/utils.go:211 on a BLEND_* row and returns an error for the entire stateChanges page, not just that node. So rolling schema and code back together reports success at every step while Blend users' history stays broken, and the blend_* tables are gone by then too, so there's nothing left to explain why. That's the part I'd want enforced rather than commented.

Applies to the reason constraint as well (BORROW, REPAY, FLASH_LOAN, BAD_DEBT, FILL, CLAIM).

Suggested fix — a guard, not dropping NOT VALID. Dropping NOT VALID here would be the wrong trade: this is the same columnstore hypertable as the Up, so validation would scan every chunk of all history on top of the AccessExclusiveLock, making the rollback far slower exactly when you want it fast. Keep NOT VALID and make the comment executable instead:

-- +migrate Down

-- Rollback requires no BLEND_* rows: the re-narrowed CHECKs below are NOT VALID
-- (cheap, no hypertable scan), so they cannot reject pre-existing rows. Fail
-- loudly here rather than leaving rows the restored constraint forbids.
DO $$
BEGIN
  IF EXISTS (
    SELECT 1 FROM state_changes
    WHERE state_change_category IN (
      'BLEND_SUPPLY', 'BLEND_COLLATERAL', 'BLEND_DEBT', 'BLEND_AUCTION',
      'BLEND_EMISSIONS', 'BLEND_BACKSTOP_EMISSIONS', 'BLEND_BACKSTOP',
      'BLEND_BACKSTOP_QUEUE')
  ) THEN
    RAISE EXCEPTION 'rollback requires no BLEND_* state_changes rows; delete them first';
  END IF;
END $$;

This should be cheap in the expected case: it's EXISTS with equality predicates on state_change_category, which is what the columnstore's bloom(state_change_category) sparse index (2025-06-10.4-statechanges.sql:56) is built to serve, so most chunks should be skipped without decompression, and it short-circuits on the first hit. Worth timing alongside the Up — I don't want to overclaim bloom's selectivity without a number.

Separately, on the Up direction. Lines 10-20 and 22-29 permanently downgrade constraints that were VALID — they were declared inline at table creation (2025-06-10.4-statechanges.sql:9-15), so convalidated = true — to NOT VALID, with no follow-up VALIDATE step. Writes are still checked so it's mostly cosmetic, but the table then permanently carries two never-validated constraints and nobody can cheaply assert the historical data conforms. If the Up turns out to be fast against a chunked columnstore table, the tidiest outcome is dropping NOT VALID from the Up (one-time scan, acceptable) while keeping it on the Down plus the guard above.

Also minor: none of the four DROP CONSTRAINT statements has IF EXISTS, so neither direction is re-runnable.

@aditya1702
aditya1702 force-pushed the blend/pr2-schema-models branch from 0965d80 to 8f1dab7 Compare August 7, 2026 14:25
@aditya1702

Copy link
Copy Markdown
Contributor Author

Re: ApplyAuctionAdjustments rounding order (#658 (comment)) — both added in 707a6ac: the discriminating subtest essentially as suggested (11/21 and −11/−21 pinned, with the note that moving trunc inside the GROUP BY flips the expectations to 10/20) and the rounding-order paragraph on applyAuctionAdjustmentsSQL. Not over-documenting a stroop — the existing "summing is exact" sentence invited exactly the wrong inference about the converted underlying, so the distinction between aggregation safety and rounding order needed stating.

@aditya1702

Copy link
Copy Markdown
Contributor Author

Re: the 2026-07-08.0 Down precondition (#658 (comment)) — guard added in 8f1dab7: a single DO block probing both the category and reason lists (both are bloom-indexed per 2025-06-10.4, so conforming chunks skip without decompression), raising before any constraint is touched. All four DROP CONSTRAINTs gained IF EXISTS. The Up keeps NOT VALID per the columnstore-scan tradeoff you outlined — the permanent convalidated = false cosmetic stands, traded against an AccessExclusiveLock-held scan of all history. The migrate round-trip test exercises the Down (empty table → guard passes; the exception path is the enforcement).

@aditya1702
aditya1702 force-pushed the blend/pr2-schema-models branch from 8f1dab7 to 2c85d31 Compare August 7, 2026 17:32
@aditya1702
aditya1702 force-pushed the blend/pr2-schema-models branch from 2c85d31 to 31a4a50 Compare August 7, 2026 18:17
@aditya1702 aditya1702 self-assigned this Aug 11, 2026
@aditya1702
aditya1702 force-pushed the blend/pr2-schema-models branch from 31a4a50 to f71c79e Compare August 14, 2026 18:18
…sons

Blend state changes follow the core convention: category names the
on-chain object (BLEND_SUPPLY, BLEND_COLLATERAL, BLEND_DEBT,
BLEND_AUCTION, BLEND_EMISSIONS, BLEND_BACKSTOP_EMISSIONS,
BLEND_BACKSTOP, BLEND_BACKSTOP_QUEUE) and reason names the action.
Amount-bearing categories reuse the generic CREDIT/DEBIT/ADD/REMOVE/
BURN verbs; only BORROW, REPAY, FLASH_LOAN, BAD_DEBT, FILL, and CLAIM
are added.
…keys

Postgres UPDATE ... FROM applies only one matching source row per target
row, so duplicate (pool, user, asset) keys in a batch silently dropped
deltas. Net deltas reject duplicates (ZeroBorrowed makes merging
order-dependent; the processor pre-aggregates); auction adjustments are
purely additive and are summed server-side before applying.
…odels

Add blend_pool_claimed (pool, user) and blend_backstop_claimed (user) tables
plus PoolClaimedModel/BackstopClaimedModel with additive BatchApplyDeltas. These
hold lifetime claimed BLND / Comet LP totals, folded from claim events during
current-state indexing — the only pass that sees every claim since Blend's first
ledger. Mirrors the net_supplied/net_borrowed cost-basis accumulator.
…account_id

Both tables are read by GetByAccount (the per-user positions path) filtering on
user_account_id, which is the second PK column and so cannot use the primary key.
Add single-column B-tree indexes mirroring idx_blend_positions_user, keeping the
index defined alongside its table in the same migration.
ApplyAuctionAdjustments converted protocol tokens to underlying with exact
numeric division, leaving a fractional tail (e.g. "1100.0000000000000000") in
net_supplied/net_borrowed while every other write to those columns stores floored
integer text. The contract uses fixed_mul_floor (floor of the positive magnitude).
Wrap the conversion in trunc(): for the signed lot/bid deltas, trunc toward zero
reproduces floor-of-magnitude-with-sign (trunc(-366.3) = -366, not floor's -367).
Adds a subtest covering a fractional product on both the positive (filler) and
negative (liquidated user) sides.
… writers

BatchApplyNetDeltas and ApplyAuctionAdjustments mutate existing rows only; a delta
for a not-yet-inserted position row silently no-ops. Document that callers must
upsert the Positions snapshot (and reserves) first, as PersistCurrentState does.
The full-snapshot upsert/zero writers overwrote last_modified_ledger outright while
the additive writers (net deltas, claimed, reserve data, reward zone) already use
GREATEST. Switch the snapshot writers to GREATEST(<table>.last_modified_ledger,
EXCLUDED/u.ledger) too, so the column never moves backward and every writer treats
it uniformly. Behavior is unchanged under the strictly ledger-ordered persist path.
StartBlock and LastModifiedLedger were int32 while every other blend row struct
uses uint32 for ledger-valued fields, casting to int32 only at the write boundary.
Align Auction with that convention.
…ments

Flooring doesn't distribute over addition, so n fills folded into one
(pool, user, asset) row overstate magnitude by at most n-1 stroops versus
per-fill flooring. The existing duplicate-key test uses amounts exact at
the fixture rates, so it cannot tell the two orders apart; this one can
(two fills of 5 at b_rate 1.1: 11 aggregated vs 10 per-fill). The doc on
applyAuctionAdjustmentsSQL states why the single trunc on the summed
delta is kept: the skew is strictly below the end-of-window rate
approximation already accepted on the same display-only fields.
The Down re-adds the narrowed CHECKs as NOT VALID, which is precisely the
modifier that skips checking existing rows — so the stated precondition
(no BLEND_* rows) was a comment, not a guarantee, and a rollback over
Blend data left rows the restored constraints forbid: unwritable, and
fatal to whole stateChanges pages in the pre-Blend resolver. A DO block
now probes both the category and reason lists (bloom sparse indexes serve
both) and raises before touching any constraint. All four DROP
CONSTRAINTs gain IF EXISTS so both directions are re-runnable. The Up
keeps NOT VALID: validating would scan every columnstore chunk under an
AccessExclusiveLock.
@aditya1702
aditya1702 force-pushed the blend/pr2-schema-models branch from f71c79e to ef82fb9 Compare August 17, 2026 19:04
@aditya1702
aditya1702 merged commit 62dbc69 into main-blend Aug 17, 2026
12 checks passed
@aditya1702
aditya1702 deleted the blend/pr2-schema-models branch August 17, 2026 20:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants