Skip to content

Extract raw event and ecosystem-specific logic into pluggable interfaces - #1335

Merged
DZakh merged 5 commits into
mainfrom
claude/dreamy-cray-vdlt4s
Jun 18, 2026
Merged

Extract raw event and ecosystem-specific logic into pluggable interfaces#1335
DZakh merged 5 commits into
mainfrom
claude/dreamy-cray-vdlt4s

Conversation

@DZakh

@DZakh DZakh commented Jun 18, 2026

Copy link
Copy Markdown
Member

Summary

Refactors event handling to decouple ecosystem-specific logic from storage and logging concerns. Introduces a new Ecosystem.t interface with three pluggable functions (toEvent, toEventLogger, toRawEvent) that each ecosystem (EVM, Fuel, SVM) implements, replacing hardcoded logic in PgStorage and Logging.

Key Changes

  • New RawEvent module: Extracted makeRawEvent and convertFieldsToJson from PgStorage into a dedicated module. The make function now accepts a cleanUpRawEventFieldsInPlace callback instead of accessing config.ecosystem directly.

  • Ecosystem interface expansion: Added three new fields to Ecosystem.t:

    • toEvent: Materializes the user-facing event from an item's opaque payload
    • toEventLogger: Builds per-item child loggers with ecosystem-specific fields (contract/event/address for EVM/Fuel; program/instruction/programId for SVM)
    • toRawEvent: Builds raw event rows for the raw_events table (throws on SVM)
  • Ecosystem implementations: Updated Evm, Fuel, and Svm modules to be factory functions (make(~logger)) that return configured Ecosystem.t instances, closing over the injected logger.

  • Event payload abstraction: Renamed event field to payload in Internal.eventItem and introduced eventPayload as an opaque type. Added payloadToEvent and payloadToGenericEvent functions to materialize from the payload.

  • Logging refactor: Replaced inline event logger construction in Logging.getItemLogger with a callback (eventLoggerMaker) set by Config.fromPublic. This breaks the circular dependency between Logging and Ecosystem.

  • Raw event type relocation: Moved RawEvents.t type definition from InternalTable to Internal so ecosystem implementations can reference it without pulling in Config dependencies.

  • Snapshot updates: Updated test snapshots to reflect field name changes in logging output (contractNamecontract, eventNameevent).

Notable Implementation Details

  • The eventPayload type uses %identity externals to maintain runtime shape compatibility while providing type safety at the boundary.
  • Ecosystem factories are called once during Config.fromPublic, with the logger injected at that point. The toEventLogger function closes over this logger, enabling future removal of the global logger.
  • RawEvent.make is called from Evm.toRawEvent and Fuel.toRawEvent; Svm.toRawEvent throws since SVM doesn't support raw events.
  • Test files updated to call Ecosystem.make(~logger) instead of accessing a static ecosystem value.

https://claude.ai/code/session_0124SBjKuB5vfwcoJyt5k3v4

Summary by CodeRabbit

Release Notes

  • Refactor

    • Made event processing and filtering consistently derive event details from a payload-based flow.
    • Updated logging to be ecosystem-aware across fetching, preprocessing, handler execution, and contract registration.
    • Improved raw-event handling: raw-event serialization now follows ecosystem-specific conversion and persists fields using snake_case naming (with optional field cleanup).
  • Tests

    • Updated fixtures, mocks, and assertions to reflect the new payload-based event representation and ecosystem schema wiring.

Items now carry an opaque `Internal.eventPayload` instead of the
user-facing `event`. The ecosystem materialises what each consumer needs
from that payload via three new `Ecosystem.t` methods:

- `toEvent`     — the event handed to handlers / contract registration
- `toEventLogger` — the per-item child logger (EVM/Fuel: contract/event/
  address; SVM: program/instruction/programId), built off a logger
  injected into the ecosystem constructor
- `toRawEvent`  — the raw_events row (EVM/Fuel; throws on SVM)

This removes consumers' dependence on the event shape (`toGenericEvent`
is gone) and gives sources a seam to later return a compact/lazy
representation of heavy fields like `transaction.input`.

The raw_events row type moves to `Internal.rawEvent` (aliased by
`InternalTable.RawEvents.t`) so the ecosystem can reference it without
pulling in `InternalTable`'s dependency on `Config`. Raw-event assembly
moves to the new `RawEvent` module.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124SBjKuB5vfwcoJyt5k3v4
@coderabbitai

coderabbitai Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 92ac1322-55d1-47b5-a2ab-25cf579e69ed

📥 Commits

Reviewing files that changed from the base of the PR and between 0ee9c07 and ae813c6.

📒 Files selected for processing (4)
  • packages/envio/src/Config.res
  • scenarios/test_codegen/test/lib_tests/FetchState_onBlock_test.res
  • scenarios/test_codegen/test/lib_tests/FetchState_test.res
  • scenarios/test_codegen/test/lib_tests/PgStorage_test.res
📝 Walkthrough

Walkthrough

Refactors internal event representation from direct event fields to opaque eventPayload. Extends Ecosystem.t with event transformation handlers. Converts EVM/Fuel/SVM static ecosystems to logger-injected make factories. Creates RawEvent module for serialization. Updates all event sources, consumers, loaders, and user context to use ecosystem-aware functions and payload-based events. Comprehensive test suite updates throughout.

Changes

Payload-based event pipeline and ecosystem-aware logging

Layer / File(s) Summary
Core internal types: eventPayload, rawEvent, eventItem, and RawEvents
packages/envio/src/Internal.res, packages/envio/src/db/InternalTable.res
Introduces opaque eventPayload type replacing fromGenericEvent/toGenericEvent externals with payloadToEvent identity. Updates eventItem.payload and item.Event variant to carry payload: eventPayload. Adds rawEvent row type for raw_events table with chain/event identifiers, addresses, and JSON-serialized block/transaction/params. Aliases RawEvents.t to Internal.rawEvent with snake_case DB column schema.
Ecosystem interface: toEvent, toEventLogger, toRawEvent handlers
packages/envio/src/Ecosystem.res
Extends Ecosystem.t with base logger and three event transformation handlers. Adds memoized accessors: getItemEvent caches materialized events on items under _event key, getItemLogger caches per-item loggers under _logger key (delegating to toEventLogger for events or creating block-scoped loggers via Logging.createChildFrom), getItemUserLogger wraps cached logger for user context.
RawEvent module: JSON conversion and row construction
packages/envio/src/RawEvent.res
New module with convertFieldsToJson helper (stringifies top-level bigints, maps None to empty object) and make function computing eventId from logIndex and blockNumber, serializing block/transaction to JSON-compatible dicts, invoking cleanup callback on block_fields, converting params via event schema with null special-casing to "null" string.
EVM, Fuel, SVM make factories with event transformation wiring
packages/envio/src/sources/Evm.res, packages/envio/src/sources/Fuel.res, packages/envio/src/sources/Svm.res
Replaces static ecosystem bindings with make(~logger: Pino.t): Ecosystem.t factories. Each defines concrete payload types with fromPayload/toPayload conversions and wires toEvent (payload→user event via payloadToEvent), toEventLogger (structured child logger with contract/event/chainId/block/log metadata), and toRawEvent (via RawEvent.make with cleanup; SVM throws unsupported).
Logging injectable logger and Config ecosystem wiring
packages/envio/src/Logging.res, packages/envio/src/Config.res
Refactors Logging with inline logAtLevel helper (extracts logger function by level from Pino instance) and userLogger constructor (accepts pre-built child logger, routes info/debug/warn/error through logAtLevel with custom u* levels, maps errorWithExn to prettified exception). Config.fromPublic constructs fresh ecosystems via Evm.make(~logger), Fuel.make(~logger), Svm.make(~logger) with logger from Logging.getLogger(), replacing prior singletons.
Event sources: payload field and per-ecosystem conversions
packages/envio/src/sources/HyperSyncSource.res, packages/envio/src/sources/HyperFuelSource.res, packages/envio/src/sources/RpcSource.res, packages/envio/src/sources/SvmHyperSyncSource.res, packages/envio/src/SimulateItems.res
All sources construct Internal.Event with payload field using per-ecosystem conversions: Evm.fromPayload for EVM/HyperSync/RPC, Fuel.fromPayload for Fuel, Internal.eventPayload for SVM, replacing prior event field and fromGenericEvent/fromEvmEventPayload conversions.
EventProcessing, FetchState, PgStorage, ChainFetching, ContractRegisterContext event consumers
packages/envio/src/EventProcessing.res, packages/envio/src/FetchState.res, packages/envio/src/PgStorage.res, packages/envio/src/ChainFetching.res, packages/envio/src/ContractRegisterContext.res
EventProcessing calls config.ecosystem.toEvent(item) for handler and preload event arguments. FetchState extracts payload field for clientAddressFilter invocation. PgStorage delegates raw event construction to config.ecosystem.toRawEvent(item) (removes local convertFieldsToJson and makeRawEvent). ContractRegisterContext uses Ecosystem.getItemEvent for returned event and Ecosystem.getItemUserLogger for log accessor. ChainFetching logs via Ecosystem.getItemLogger in sync and async contract-register error paths.
UserContext and LoadLayer: ecosystem-aware logging and loading
packages/envio/src/UserContext.res, packages/envio/src/LoadLayer.res, packages/envio/src/LoadLayer.resi
Threads ~ecosystem: Ecosystem.t through LoadLayer.loadById, loadByFilter, loadEffect and public signatures. Updates all error/cache/trace logging to use Ecosystem.getItemLogger(item, ~ecosystem). UserContext passes params.config.ecosystem to all LoadLayer calls (effect/entity get/getWhere/getOrThrow/getOrCreate), derives user logger via Ecosystem.getItemUserLogger in EffectContext.log and handlerTraps, uses Ecosystem.getItemLogger for raised-logger construction.
EventConfigBuilder: address filter callback parameter type
packages/envio/src/EventConfigBuilder.res
Updates compileAddressFilter and buildAddressFilter signatures to accept Internal.eventPayload instead of Internal.event as first filter parameter. Filter body logic (using event.params, blockNumber, indexingAddresses) remains unchanged.
Test suite: payload shapes, ecosystem factories, and assertions
scenarios/test_codegen/test/*
Mock event construction switches from event to payload with per-ecosystem conversions across all test files. Schema access switches from *.ecosystem.* to *.make(~logger=Logging.getLogger()).* in address-filter, event-block-filter, and onBlock tests. LogTesting fixture creates ecosystem via Evm.make. PgStorage_test asserts via ecosystem.toRawEvent with snake_case field expectations and bigint stringification. LoadLayer_test passes ~ecosystem=MockIndexer.config.ecosystem to all loader calls. Consistent mock event shape updates across fixtures, helpers, and assertions.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • enviodev/hyperindex#1275: Both PRs modify packages/envio/src/EventProcessing.res handler event derivation—main PR updates to config.ecosystem.toEvent(item), retrieved PR refactors event-batch persistence flow.
  • enviodev/hyperindex#1306: Both PRs thread ecosystem/logging through LoadLayer loaders—main PR adds ~ecosystem parameter to loadById/loadByFilter/loadEffect, retrieved PR refactors loaders to operate on EntityFilter.t.
  • enviodev/hyperindex#1329: Both PRs update address-filter predicates and FetchState.handleQueryResult—main PR adapts filter callbacks to consume Internal.eventPayload instead of Internal.event, retrieved PR implements the address-filter feature itself.
  • enviodev/hyperindex#1332: Both PRs touch packages/envio/src/ChainFetching.res contract-register orchestration—main PR changes failure logger to use Ecosystem.getItemLogger, retrieved PR introduces per-chain-chainState refactor affecting the same error paths.

Suggested reviewers

  • JonoPrest
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title accurately captures the main refactoring objective: extracting raw event logic and ecosystem-specific handling into pluggable interfaces that decouple storage and logging concerns.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (1)
packages/envio/src/RawEvent.res (1)

40-51: 💤 Low value

Workaround for null params serializes to string "null" instead of JSON null.

The special case converts null params to the string literal "null", which will be stored differently than actual JSON null in the database. The comment acknowledges this is a workaround. If consumers expect actual JSON null, they may need adjustment.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/envio/src/RawEvent.res` around lines 40 - 51, The code currently
converts null params to the string literal "null" instead of preserving them as
actual JSON null, creating inconsistency in database storage. To fix this,
modify the eventConfig.paramsRawEventSchema to properly support nullable params,
then remove the special case condition checking if params equals null and the
workaround that converts it to the string "null". This will allow actual null
values to pass through naturally from the reverseConvertOrThrow call, ensuring
consumers receive consistent JSON null values rather than string literals.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/envio/src/PgStorage.res`:
- Line 846: The code at line 846 calls config.ecosystem.toRawEvent() without
verifying that the ecosystem type is not SVM, even though SVM's toRawEvent
implementation throws an error for raw events. The config.enableRawEvents guard
at line 843 only controls whether to process raw events, not whether the
ecosystem supports them. Fix this by either adding validation during config
parsing to reject enableRawEvents: true when ecosystem is SVM, or adding an
explicit ecosystem type check before calling config.ecosystem.toRawEvent() at
line 846 to ensure SVM ecosystems are excluded. Choose whichever approach aligns
better with your codebase design patterns.

In `@scenarios/test_codegen/test/lib_tests/FetchState_onBlock_test.res`:
- Line 62: The payload field assignment uses Utils.magic without explicit type
annotations, which violates the coding guidelines for type casting in ReScript.
Modify the payload field on line 62 to add an explicit type annotation using the
pipe operator syntax. The Utils.magic call should be annotated with both the
input type (the string "Mock event in fetchstate test") and the output type it
should be cast to, following the pattern: value->(Utils.magic: inputType =>
outputType). Ensure the type annotation clearly specifies what type the mock
string is being converted to.

In `@scenarios/test_codegen/test/lib_tests/FetchState_test.res`:
- Line 71: The payload field uses an untyped Utils.magic call on line 71 in
FetchState_test.res, which violates the coding guideline requiring explicit type
annotations. Change the call from Utils.magic("Mock event in fetchstate test")
to use the explicit cast syntax pattern value->(Utils.magic: inputType =>
outputType), where inputType should be string (the type of the input string
literal) and outputType should be the expected type of the payload field based
on the context of the mock event structure.

In `@scenarios/test_codegen/test/lib_tests/PgStorage_test.res`:
- Line 1065: The Utils.magic cast on the line with pattern `_ =>
Internal.eventPayload` uses a wildcard for the input type instead of an explicit
type annotation. Replace the wildcard `_` with the actual explicit input type
that is being cast to Internal.eventPayload, following the pattern
`(Utils.magic: inputType => Internal.eventPayload)` to comply with ReScript
coding guidelines requiring explicit type annotations for Utils.magic casts.

---

Nitpick comments:
In `@packages/envio/src/RawEvent.res`:
- Around line 40-51: The code currently converts null params to the string
literal "null" instead of preserving them as actual JSON null, creating
inconsistency in database storage. To fix this, modify the
eventConfig.paramsRawEventSchema to properly support nullable params, then
remove the special case condition checking if params equals null and the
workaround that converts it to the string "null". This will allow actual null
values to pass through naturally from the reverseConvertOrThrow call, ensuring
consumers receive consistent JSON null values rather than string literals.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 5418d4c6-8615-4cad-9a72-ce57626177ea

📥 Commits

Reviewing files that changed from the base of the PR and between dfb5b1d and c84f0ba.

⛔ Files ignored due to path filters (4)
  • scenarios/test_codegen/test/__snapshots__/Logging.both-prettyconsole.snap is excluded by !**/*.snap
  • scenarios/test_codegen/test/__snapshots__/Logging.console-pretty.snap is excluded by !**/*.snap
  • scenarios/test_codegen/test/__snapshots__/Logging.console-raw.snap is excluded by !**/*.snap
  • scenarios/test_codegen/test/__snapshots__/Logging.ecs-console.snap is excluded by !**/*.snap
📒 Files selected for processing (31)
  • packages/envio/src/Config.res
  • packages/envio/src/ContractRegisterContext.res
  • packages/envio/src/Ecosystem.res
  • packages/envio/src/EventProcessing.res
  • packages/envio/src/FetchState.res
  • packages/envio/src/Internal.res
  • packages/envio/src/Logging.res
  • packages/envio/src/PgStorage.res
  • packages/envio/src/RawEvent.res
  • packages/envio/src/SimulateItems.res
  • packages/envio/src/db/InternalTable.res
  • packages/envio/src/sources/Evm.res
  • packages/envio/src/sources/Fuel.res
  • packages/envio/src/sources/HyperFuelSource.res
  • packages/envio/src/sources/HyperSyncSource.res
  • packages/envio/src/sources/RpcSource.res
  • packages/envio/src/sources/Svm.res
  • packages/envio/src/sources/SvmHyperSyncSource.res
  • scenarios/test_codegen/test/ClientAddressFilter_test.res
  • scenarios/test_codegen/test/EventBlockFilter_test.res
  • scenarios/test_codegen/test/IndexerState_test.res
  • scenarios/test_codegen/test/OnBlockSchema_test.res
  • scenarios/test_codegen/test/SvmHyperSyncSource_test.res
  • scenarios/test_codegen/test/__mocks__/MockEvents.res
  • scenarios/test_codegen/test/fixtures/LogTesting.res
  • scenarios/test_codegen/test/helpers/MockIndexer.res
  • scenarios/test_codegen/test/lib_tests/FetchState_onBlock_test.res
  • scenarios/test_codegen/test/lib_tests/FetchState_test.res
  • scenarios/test_codegen/test/lib_tests/PgStorage_test.res
  • scenarios/test_codegen/test/lib_tests/SameSignatureEventDecode_test.res
  • scenarios/test_codegen/test/rollback/ChainMocking.res

Comment thread packages/envio/src/PgStorage.res
Comment thread scenarios/test_codegen/test/lib_tests/FetchState_onBlock_test.res Outdated
Comment thread scenarios/test_codegen/test/lib_tests/FetchState_test.res Outdated
Comment thread scenarios/test_codegen/test/lib_tests/PgStorage_test.res Outdated
…stem-threaded loggers

Addresses PR review feedback on the item/event decoupling:

- Memoise the materialised event on the item (mirrors the logger cache),
  via `Ecosystem.getItemEvent`.
- `clientAddressFilter` now operates on the opaque `eventPayload` directly
  instead of casting it to the user-facing event.
- `Internal.rawEvent` uses snake_case fields matching the DB columns, with
  no `@as` indirection.
- Drop the global event-logger-maker ref: `getItemLogger`/`getItemUserLogger`
  move to `Ecosystem` and take `~ecosystem` explicitly, threaded through
  EventProcessing, ContractRegisterContext, UserContext, ChainFetching and
  LoadLayer. The base logger is a field on `Ecosystem.t`.
- Remove the shared `genericEvent` from the payload path: each ecosystem
  converts the opaque payload to its own `evmEventPayload` / `fuelEventPayload`
  (SVM uses its instruction type). `RawEvent.make` takes the extracted block
  and transaction so it stays payload-shape-agnostic.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124SBjKuB5vfwcoJyt5k3v4
`evmEventPayload`/`fuelEventPayload` and their identity casts move out of
`Internal` and into `Evm`/`Fuel` as `payload` + `fromPayload`/`toPayload`,
where the ecosystem-specific shapes belong. `Internal` keeps only the opaque
`eventPayload` (the item field depends on it) and the generic
`payloadToEvent`. Sources and test helpers reference the ecosystem casts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124SBjKuB5vfwcoJyt5k3v4
- Reject `rawEvents: true` for the SVM ecosystem during config parsing,
  so it fails fast with a clear message instead of hitting Svm.toRawEvent's
  throw mid-indexing.
- Add explicit input/output type annotations to the `Utils.magic` casts in
  the FetchState and PgStorage tests, per the ReScript guideline.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0124SBjKuB5vfwcoJyt5k3v4
@coderabbitai

coderabbitai Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Caution

Failed to replace (edit) comment. This is likely due to insufficient permissions or the comment being deleted.

Error details
{}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants