Skip to content

gnd(test): Add mock-based subgraph test runner#6361

Draft
dimitrovmaksim wants to merge 16 commits intographprotocol:masterfrom
dimitrovmaksim:feat/gnd-cli-test-rewrite
Draft

gnd(test): Add mock-based subgraph test runner#6361
dimitrovmaksim wants to merge 16 commits intographprotocol:masterfrom
dimitrovmaksim:feat/gnd-cli-test-rewrite

Conversation

@dimitrovmaksim
Copy link
Member

@dimitrovmaksim dimitrovmaksim commented Feb 13, 2026

Summary

Implements a native test runner for gnd test that executes JSON-defined tests through real graph-node infrastructure (store, WASM runtime, trigger processing) with only the blockchain layer mocked. Provides a faster, integrated alternative to Matchstick while maintaining backward compatibility.

Key Features

  • JSON test format - Define blocks, events, eth_calls, and GraphQL assertions
  • Real graph-node pipeline - Tests run through actual indexing infrastructure
  • Auto block triggers - All block handler filters work (once, polling, no filter)
  • startBlock support - Auto-extracts from manifest and numbers blocks accordingly
  • eth_call mocking - Pre-populate call cache before indexing
  • Smart assertions - Order-insensitive comparison, type coercion, detailed diffs
  • Database isolation - Fresh pgtemp database per test
  • Backward compatible - --matchstick flag preserves legacy behavior

Architecture

JSON test → Parse → ABI encode events → StaticStreamBuilder (mock stream)
  → Real graph-node indexing (WASM + triggers + storage)
  → GraphQL assertions → Report results

Design decisions:

  • Reuses actual graph-node stores, runtime, and trigger processing
  • Only blockchain RPC is mocked (via StaticStreamBuilder)
  • Fresh isolated database per test (pgtemp)
  • Pre-populated eth_call cache for deterministic results

Usage

gnd test                    # Run all tests in tests/
gnd test tests/foo.json     # Run specific test
gnd test --skip-build       # Skip auto-build
gnd test --matchstick       # Legacy Matchstick mode
gnd test --help             # To see usage and all options 

Test Format Example

{
  "name": "Transfer creates entity",
  "blocks": [
    {
      "number": 1,
      "events": [
        {
          "address": "0x1234...",
          "event": "Transfer(address indexed from, address indexed to, uint256 value)",
          "params": {"from": "0xaaaa...", "to": "0xbbbb...", "value": "1000"}
        }
      ],
      "ethCalls": [
        {
          "address": "0x1234...",
          "function": "balanceOf(address)(uint256)",
          "params": ["0xaaaa..."],
          "returns": ["1000000000000000000"]
        }
      ]
    }
  ],
  "assertions": [
    {
      "query": "{ transfer(id: \"1\") { from to value } }",
      "expected": {"transfer": {"from": "0xaaaa...", "to": "0xbbbb...", "value": "1000"}}
    }
  ]
}

Implementation

Modified Graph-Node Files

  • graph/src/data_source/subgraph.rs - Minor exports for startBlock support

Supported Features

Feature Status
Log events ✅ Supported
Block handlers (once, polling, no filter) ✅ Supported
eth_call mocking ✅ Supported
Dynamic/template data sources ⚠️ Not Tested
Transaction receipts ❌ Not implemented
File data sources / IPFS ❌ Not implemented
Call triggers (traces) ❌ Not implemented

Breaking Changes

None. Adds native testing while preserving --matchstick for backward compatibility.

Code Attribution

⚠️ Almost all implementation code was generated by Claude Code through investigation of graph-node's indexing pipeline, existing test infrastructure, and iterative design refinement.

Human contributions: Initial requirements, code review, validation, and bug fixes.

Future Work

  • Configurable timeout (env var or CLI flag)
  • Multi-datasource startBlock warning
  • Test validation (block ordering, address format)
  • --json CI output format
  • Parallel test execution
  • Test filtering (--filter)

Checklist

  • Implementation complete
  • User documentation (776-line README)
  • Backward compatible via --matchstick
  • Validated with real subgraphs
  • All block handler filters supported
  • startBlock handling working
  • No breaking changes

Replaces monolithic gnd/src/commands/test.rs with organized test/
directory containing:
- mod.rs: Main entry point and test orchestration
- runner.rs: Test execution and infrastructure setup
- assertion.rs: GraphQL assertion logic
- block_stream.rs: Mock block stream implementation
- noop.rs: Stub trait implementations
- schema.rs: JSON schema and test types
- trigger.rs: ABI encoding for test triggers
- output.rs: Test result formatting
- mock_chain.rs: Block pointer helpers

Updates main.rs to make Test command async (.await).
Adds dependencies for test runner (graph-chain-ethereum,
graph-graphql, graph-store-postgres).
Adds supporting modules for test infrastructure:
- mock_chain: Helpers for block pointer construction
- schema: JSON schema types and parsing
- output: Console output formatting
- trigger: ABI encoding of test triggers
Adds module declarations for refactored components:
- mod assertion
- mod block_stream
- mod noop

Updates module documentation to reflect the new structure and improved
separation of concerns.
Removes ~500 lines from runner.rs by delegating to new focused modules:
- block_stream: Mock block delivery infrastructure
- noop: Stub trait implementations
- assertion: GraphQL assertion logic

runner.rs now focuses exclusively on test orchestration:
- setup_stores: Initialize PostgreSQL and chain store
- setup_chain: Construct mock Ethereum chain
- setup_context: Wire up graph-node components
- wait_for_sync: Poll store until indexing completes

Reduced from 1198 to 729 lines (39% reduction).
Improves readability by separating concerns.
Moves assertion execution to gnd/src/commands/test/assertion.rs:
- run_assertions: Execute all test assertions
- run_single_assertion: Execute and compare a single query
- r_value_to_json: Convert graph-node's r::Value to serde_json
- json_equal: Compare JSON with string-vs-number coercion

Makes TestContext fields pub(super) to allow assertion module access.
Moves unused adapter stubs to gnd/src/commands/test/noop.rs:
- StaticBlockRefetcher
- NoopRuntimeAdapter / NoopRuntimeAdapterBuilder
- NoopAdapterSelector
- NoopTriggersAdapter

These satisfy Chain constructor trait bounds but are never called during
normal test execution since triggers are pre-built and host functions
are not available in mocks.
- Add baseFeePerGas field to TestBlock schema
- Parse and apply base fee when creating test blocks
- Replace graph-node helper functions with direct alloy types
- Extract dummy_transaction creation into dedicated function
- Use alloy Block::empty() constructor for cleaner block creation
- Rename 'triggers' field to 'events' in TestBlock
- Remove TestTrigger enum and BlockTrigger type
- Keep LogEvent as the only event type users specify
- Auto-inject Start and End block triggers for every block
- This ensures block handlers fire correctly without explicit config
- Update docs to reflect that block triggers are automatic
- Extract min startBlock from manifest in extract_start_block_from_manifest()
- Use startBlock as default test block numbering base
- Create start_block_override to bypass on-chain validation
- Pass override through setup_context() to SubgraphRegistrar
- This allows testing subgraphs that specify startBlock without needing a real chain
Signed-off-by: Maksim Dimitrov <dimitrov.maksim@gmail.com>
Signed-off-by: Maksim Dimitrov <dimitrov.maksim@gmail.com>
Signed-off-by: Maksim Dimitrov <dimitrov.maksim@gmail.com>
Signed-off-by: Maksim Dimitrov <dimitrov.maksim@gmail.com>
Signed-off-by: Maksim Dimitrov <dimitrov.maksim@gmail.com>
  - Replace positional `manifest` arg with `--manifest` / `-m` flag (default: subgraph.yaml)
  - Add positional `tests` args accepting file or directory
  - When no args given, default to scanning `tests/`
  - Bare filenames resolve to `tests/<filename>` for convenience (e.g., `gnd test foo.json` → `tests/foo.json`)
  - Remove `--test-dir` flag (replaced by positional args)
  - Update README with new usage examples

Signed-off-by: Maksim Dimitrov <dimitrov.maksim@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant