​

Components

BlockDB’s capabilities come from a set of gRPC services and background services, all of which can be composed flexibly through cmd/blockdb’s -services flag.

​

meta-service

meta-service is a standalone metadata process responsible for the lifecycle and schema definitions of logical tables: The core read and write services fetch table metadata from meta-service through a Meta client (pkg/blockdb/meta) and hold no physical schema information themselves.

​

read

Provides row-level reads for regular tables, implementing TableReadService:

• GetRow: read a single row by business primary key.
• BatchGetRows: read multiple primary keys in one RPC.
• FilterRows: filter rows by condition, with sorting and pagination.

It is the entry point for regular-table read semantics.

​

block-read

Provides block-oriented reads for Block tables, implementing BlockReadService and L2BlockReadService:

• GetEvent: read events at a block.
• GetState: read a state snapshot at a block, automatically injecting height ≤ target height.
• GetConsensusHeight: get the table’s current consensus height.
• GetBlockRows: get all rows at a given block height.

It is the entry point for block read semantics and supports historical height lookups.

​

scan

Provides full-table streaming scans, implementing TableScanService:

• Scan: accepts a SQL query and returns result rows as a stream, avoiding memory blowups.

Use it for large offline reads. Because it is resource-intensive, deploy it standalone so it does not affect online reads and writes.

​

write

The entry point for real-time writes to all logical tables, aggregating TableWriteService, BlockWriteService, and TimeWriteService in one service:

• Regular tables: UpsertRows / DeleteRows.
• Block tables: UpsertBlock.
• Time tables: UpsertTimeRows.

Write requests are enqueued in an in-memory async queue (asyncqueue) and flushed by multiple threads, delivering near real-time write responses while the queue capacity protects the database from overload. See Write architecture.

​

batch-write

Manages asynchronous batch write jobs for massive datasets, implementing BatchWriteService:

• InitWriteJob: initialize a write job and return an S3 presigned upload URL plus a job ID.
• CommitWriteJob: commit the job, triggering background worker processing.
• CancelWriteJob: cancel a job.

You stage millions of rows through S3; submission throughput is unbounded and ingestion throughput reaches hundreds of thousands to over a million rows per second. See Write architecture.

​

batch-worker

The background worker that executes batch write jobs, paired with batch-write:

• Consumes the job files already staged in S3.
• Performs sorting, dedup, and batched ingestion (see Write architecture).

Deploy it standalone so it stays isolated from real-time writes and can scale independently.

​

subscribe

Provides streaming block event subscription, implementing BlockSubscribeService:

• Subscribe: subscribe to block write events for one or more Block tables.

Its event source is the BlockSubscribeEvent published by the write service through Kafka (eventbus). Subscribers receive events as a gRPC stream, driving downstream incremental consumption. Because it maintains long-lived connections, deploy it standalone.

​

state-compactor

A periodic background task that recomputes the state latest table by scanning _archive, keeping the latest-state table consistent with the archived data. It must run as a single instance to avoid concurrent write conflicts.

​

Core code packages

The core packages behind these services (under pkg/blockdb/):