Skip to main content

​
Architecture

BlockDB is a logical database layer. It consists of a set of independent gRPC services that can either be assembled into a single stateless blockdb process or split into separate services for independent deployment and scaling.

​
Multi-service design

BlockDB splits its capabilities into several services that can be started independently. The core process cmd/blockdb selects which services to start with the -services flag:

​
Metadata service

cmd/meta-service is a standalone metadata process responsible for the lifecycle and definitions of logical tables. It contains three sub-services: TableMetaService (regular tables), BlockMetaService (Block tables), and TimeMetaService (Time tables). The core services read table metadata through a Meta client and hold no physical schema information themselves.

​
Integrated or standalone deployment

The same logical database can be composed freely depending on your scenario:
  • In-process integration: put several services in one blockdb process, sharing dependencies locally. Convenient for development and small to medium deployments.
  • Standalone deployment: split services into separate processes by resource profile, each scaling horizontally and operated independently.
Composing or splitting services in a process does not change the external API — only the runtime shape. The current recommended production layout groups services by read / write / background boundaries:
  • All read APIs except scan in one binary: read + block-read.
  • All write APIs in one service: write.
  • scan and subscribe deployed standalone: keeps large scans from dragging down the read services and long-lived subscription connections from affecting others.
  • Batch write workers standalone: batch-worker scales independently and stays isolated from real-time reads and writes.

​
Underlying dependencies

As a logical layer, BlockDB does not manage data files itself. It depends on:
  • Physical database: a MySQL-protocol physical online database; BlockDB performs the actual reads and writes through standard SQL.
  • Redis: used as a cache (for example the block read cache).
  • Kafka: the event bus that distributes block write events and powers subscriptions.
  • Object storage: staging for batch writes, plus storage for block bundles and similar files.

​
Design highlights

  • Stateless: the core service processes hold no local state, so they can be composed freely and scaled horizontally.
  • Metadata separated from compute: meta-service manages table metadata while the core read/write services stay stateless.
  • Split by resource profile: reads, writes, scans, and subscriptions are split into different services according to their resource usage and concurrency patterns, making independent tuning and scaling possible.
For each module’s directory and responsibility, see Services and components; for the batch write flow, see Write architecture.