2026-02-15 07:20:28 -05:00
2026-02-15 07:20:28 -05:00
2026-01-20 09:34:33 -05:00
2026-01-20 09:34:33 -05:00
2026-01-20 09:34:33 -05:00
2026-01-20 09:34:33 -05:00
2026-01-20 09:34:33 -05:00
2026-01-20 11:15:44 -05:00
2026-01-20 09:34:33 -05:00
2026-01-20 09:34:33 -05:00
2026-01-20 09:34:33 -05:00
2026-01-20 09:34:33 -05:00

ZynamoDB

A DynamoDB-compatible database built with Zig and RocksDB.

Why Zig?

Zig was chosen over C++ for several reasons:

  1. Built-in Memory Safety - Compile-time safety checks without garbage collection
  2. Seamless C Interop - RocksDB's C API can be imported directly with @cImport
  3. Simple Build System - build.zig replaces complex CMake/Makefile configurations
  4. No Hidden Control Flow - Explicit error handling, no exceptions
  5. Modern Tooling - Built-in test framework, documentation generator, and package manager

Features

Implemented Operations

  • CreateTable
  • DeleteTable
  • DescribeTable
  • ListTables
  • PutItem
  • GetItem
  • DeleteItem
  • Query (basic)
  • Scan

Planned Operations

  • 🚧 UpdateItem
  • 🚧 BatchGetItem
  • 🚧 BatchWriteItem
  • 🚧 TransactGetItems
  • 🚧 TransactWriteItems
  • 🚧 Global Secondary Indexes
  • 🚧 Local Secondary Indexes

Quick Start

# Build the development container
docker-compose build dev

# Start a shell in the container
docker-compose run --rm dev

# Inside the container:
zig build run

Native Build (requires Zig 0.13+ and RocksDB)

# Install dependencies (Ubuntu/Debian)
sudo apt install librocksdb-dev libsnappy-dev liblz4-dev libzstd-dev

# Build and run
zig build run

Usage

Starting the Server

# Default (port 8000)
zig build run

# Custom port
zig build run -- --port 8080

# Custom data directory
zig build run -- --data-dir /var/lib/zynamodb

Using AWS CLI

# Create a table
aws dynamodb create-table \
    --endpoint-url http://localhost:8000 \
    --table-name Users \
    --key-schema AttributeName=pk,KeyType=HASH \
    --attribute-definitions AttributeName=pk,AttributeType=S \
    --billing-mode PAY_PER_REQUEST

# Put an item
aws dynamodb put-item \
    --endpoint-url http://localhost:8000 \
    --table-name Users \
    --item '{"pk":{"S":"user123"},"name":{"S":"Alice"},"email":{"S":"alice@example.com"}}'

# Get an item
aws dynamodb get-item \
    --endpoint-url http://localhost:8000 \
    --table-name Users \
    --key '{"pk":{"S":"user123"}}'

# Scan the table
aws dynamodb scan \
    --endpoint-url http://localhost:8000 \
    --table-name Users

# List tables
aws dynamodb list-tables --endpoint-url http://localhost:8000

Using Python (boto3)

import boto3

# Connect to local ZynamoDB
dynamodb = boto3.client(
    'dynamodb',
    endpoint_url='http://localhost:8000',
    region_name='us-east-1',
    aws_access_key_id='fake',
    aws_secret_access_key='fake'
)

# Create table
dynamodb.create_table(
    TableName='Products',
    KeySchema=[{'AttributeName': 'pk', 'KeyType': 'HASH'}],
    AttributeDefinitions=[{'AttributeName': 'pk', 'AttributeType': 'S'}],
    BillingMode='PAY_PER_REQUEST'
)

# Put item
dynamodb.put_item(
    TableName='Products',
    Item={
        'pk': {'S': 'prod-001'},
        'name': {'S': 'Widget'},
        'price': {'N': '29.99'}
    }
)

# Get item
response = dynamodb.get_item(
    TableName='Products',
    Key={'pk': {'S': 'prod-001'}}
)
print(response.get('Item'))

Development

Project Structure

dynamodb-compat/
├── Dockerfile              # Dev container with Zig + RocksDB
├── docker-compose.yml      # Container orchestration
├── build.zig               # Zig build configuration
├── Makefile                # Convenience commands
├── src/
│   ├── main.zig            # Entry point
│   ├── rocksdb.zig         # RocksDB C bindings
│   ├── http.zig            # HTTP server
│   ├── bench.zig           # Performance benchmarks
│   └── dynamodb/
│       ├── types.zig       # DynamoDB protocol types
│       ├── storage.zig     # Storage engine (RocksDB mapping)
│       └── handler.zig     # API request handlers
└── tests/
    └── integration.zig     # Integration tests

Build Commands

# Build
make build              # Debug build
make release            # Optimized release

# Test
make test               # Unit tests
make test-integration   # Integration tests
make test-all           # All tests

# Run
make run                # Start server
make run-port PORT=8080 # Custom port

# Benchmark
make bench              # Run benchmarks

# Docker
make docker-build       # Build container
make docker-shell       # Open shell
make docker-test        # Run tests in container

Running Tests

# Unit tests
zig build test

# Integration tests
zig build test-integration

# With Docker
docker-compose run --rm dev zig build test

Running Benchmarks

zig build bench

# Or with make
make bench

Architecture

Storage Model

Data is stored in RocksDB with the following key prefixes:

Prefix Purpose Format
_meta: Table metadata _meta:{table_name}
_data: Item data _data:{table}:{pk} or _data:{table}:{pk}:{sk}
_gsi: Global secondary index _gsi:{table}:{index}:{pk}:{sk}
_lsi: Local secondary index _lsi:{table}:{index}:{pk}:{sk}

HTTP Server

  • Custom HTTP/1.1 implementation using Zig's stdlib
  • Thread-per-connection model (suitable for moderate load)
  • Parses X-Amz-Target header to route DynamoDB operations

DynamoDB Protocol

  • JSON request/response format
  • Standard DynamoDB error responses
  • Compatible with AWS SDKs (boto3, AWS CLI, etc.)

Configuration

Command Line Options

Option Description Default
-p, --port HTTP port 8000
-h, --host Bind address 0.0.0.0
-d, --data-dir RocksDB data directory ./data
-v, --verbose Enable verbose logging false

Environment Variables

Variable Description
DYNAMODB_PORT Override port
ROCKSDB_DATA_DIR Override data directory

Performance

Preliminary benchmarks on development hardware:

Operation Ops/sec
PutItem ~15,000
GetItem ~25,000
Scan (1K items) ~50,000

Run make bench for actual numbers on your hardware.

Comparison with DynamoDB Local

Feature ZynamoDB DynamoDB Local
Language Zig Java
Storage RocksDB SQLite
Memory ~10MB ~200MB+
Startup Instant 2-5 seconds
Persistence Yes Optional

License

MIT

Contributing

Contributions welcome! Please read the contributing guidelines first.

Areas that need work:

  • UpdateItem with expression parsing
  • Batch operations
  • Secondary indexes
  • Streams support
  • Better JSON parsing (currently simplified)
Description
No description provided
Readme 226 KiB
Languages
Zig 96.7%
Makefile 1.5%
Shell 1.1%
Dockerfile 0.7%