jormun-db/TODO.md

JormunDB Implementation TODO

This tracks the rewrite from Zig to Odin and remaining features.

✅ Completed

• Project structure
• Makefile with build/run/test targets
• README with usage instructions
• ARCHITECTURE documentation
• RocksDB FFI bindings (rocksdb/rocksdb.odin)
• Core types (dynamodb/types.odin)
• Key codec with varint encoding (key_codec/key_codec.odin)
• Main entry point with arena pattern demo
• LICENSE file
• .gitignore

🚧 In Progress (Need to Complete)

Core Modules

• dynamodb/json.odin - DynamoDB JSON parsing and serialization

  • Parse {"S": "value"} format
  • Serialize AttributeValue to DynamoDB JSON
  • Parse request bodies (PutItem, GetItem, etc.)

• item_codec/item_codec.odin - Binary TLV encoding for items

  • Encode Item to binary TLV format
  • Decode binary TLV back to Item
  • Type tag handling for all DynamoDB types

• dynamodb/storage.odin - Storage engine with RocksDB

  • Table metadata management
  • create_table, delete_table, describe_table, list_tables
  • put_item, get_item, delete_item
  • scan, query with pagination
  • Table-level RW locks

• dynamodb/handler.odin - HTTP request handlers

  • Route X-Amz-Target to handler functions
  • handle_create_table, handle_put_item, etc.
  • Build responses with proper error handling
  • Arena allocator integration

HTTP Server

• HTTP server implementation
  • Accept TCP connections
  • Parse HTTP POST requests
  • Read JSON bodies
  • Send HTTP responses with headers
  • Keep-alive support
  • Options:
    • Use core:net directly
    • Use C FFI with libmicrohttpd
    • Use Odin's vendor:microui (if suitable)

Expression Parsers (Priority 3)

• KeyConditionExpression parser

  • Tokenizer for expressions
  • Parse pk = :pk AND sk > :sk
  • Support begins_with, BETWEEN
  • ExpressionAttributeNames/Values

• UpdateExpression parser (later)

  • SET operations
  • REMOVE operations
  • ADD operations
  • DELETE operations

📋 Testing

• Unit tests for key_codec
• Unit tests for item_codec
• Unit tests for JSON parsing
• Integration tests with AWS CLI
• Benchmark suite

🔧 Build & Tooling

• Verify Makefile works on macOS
• Verify Makefile works on Linux
• Add Docker support (optional)
• Add install script

📚 Documentation

• Code comments for public APIs
• Usage examples in README
• API compatibility matrix
• Performance tuning guide

🎯 Priority Order

1. HTTP Server - Need this to accept requests
2. JSON Parsing - Need this to understand DynamoDB format
3. Storage Engine - Core CRUD operations
4. Handlers - Wire everything together
5. Item Codec - Efficient binary storage
6. Expression Parsers - Query functionality

📝 Notes

Zig → Odin Translation Patterns

Memory Management:

// Zig
const item = try allocator.create(Item);
defer allocator.destroy(item);

// Odin
item := new(Item)
// No defer needed if using arena

Error Handling:

// Zig
fn foo() !Result {
    return error.Failed;
}
const x = try foo();

// Odin
foo :: proc() -> (Result, bool) {
    return {}, false
}
x := foo() or_return

Slices:

// Zig
const slice: []const u8 = data;

// Odin
slice: []byte = data

Maps:

// Zig
var map = std.StringHashMap(Value).init(allocator);
defer map.deinit();

// Odin
map := make(map[string]Value)
defer delete(map)

Key Decisions

1. Use Maybe(T) instead of ?T - Odin's optional type
2. Use or_return instead of try - Odin's error propagation
3. Use context.allocator - Implicit allocator from context
4. Use #partial switch - For union type checking
5. Use transmute - For zero-cost type conversions

Reference Zig Files

When implementing, reference these Zig files:

• src/dynamodb/json.zig - 400 lines, DynamoDB JSON format
• src/dynamodb/storage.zig - 460 lines, storage engine
• src/dynamodb/handler.zig - 500+ lines, request handlers
• src/item_codec.zig - 350 lines, TLV encoding
• src/http.zig - 250 lines, HTTP server

Quick Test Commands

# Build and test
make build
make test

# Run server
make run

# Test with AWS CLI
aws dynamodb list-tables --endpoint-url http://localhost:8000