187 lines
4.3 KiB
Markdown
187 lines
4.3 KiB
Markdown
# JormunDB Implementation TODO
|
|
|
|
This tracks the rewrite from Zig to Odin and remaining features.
|
|
|
|
## ✅ Completed
|
|
|
|
- [x] Project structure
|
|
- [x] Makefile with build/run/test targets
|
|
- [x] README with usage instructions
|
|
- [x] ARCHITECTURE documentation
|
|
- [x] RocksDB FFI bindings (rocksdb/rocksdb.odin)
|
|
- [x] Core types (dynamodb/types.odin)
|
|
- [x] Key codec with varint encoding (key_codec/key_codec.odin)
|
|
- [x] Main entry point with arena pattern demo
|
|
- [x] LICENSE file
|
|
- [x] .gitignore
|
|
|
|
## 🚧 In Progress (Need to Complete)
|
|
|
|
### Core Modules
|
|
|
|
- [ ] **dynamodb/json.odin** - DynamoDB JSON parsing and serialization
|
|
- Parse `{"S": "value"}` format
|
|
- Serialize AttributeValue to DynamoDB JSON
|
|
- Parse request bodies (PutItem, GetItem, etc.)
|
|
|
|
- [ ] **item_codec/item_codec.odin** - Binary TLV encoding for items
|
|
- Encode Item to binary TLV format
|
|
- Decode binary TLV back to Item
|
|
- Type tag handling for all DynamoDB types
|
|
|
|
- [ ] **dynamodb/storage.odin** - Storage engine with RocksDB
|
|
- Table metadata management
|
|
- create_table, delete_table, describe_table, list_tables
|
|
- put_item, get_item, delete_item
|
|
- scan, query with pagination
|
|
- Table-level RW locks
|
|
|
|
- [ ] **dynamodb/handler.odin** - HTTP request handlers
|
|
- Route X-Amz-Target to handler functions
|
|
- handle_create_table, handle_put_item, etc.
|
|
- Build responses with proper error handling
|
|
- Arena allocator integration
|
|
|
|
### HTTP Server
|
|
|
|
- [ ] **HTTP server implementation**
|
|
- Accept TCP connections
|
|
- Parse HTTP POST requests
|
|
- Read JSON bodies
|
|
- Send HTTP responses with headers
|
|
- Keep-alive support
|
|
- Options:
|
|
- Use `core:net` directly
|
|
- Use C FFI with libmicrohttpd
|
|
- Use Odin's vendor:microui (if suitable)
|
|
|
|
### Expression Parsers (Priority 3)
|
|
|
|
- [ ] **KeyConditionExpression parser**
|
|
- Tokenizer for expressions
|
|
- Parse `pk = :pk AND sk > :sk`
|
|
- Support begins_with, BETWEEN
|
|
- ExpressionAttributeNames/Values
|
|
|
|
- [ ] **UpdateExpression parser** (later)
|
|
- SET operations
|
|
- REMOVE operations
|
|
- ADD operations
|
|
- DELETE operations
|
|
|
|
## 📋 Testing
|
|
|
|
- [ ] Unit tests for key_codec
|
|
- [ ] Unit tests for item_codec
|
|
- [ ] Unit tests for JSON parsing
|
|
- [ ] Integration tests with AWS CLI
|
|
- [ ] Benchmark suite
|
|
|
|
## 🔧 Build & Tooling
|
|
|
|
- [ ] Verify Makefile works on macOS
|
|
- [ ] Verify Makefile works on Linux
|
|
- [ ] Add Docker support (optional)
|
|
- [ ] Add install script
|
|
|
|
## 📚 Documentation
|
|
|
|
- [ ] Code comments for public APIs
|
|
- [ ] Usage examples in README
|
|
- [ ] API compatibility matrix
|
|
- [ ] Performance tuning guide
|
|
|
|
## 🎯 Priority Order
|
|
|
|
1. **HTTP Server** - Need this to accept requests
|
|
2. **JSON Parsing** - Need this to understand DynamoDB format
|
|
3. **Storage Engine** - Core CRUD operations
|
|
4. **Handlers** - Wire everything together
|
|
5. **Item Codec** - Efficient binary storage
|
|
6. **Expression Parsers** - Query functionality
|
|
|
|
## 📝 Notes
|
|
|
|
### Zig → Odin Translation Patterns
|
|
|
|
**Memory Management:**
|
|
```zig
|
|
// Zig
|
|
const item = try allocator.create(Item);
|
|
defer allocator.destroy(item);
|
|
```
|
|
```odin
|
|
// Odin
|
|
item := new(Item)
|
|
// No defer needed if using arena
|
|
```
|
|
|
|
**Error Handling:**
|
|
```zig
|
|
// Zig
|
|
fn foo() !Result {
|
|
return error.Failed;
|
|
}
|
|
const x = try foo();
|
|
```
|
|
```odin
|
|
// Odin
|
|
foo :: proc() -> (Result, bool) {
|
|
return {}, false
|
|
}
|
|
x := foo() or_return
|
|
```
|
|
|
|
**Slices:**
|
|
```zig
|
|
// Zig
|
|
const slice: []const u8 = data;
|
|
```
|
|
```odin
|
|
// Odin
|
|
slice: []byte = data
|
|
```
|
|
|
|
**Maps:**
|
|
```zig
|
|
// Zig
|
|
var map = std.StringHashMap(Value).init(allocator);
|
|
defer map.deinit();
|
|
```
|
|
```odin
|
|
// Odin
|
|
map := make(map[string]Value)
|
|
defer delete(map)
|
|
```
|
|
|
|
### Key Decisions
|
|
|
|
1. **Use `Maybe(T)` instead of `?T`** - Odin's optional type
|
|
2. **Use `or_return` instead of `try`** - Odin's error propagation
|
|
3. **Use `context.allocator`** - Implicit allocator from context
|
|
4. **Use `#partial switch`** - For union type checking
|
|
5. **Use `transmute`** - For zero-cost type conversions
|
|
|
|
### Reference Zig Files
|
|
|
|
When implementing, reference these Zig files:
|
|
- `src/dynamodb/json.zig` - 400 lines, DynamoDB JSON format
|
|
- `src/dynamodb/storage.zig` - 460 lines, storage engine
|
|
- `src/dynamodb/handler.zig` - 500+ lines, request handlers
|
|
- `src/item_codec.zig` - 350 lines, TLV encoding
|
|
- `src/http.zig` - 250 lines, HTTP server
|
|
|
|
### Quick Test Commands
|
|
|
|
```bash
|
|
# Build and test
|
|
make build
|
|
make test
|
|
|
|
# Run server
|
|
make run
|
|
|
|
# Test with AWS CLI
|
|
aws dynamodb list-tables --endpoint-url http://localhost:8000
|
|
```
|