diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 8352654..a11e889 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1,4 +1,5 @@ ## JormunDB Architecture +# !!THIS IS NO LONGER ENTIRELY ACCURATE IGNORE OR UPDATE WITH ACCURATE INFO!! This document explains the internal architecture of JormunDB, including design decisions, storage formats, and the arena-per-request memory management pattern. diff --git a/TODO.md b/TODO.md index 16812b9..df63de1 100644 --- a/TODO.md +++ b/TODO.md @@ -47,23 +47,11 @@ Goal: "aws cli works reliably for CreateTable/ListTables/PutItem/GetItem/DeleteI - [x] Expand operator coverage: BETWEEN and begins_with are implemented in parser - [x] **Sort key condition filtering in query** — **DONE**: `query()` now accepts optional `Sort_Key_Condition` and applies it (=, <, <=, >, >=, BETWEEN, begins_with) ---- +### 5) Service Features +- [ ] Configuration settings like environment variables for defining users and credentials +- [ ] Configuration settings for setting up master and replica nodes -## Next (feature parity with Zig + API completeness) -### 5) UpdateItem / conditional logic groundwork -- [x] `UpdateItem` handler registered in router (currently returns clear "not yet supported" error) -- [x] Implement `UpdateItem` (initially minimal: SET for scalar attrs) - - [x] `UpdateItem` needs UPDATED_NEW/UPDATED_OLD response filtering for perfect parity with Dynamo - - **DONE**: `filter_updated_attributes` extracts modified paths from `Update_Plan` and filters the response item to only include those attributes. `get_update_plan_modified_paths` + `filter_item_to_paths` in `transact.odin`. -- [x] Add `ConditionExpression` support for Put/Delete/Update (start with simple comparisons) -- [x] Define internal "update plan" representation (parsed ops → applied mutations) - -### 6) Response completeness / options -- [x] `ReturnValues` handling where relevant (NONE/ALL_OLD/UPDATED_NEW etc. — even partial support is useful) -- [x] `ProjectionExpression` (return subset of attributes) -- [x] `FilterExpression` (post-query filter for Scan/Query) - -### 7) Test coverage / tooling +### 6) Test coverage / tooling - [ ] Add integration tests mirroring AWS CLI script flows: - create table → put → get → scan → query → delete - [ ] Add fuzz-ish tests for: @@ -71,45 +59,19 @@ Goal: "aws cli works reliably for CreateTable/ListTables/PutItem/GetItem/DeleteI - expression parsing robustness - TLV decode failure cases (corrupt bytes) ---- - -## Later (big features) -These align with the "Future Enhancements" list in ARCHITECTURE.md. - -### 8) Secondary indexes +### 7) Secondary indexes - [ ] Global Secondary Indexes (GSI) - [ ] Local Secondary Indexes (LSI) - [ ] Index backfill + write-path maintenance -### 9) Batch + transactions -- [x] BatchWriteItem -- [x] BatchGetItem -- [x] Transactions (TransactWriteItems / TransactGetItems) - - **DONE**: `transact.odin` implements all-or-nothing semantics: - - Put, Delete, Update, ConditionCheck action types - - Pre-flight condition evaluation (all conditions checked before any mutation) - - Deterministic table lock ordering to prevent deadlocks - - `transact_handlers.odin` contains HTTP handlers and JSON parsing - - TransactGetItems supports ProjectionExpression per item - -### 10) Performance / ops +### 8) Performance / ops - [ ] Connection reuse / keep-alive tuning - [ ] Bloom filters / RocksDB options tuning for common patterns - [ ] Optional compression policy (LZ4/Zstd knobs) - [ ] Parallel scan (segment scanning) ---- - -## Replication / WAL +### 9) Replication / WAL (There is a C++ shim stubbed out for WAL iteration and applying write batches.) - [ ] Implement WAL iterator: `latest_sequence`, `wal_iter_next` returning writebatch blob - [ ] Implement apply-writebatch on follower - [ ] Add a minimal replication test harness (leader generates N ops → follower applies → compare) - ---- - -## Housekeeping -- [x] Fix TODO hygiene: keep this file short and "actionable" - - Added "Bug Fixes Applied" section documenting what changed and why -- [ ] Add a CONTRIBUTING quick checklist (allocator rules, formatting, tests) -- [ ] Add "known limitations" section in README (unsupported DynamoDB features) \ No newline at end of file