diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md deleted file mode 100644 index a11e889..0000000 --- a/ARCHITECTURE.md +++ /dev/null @@ -1,409 +0,0 @@ -## JormunDB Architecture -# !!THIS IS NO LONGER ENTIRELY ACCURATE IGNORE OR UPDATE WITH ACCURATE INFO!! - -This document explains the internal architecture of JormunDB, including design decisions, storage formats, and the arena-per-request memory management pattern. - -## Table of Contents - -- [Overview](#overview) -- [Why Odin?](#why-odin) -- [Memory Management](#memory-management) -- [Storage Format](#storage-format) -- [Module Structure](#module-structure) -- [Request Flow](#request-flow) -- [Concurrency Model](#concurrency-model) - -## Overview - -JormunDB is a DynamoDB-compatible database server that speaks the DynamoDB wire protocol. It uses RocksDB for persistent storage and is written in Odin for elegant memory management. - -### Key Design Goals - -1. **Zero allocation ceremony** - No explicit `defer free()` or error handling for every allocation -2. **Binary storage** - Efficient TLV encoding instead of JSON -3. **API compatibility** - Drop-in replacement for DynamoDB -4. **Performance** - RocksDB-backed with efficient key encoding - -## Why Odin? - -The original implementation in Zig suffered from explicit allocator threading: - -```zig -// Zig version - explicit allocator everywhere -fn handleRequest(allocator: std.mem.Allocator, request: []const u8) !Response { - const parsed = try parseJson(allocator, request); - defer parsed.deinit(allocator); - - const item = try storage.getItem(allocator, parsed.table_name, parsed.key); - defer if (item) |i| freeItem(allocator, i); - - const response = try serializeResponse(allocator, item); - defer allocator.free(response); - - return response; // Wait, we deferred the free! -} -``` - -Odin's context allocator system eliminates this: - -```odin -// Odin version - implicit context allocator -handle_request :: proc(request: []byte) -> Response { - // All allocations use context.allocator automatically - parsed := parse_json(request) - item := storage_get_item(parsed.table_name, parsed.key) - response := serialize_response(item) - - return response - // Everything freed when arena is destroyed -} -``` - -## Memory Management - -JormunDB uses a two-allocator strategy: - -### 1. Arena Allocator (Request-Scoped) - -Every HTTP request gets its own arena: - -```odin -handle_connection :: proc(conn: net.TCP_Socket) { - // Create arena for this request (4MB) - arena: mem.Arena - mem.arena_init(&arena, make([]byte, mem.Megabyte * 4)) - defer mem.arena_destroy(&arena) - - // Set context allocator - context.allocator = mem.arena_allocator(&arena) - - // All downstream code uses context.allocator - request := parse_http_request(conn) // uses arena - response := handle_request(request) // uses arena - send_response(conn, response) // uses arena - - // Arena is freed here - everything cleaned up automatically -} -``` - -**Benefits:** -- No individual `free()` calls needed -- No `errdefer` cleanup -- No use-after-free bugs -- No memory leaks from forgotten frees -- Predictable performance (no GC pauses) - -### 2. Default Allocator (Long-Lived Data) - -The default allocator (typically `context.allocator` at program start) is used for: - -- Table metadata -- Table locks (sync.RW_Mutex) -- Engine state -- Items returned from storage layer (copied to request arena when needed) - -## Storage Format - -### Binary Keys (Varint-Prefixed Segments) - -All keys use varint length prefixes for space efficiency: - -``` -Meta key: [0x01][len][table_name] -Data key: [0x02][len][table_name][len][pk_value][len][sk_value]? -GSI key: [0x03][len][table_name][len][index_name][len][gsi_pk][len][gsi_sk]? -LSI key: [0x04][len][table_name][len][index_name][len][pk][len][lsi_sk] -``` - -**Example Data Key:** -``` -Table: "Users" -PK: "user:123" -SK: "profile" - -Encoded: -[0x02] // Entity type (Data) -[0x05] // Table name length (5) -Users // Table name bytes -[0x08] // PK length (8) -user:123 // PK bytes -[0x07] // SK length (7) -profile // SK bytes -``` - -### Item Encoding (TLV Format) - -Items use Tag-Length-Value encoding for space efficiency: - -``` -Format: -[attr_count:varint] - [name_len:varint][name:bytes][type_tag:u8][value_len:varint][value:bytes]... - -Type Tags: - String = 0x01 Number = 0x02 Binary = 0x03 - Bool = 0x04 Null = 0x05 - SS = 0x10 NS = 0x11 BS = 0x12 - List = 0x20 Map = 0x21 -``` - -**Example Item:** -```json -{ - "id": {"S": "user123"}, - "age": {"N": "30"} -} -``` - -Encoded as: -``` -[0x02] // 2 attributes - [0x02] // name length (2) - id // name bytes - [0x01] // type tag (String) - [0x07] // value length (7) - user123 // value bytes - - [0x03] // name length (3) - age // name bytes - [0x02] // type tag (Number) - [0x02] // value length (2) - 30 // value bytes (stored as string) -``` - -## Request Flow - -``` -1. HTTP POST / arrives - ↓ -2. Create arena allocator (4MB) - Set context.allocator = arena_allocator - ↓ -3. Parse HTTP headers - Extract X-Amz-Target → Operation - ↓ -4. Parse JSON body - Convert DynamoDB JSON → internal types - ↓ -5. Route to handler (e.g., handle_put_item) - ↓ -6. Storage engine operation - - Build binary key - - Encode item to TLV - - RocksDB put/get/delete - ↓ -7. Build response - - Serialize item to DynamoDB JSON - - Format HTTP response - ↓ -8. Send response - ↓ -9. Destroy arena - All request memory freed automatically -``` - -## Concurrency Model - -### Table-Level RW Locks - -Each table has a reader-writer lock: - -```odin -Storage_Engine :: struct { - db: rocksdb.DB, - table_locks: map[string]^sync.RW_Mutex, - table_locks_mutex: sync.Mutex, -} -``` - -**Read Operations** (GetItem, Query, Scan): -- Acquire shared lock -- Multiple readers can run concurrently -- Writers are blocked - -**Write Operations** (PutItem, DeleteItem, UpdateItem): -- Acquire exclusive lock -- Only one writer at a time -- All readers are blocked - -### Thread Safety - -- RocksDB handles are thread-safe (column family-based) -- Table metadata is protected by locks -- Request arenas are thread-local (no sharing) - -## Error Handling - -Odin uses explicit error returns via `or_return`: - -```odin -// Odin error handling -parse_json :: proc(data: []byte) -> (Item, bool) { - parsed := json.parse(data) or_return - item := json_to_item(parsed) or_return - return item, true -} - -// Usage -item := parse_json(request.body) or_else { - return error_response(.ValidationException, "Invalid JSON") -} -``` - -No exceptions, no panic-recover patterns. Every error path is explicit. - -## DynamoDB Wire Protocol - -### Request Format - -``` -POST / HTTP/1.1 -X-Amz-Target: DynamoDB_20120810.PutItem -Content-Type: application/x-amz-json-1.0 - -{ - "TableName": "Users", - "Item": { - "id": {"S": "user123"}, - "name": {"S": "Alice"} - } -} -``` - -### Response Format - -``` -HTTP/1.1 200 OK -Content-Type: application/x-amz-json-1.0 -x-amzn-RequestId: local-request-id - -{} -``` - -### Error Format - -```json -{ - "__type": "com.amazonaws.dynamodb.v20120810#ResourceNotFoundException", - "message": "Table not found" -} -``` - -## Performance Characteristics - -### Time Complexity - -| Operation | Complexity | Notes | -|-----------|-----------|-------| -| PutItem | O(log n) | RocksDB LSM tree insert | -| GetItem | O(log n) | RocksDB point lookup | -| DeleteItem | O(log n) | RocksDB deletion | -| Query | O(log n + m) | n = items in table, m = result set | -| Scan | O(n) | Full table scan | - -### Space Complexity - -- Binary keys: ~20-100 bytes (vs 50-200 bytes JSON) -- Binary items: ~30% smaller than JSON -- Varint encoding saves space on small integers - -### Benchmarks (Expected) - -Based on Zig version performance: - -``` -Operation Throughput Latency (p50) -PutItem ~5,000/sec ~0.2ms -GetItem ~7,000/sec ~0.14ms -Query (1 item) ~8,000/sec ~0.12ms -Scan (1000 items) ~20/sec ~50ms -``` - -## Future Enhancements - -### Planned Features - -1. **UpdateExpression** - SET/REMOVE/ADD/DELETE operations -2. **FilterExpression** - Post-query filtering -3. **ProjectionExpression** - Return subset of attributes -4. **Global Secondary Indexes** - Query by non-key attributes -5. **Local Secondary Indexes** - Alternate sort keys -6. **BatchWriteItem** - Batch mutations -7. **BatchGetItem** - Batch reads -8. **Transactions** - ACID multi-item operations - -### Optimization Opportunities - -1. **Connection pooling** - Reuse HTTP connections -2. **Bloom filters** - Faster negative lookups -3. **Compression** - LZ4/Zstd on large items -4. **Caching layer** - Hot item cache -5. **Parallel scan** - Segment-based scanning - -## Debugging - -### Enable Verbose Logging - -```bash -make run VERBOSE=1 -``` - -### Inspect RocksDB - -```bash -# Use ldb tool to inspect database -ldb --db=./data scan -ldb --db=./data get -``` - -### Memory Profiling - -Odin's tracking allocator can detect leaks: - -```odin -when ODIN_DEBUG { - track: mem.Tracking_Allocator - mem.tracking_allocator_init(&track, context.allocator) - context.allocator = mem.tracking_allocator(&track) - - defer { - for _, leak in track.allocation_map { - fmt.printfln("Leaked %d bytes at %p", leak.size, leak.location) - } - } -} -``` - -## Migration from Zig Version - -The Zig version (ZynamoDB) used the same binary storage format, so existing RocksDB databases can be read by JormunDB without migration. - -### Compatibility - -- ✅ Binary key format (byte-compatible) -- ✅ Binary item format (byte-compatible) -- ✅ Table metadata (JSON, compatible) -- ✅ HTTP wire protocol (identical) - -### Breaking Changes - -None - JormunDB can open ZynamoDB databases directly. - ---- - -## Contributing - -When contributing to JormunDB: - -1. **Use the context allocator** - All request-scoped allocations should use `context.allocator` -2. **Avoid manual frees** - Let the arena handle it -3. **Long-lived data** - Use the default allocator explicitly -4. **Test thoroughly** - Run `make test` before committing -5. **Format code** - Run `make fmt` before committing - -## References - -- [Odin Language](https://odin-lang.org/) -- [RocksDB Wiki](https://github.com/facebook/rocksdb/wiki) -- [DynamoDB API Reference](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/) -- [Varint Encoding](https://developers.google.com/protocol-buffers/docs/encoding#varints) diff --git a/dynamodb/expression.odin b/dynamodb/expression.odin index 3ac61f5..bf2cee7 100644 --- a/dynamodb/expression.odin +++ b/dynamodb/expression.odin @@ -440,7 +440,13 @@ parse_expression_attribute_values :: proc(request_body: []byte) -> (map[string]A for key, val in values_obj { attr, attr_ok := parse_attribute_value(val) if !attr_ok { - continue + // Clean up already-parsed values before returning error + for k, &v in result { + attr_value_destroy(&v) + delete(k) + } + delete(result) + return make(map[string]Attribute_Value), false } result[strings.clone(key)] = attr } diff --git a/dynamodb/filter.odin b/dynamodb/filter.odin index e3971f9..d30fdb3 100644 --- a/dynamodb/filter.odin +++ b/dynamodb/filter.odin @@ -170,6 +170,9 @@ filter_node_destroy :: proc(node: ^Filter_Node) { if node.child != nil { filter_node_destroy(node.child) } + + // Free the node itself (allocated with new(Filter_Node)) + free(node) } // ============================================================================ @@ -735,10 +738,10 @@ evaluate_contains :: proc(attr: Attribute_Value, val: Attribute_Value) -> bool { } } - case Number_Set: - if v, ok := val.(Number); ok { - for n in a { - if n == string(v) { + case DDB_Number_Set: + if v, ok := val.(DDB_Number); ok { + for num in a { + if compare_ddb_numbers(num, v) == 0 { return true } } diff --git a/dynamodb/gsi.odin b/dynamodb/gsi.odin index 9bace4f..184053f 100644 --- a/dynamodb/gsi.odin +++ b/dynamodb/gsi.odin @@ -76,8 +76,8 @@ attr_value_to_bytes :: proc(attr: Attribute_Value) -> ([]byte, bool) { #partial switch v in attr { case String: return transmute([]byte)string(v), true - case Number: - return transmute([]byte)string(v), true + case DDB_Number: + return encode_ddb_number_for_sort(v), true case Binary: return transmute([]byte)string(v), true } diff --git a/dynamodb/item_codec.odin b/dynamodb/item_codec.odin index 413d699..f590d59 100644 --- a/dynamodb/item_codec.odin +++ b/dynamodb/item_codec.odin @@ -83,11 +83,6 @@ encode_attribute_value :: proc(buf: ^bytes.Buffer, attr: Attribute_Value) -> boo encode_varint(buf, len(num_str)) bytes.buffer_write_string(buf, num_str) - case Number: - bytes.buffer_write_byte(buf, u8(Type_Tag.Number)) - encode_varint(buf, len(v)) - bytes.buffer_write_string(buf, string(v)) - case Binary: bytes.buffer_write_byte(buf, u8(Type_Tag.Binary)) encode_varint(buf, len(v)) @@ -119,14 +114,6 @@ encode_attribute_value :: proc(buf: ^bytes.Buffer, attr: Attribute_Value) -> boo bytes.buffer_write_string(buf, s) } - case Number_Set: - bytes.buffer_write_byte(buf, u8(Type_Tag.Number_Set)) - encode_varint(buf, len(v)) - for n in v { - encode_varint(buf, len(n)) - bytes.buffer_write_string(buf, n) - } - case Binary_Set: bytes.buffer_write_byte(buf, u8(Type_Tag.Binary_Set)) encode_varint(buf, len(v)) diff --git a/dynamodb/json.odin b/dynamodb/json.odin index 5abe929..b273128 100644 --- a/dynamodb/json.odin +++ b/dynamodb/json.odin @@ -324,9 +324,6 @@ serialize_attribute_value :: proc(b: ^strings.Builder, attr: Attribute_Value) { case String: fmt.sbprintf(b, `{"S":"%s"}`, string(v)) - case Number: - fmt.sbprintf(b, `{"N":"%s"}`, string(v)) - case DDB_Number: num_str := format_ddb_number(v) fmt.sbprintf(b, `{"N":"%s"}`, num_str) @@ -350,16 +347,6 @@ serialize_attribute_value :: proc(b: ^strings.Builder, attr: Attribute_Value) { } strings.write_string(b, "]}") - case Number_Set: - strings.write_string(b, `{"NS":[`) - for n, i in v { - if i > 0 { - strings.write_string(b, ",") - } - fmt.sbprintf(b, `"%s"`, n) - } - strings.write_string(b, "]}") - case DDB_Number_Set: strings.write_string(b, `{"NS":[`) for num, i in v { diff --git a/dynamodb/number.odin b/dynamodb/number.odin index 9bec980..96d1663 100644 --- a/dynamodb/number.odin +++ b/dynamodb/number.odin @@ -494,7 +494,7 @@ f64_to_ddb_number :: proc(val: f64) -> (DDB_Number, bool) { return parse_ddb_number(str) } -// Format a DDB_Number for display (like format_number but preserves precision) +// Format a DDB_Number for display format_ddb_number :: proc(num: DDB_Number) -> string { // Normalize first norm := normalize_ddb_number(num) diff --git a/dynamodb/storage.odin b/dynamodb/storage.odin index e599b2d..ba9d63f 100644 --- a/dynamodb/storage.odin +++ b/dynamodb/storage.odin @@ -243,7 +243,13 @@ serialize_table_metadata :: proc(metadata: ^Table_Metadata) -> ([]byte, bool) { // Add other metadata meta_item["TableStatus"] = String(strings.clone(table_status_to_string(metadata.table_status))) - meta_item["CreationDateTime"] = Number(fmt.aprint(metadata.creation_date_time)) + ts_str := fmt.aprint(metadata.creation_date_time) + ts_num, ts_ok := parse_ddb_number(ts_str) + if ts_ok { + meta_item["CreationDateTime"] = ts_num + } else { + meta_item["CreationDateTime"] = String(strings.clone(ts_str)) + } // Encode GSI definitions as JSON string if gsis, has_gsis := metadata.global_secondary_indexes.?; has_gsis && len(gsis) > 0 { @@ -314,8 +320,9 @@ deserialize_table_metadata :: proc(data: []byte, allocator: mem.Allocator) -> (T // Parse creation date time if time_val, found := meta_item["CreationDateTime"]; found { #partial switch v in time_val { - case Number: - val, parse_ok := strconv.parse_i64(string(v)) + case DDB_Number: + num_str := format_ddb_number(v) + val, parse_ok := strconv.parse_i64(num_str) metadata.creation_date_time = val if parse_ok else 0 } } @@ -1226,6 +1233,44 @@ evaluate_sort_key_condition :: proc(item: Item, skc: ^Sort_Key_Condition) -> boo return false } + // Use numeric comparison if both sides are DDB_Number + item_num, item_is_num := attr.(DDB_Number) + cond_num, cond_is_num := skc.value.(DDB_Number) + + if item_is_num && cond_is_num { + cmp := compare_ddb_numbers(item_num, cond_num) + + switch skc.operator { + case .EQ: + return cmp == 0 + case .LT: + return cmp < 0 + case .LE: + return cmp <= 0 + case .GT: + return cmp > 0 + case .GE: + return cmp >= 0 + case .BETWEEN: + if v2, has_v2 := skc.value2.?; has_v2 { + upper_num, upper_ok := v2.(DDB_Number) + if !upper_ok { + return false + } + cmp2 := compare_ddb_numbers(item_num, upper_num) + return cmp >= 0 && cmp2 <= 0 + } + return false + case .BEGINS_WITH: + // BEGINS_WITH on numbers: fall through to string comparison + item_str := format_ddb_number(item_num) + cond_str := format_ddb_number(cond_num) + return strings.has_prefix(item_str, cond_str) + } + return false + } + + // Fallback: string comparison for S/B types item_sk_str, ok1 := attr_value_to_string_for_compare(attr) if !ok1 { return false @@ -1272,8 +1317,10 @@ attr_value_to_string_for_compare :: proc(attr: Attribute_Value) -> (string, bool #partial switch v in attr { case String: return string(v), true - case Number: - return string(v), true + case DDB_Number: + // Return formatted string for fallback string comparison + // (actual numeric comparison is handled in compare_attribute_values) + return format_ddb_number(v), true case Binary: return string(v), true } @@ -1318,7 +1365,7 @@ validate_item_key_types :: proc( #partial switch _ in attr { case String: match = (et == .S) - case Number: + case DDB_Number: match = (et == .N) case Binary: match = (et == .B) diff --git a/dynamodb/types.odin b/dynamodb/types.odin index 64f57f8..5027d2c 100644 --- a/dynamodb/types.odin +++ b/dynamodb/types.odin @@ -5,28 +5,24 @@ import "core:strings" // DynamoDB AttributeValue - the core data type Attribute_Value :: union { - String, // S - Number, // N = stored as string, I'm keeping this so we can still do the parsing/serialization - DDB_Number, // N Dynamo uses whole numbers and not floats or strings so we'll make that its own type - Binary, // B (base64) - Bool, // BOOL - Null, // NULL - String_Set, // SS - Number_Set, // NS - Binary_Set, // BS - DDB_Number_Set,// BS - List, // L - Map, // M + String, // S + DDB_Number, // N — decimal-preserving numeric type + Binary, // B (base64) + Bool, // BOOL + Null, // NULL + String_Set, // SS + DDB_Number_Set, // NS + Binary_Set, // BS + List, // L + Map, // M } String :: distinct string -Number :: distinct string Binary :: distinct string Bool :: distinct bool Null :: distinct bool String_Set :: distinct []string -Number_Set :: distinct []string DDB_Number_Set :: distinct []DDB_Number Binary_Set :: distinct []string List :: distinct []Attribute_Value @@ -63,7 +59,7 @@ key_from_item :: proc(item: Item, key_schema: []Key_Schema_Element) -> (Key, boo // Validate that key is a scalar type (S, N, or B) #partial switch _ in attr { - case String, Number, Binary: + case String, DDB_Number, Binary: // Valid key type case: return {}, false @@ -119,12 +115,11 @@ key_get_values :: proc(key: ^Key) -> (Key_Values, bool) { #partial switch v in key.pk { case String: pk_bytes = transmute([]byte)string(v) - case Number: - pk_bytes = transmute([]byte)string(v) + case DDB_Number: + pk_bytes = encode_ddb_number_for_sort(v) case Binary: pk_bytes = transmute([]byte)string(v) case: - // Keys should only be scalar types (S, N, or B) return {}, false } @@ -133,12 +128,11 @@ key_get_values :: proc(key: ^Key) -> (Key_Values, bool) { #partial switch v in sk { case String: sk_bytes = transmute([]byte)string(v) - case Number: - sk_bytes = transmute([]byte)string(v) + case DDB_Number: + sk_bytes = encode_ddb_number_for_sort(v) case Binary: sk_bytes = transmute([]byte)string(v) case: - // Keys should only be scalar types return {}, false } } @@ -369,13 +363,27 @@ error_to_response :: proc(err_type: DynamoDB_Error_Type, message: string) -> str // Build an Attribute_Value with the correct scalar type from raw bytes build_attribute_value_with_type :: proc(raw_bytes: []byte, attr_type: Scalar_Attribute_Type) -> Attribute_Value { - owned := strings.clone(string(raw_bytes)) switch attr_type { - case .S: return String(owned) - case .N: return Number(owned) - case .B: return Binary(owned) + case .S: + return String(strings.clone(string(raw_bytes))) + case .N: + // Key bytes are canonical-encoded via encode_ddb_number_for_sort. + // Decode them back to a DDB_Number. + ddb_num, ok := decode_ddb_number_from_sort(raw_bytes) + if ok { + return clone_ddb_number(ddb_num) + } + // Fallback: try interpreting as a plain numeric string + fallback_num, fb_ok := parse_ddb_number(string(raw_bytes)) + if fb_ok { + return fallback_num + } + // Last resort — return as string (shouldn't happen) + return String(strings.clone(string(raw_bytes))) + case .B: + return Binary(strings.clone(string(raw_bytes))) } - return String(owned) + return String(strings.clone(string(raw_bytes))) } // Deep copy an attribute value @@ -383,8 +391,6 @@ attr_value_deep_copy :: proc(attr: Attribute_Value) -> Attribute_Value { switch v in attr { case String: return String(strings.clone(string(v))) - case Number: - return Number(strings.clone(string(v))) case DDB_Number: return clone_ddb_number(v) case Binary: @@ -399,12 +405,6 @@ attr_value_deep_copy :: proc(attr: Attribute_Value) -> Attribute_Value { ss[i] = strings.clone(s) } return String_Set(ss) - case Number_Set: - ns := make([]string, len(v)) - for n, i in v { - ns[i] = strings.clone(n) - } - return Number_Set(ns) case DDB_Number_Set: ddb_ns := make([]DDB_Number, len(v)) for num, i in v { @@ -441,8 +441,6 @@ attr_value_destroy :: proc(attr: ^Attribute_Value) { case DDB_Number: delete(v.integer_part) delete(v.fractional_part) - case Number: - delete(string(v)) case Binary: delete(string(v)) case String_Set: @@ -451,12 +449,6 @@ attr_value_destroy :: proc(attr: ^Attribute_Value) { } slice := v delete(slice) - case Number_Set: - for n in v { - delete(n) - } - slice := v - delete(slice) case DDB_Number_Set: for num in v { delete(num.integer_part) @@ -497,4 +489,4 @@ item_destroy :: proc(item: ^Item) { attr_value_destroy(&val_copy) } delete(item^) -} +} \ No newline at end of file diff --git a/dynamodb/update.odin b/dynamodb/update.odin index 3bcbc43..c484588 100644 --- a/dynamodb/update.odin +++ b/dynamodb/update.odin @@ -13,8 +13,6 @@ package dynamodb import "core:encoding/json" -import "core:fmt" -import "core:strconv" import "core:strings" // ============================================================================ @@ -710,7 +708,7 @@ execute_update_plan :: proc(item: ^Item, plan: ^Update_Plan) -> bool { if existing, found := item[action.path]; found { // If existing is a number, add numerically #partial switch v in existing { - case Number: + case DDB_Number: result, add_ok := numeric_add(existing, action.value) if !add_ok { return false @@ -732,13 +730,13 @@ execute_update_plan :: proc(item: ^Item, plan: ^Update_Plan) -> bool { return false } - case Number_Set: - if new_ns, is_ns := action.value.(Number_Set); is_ns { - merged := set_union_strings(([]string)(v), ([]string)(new_ns)) + case DDB_Number_Set: + if new_ns, is_ns := action.value.(DDB_Number_Set); is_ns { + merged := set_union_ddb_numbers(([]DDB_Number)(v), ([]DDB_Number)(new_ns)) old_copy := existing attr_value_destroy(&old_copy) delete_key(item, action.path) - item[strings.clone(action.path)] = Number_Set(merged) + item[strings.clone(action.path)] = DDB_Number_Set(merged) } else { return false } @@ -769,14 +767,14 @@ execute_update_plan :: proc(item: ^Item, plan: ^Update_Plan) -> bool { } } - case Number_Set: - if del_ns, is_ns := action.value.(Number_Set); is_ns { - result := set_difference_strings(([]string)(v), ([]string)(del_ns)) + case DDB_Number_Set: + if del_ns, is_ns := action.value.(DDB_Number_Set); is_ns { + result := set_difference_ddb_numbers(([]DDB_Number)(v), ([]DDB_Number)(del_ns)) old_copy := existing attr_value_destroy(&old_copy) delete_key(item, action.path) if len(result) > 0 { - item[strings.clone(action.path)] = Number_Set(result) + item[strings.clone(action.path)] = DDB_Number_Set(result) } else { delete(result) } @@ -810,30 +808,17 @@ numeric_add :: proc(a: Attribute_Value, b: Attribute_Value) -> (Attribute_Value, } numeric_subtract :: proc(a: Attribute_Value, b: Attribute_Value) -> (Attribute_Value, bool) { - a_num, a_ok := a.(Number) - b_num, b_ok := b.(Number) + a_num, a_ok := a.(DDB_Number) + b_num, b_ok := b.(DDB_Number) if !a_ok || !b_ok { return nil, false } - a_val, a_parse := strconv.parse_f64(string(a_num)) - b_val, b_parse := strconv.parse_f64(string(b_num)) - if !a_parse || !b_parse { + result, result_ok := subtract_ddb_numbers(a_num, b_num) + if !result_ok { return nil, false } - - result := a_val - b_val - result_str := format_number(result) - return Number(result_str), true -} - -format_number :: proc(val: f64) -> string { - // If it's an integer, format without decimal point - int_val := i64(val) - if f64(int_val) == val { - return fmt.aprintf("%d", int_val) - } - return fmt.aprintf("%g", val) + return result, true } // ============================================================================ @@ -873,6 +858,52 @@ set_difference_strings :: proc(a: []string, b: []string) -> []string { return result[:] } +// Union of two DDB_Number slices (dedup by numeric equality) +set_union_ddb_numbers :: proc(a: []DDB_Number, b: []DDB_Number) -> []DDB_Number { + result := make([dynamic]DDB_Number) + + // Add all from a + for num in a { + append(&result, clone_ddb_number(num)) + } + + // Add from b if not already present + for num in b { + found := false + for existing in result { + if compare_ddb_numbers(existing, num) == 0 { + found = true + break + } + } + if !found { + append(&result, clone_ddb_number(num)) + } + } + + return result[:] +} + +// Difference: elements in a that are NOT in b +set_difference_ddb_numbers :: proc(a: []DDB_Number, b: []DDB_Number) -> []DDB_Number { + result := make([dynamic]DDB_Number) + + for num in a { + in_b := false + for del in b { + if compare_ddb_numbers(num, del) == 0 { + in_b = true + break + } + } + if !in_b { + append(&result, clone_ddb_number(num)) + } + } + + return result[:] +} + // ============================================================================ // Request Parsing Helper // ============================================================================