biondizzle/jormun-db
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix leaks

Browse Source
This commit is contained in:
biondizzle
2026-02-16 10:52:35 -05:00
parent a77676bbc7
commit 96de080d10
10 changed files with 165 additions and 521 deletions
Download Patch File Download Diff File Expand all files Collapse all files

409
ARCHITECTURE.md
View File

@@ -1,409 +0,0 @@
## JormunDB Architecture
# !!THIS IS NO LONGER ENTIRELY ACCURATE IGNORE OR UPDATE WITH ACCURATE INFO!!
This document explains the internal architecture of JormunDB, including design decisions, storage formats, and the arena-per-request memory management pattern.
## Table of Contents
- [Overview](#overview)
- [Why Odin?](#why-odin)
- [Memory Management](#memory-management)
- [Storage Format](#storage-format)
- [Module Structure](#module-structure)
- [Request Flow](#request-flow)
- [Concurrency Model](#concurrency-model)
## Overview
JormunDB is a DynamoDB-compatible database server that speaks the DynamoDB wire protocol. It uses RocksDB for persistent storage and is written in Odin for elegant memory management.
### Key Design Goals
1. **Zero allocation ceremony** - No explicit `defer free()` or error handling for every allocation
2. **Binary storage** - Efficient TLV encoding instead of JSON
3. **API compatibility** - Drop-in replacement for DynamoDB
4. **Performance** - RocksDB-backed with efficient key encoding
## Why Odin?
The original implementation in Zig suffered from explicit allocator threading:
```zig
// Zig version - explicit allocator everywhere
fn handleRequest(allocator: std.mem.Allocator, request: []const u8) !Response {
const parsed = try parseJson(allocator, request);
defer parsed.deinit(allocator);
const item = try storage.getItem(allocator, parsed.table_name, parsed.key);
defer if (item) |i| freeItem(allocator, i);
const response = try serializeResponse(allocator, item);
defer allocator.free(response);
return response; // Wait, we deferred the free!
}
```
Odin's context allocator system eliminates this:
```odin
// Odin version - implicit context allocator
handle_request :: proc(request: []byte) -> Response {
// All allocations use context.allocator automatically
parsed := parse_json(request)
item := storage_get_item(parsed.table_name, parsed.key)
response := serialize_response(item)
return response
// Everything freed when arena is destroyed
}
```
## Memory Management
JormunDB uses a two-allocator strategy:
### 1. Arena Allocator (Request-Scoped)
Every HTTP request gets its own arena:
```odin
handle_connection :: proc(conn: net.TCP_Socket) {
// Create arena for this request (4MB)
arena: mem.Arena
mem.arena_init(&arena, make([]byte, mem.Megabyte * 4))
defer mem.arena_destroy(&arena)
// Set context allocator
context.allocator = mem.arena_allocator(&arena)
// All downstream code uses context.allocator
request := parse_http_request(conn) // uses arena
response := handle_request(request) // uses arena
send_response(conn, response) // uses arena
// Arena is freed here - everything cleaned up automatically
}
```
**Benefits:**
- No individual `free()` calls needed
- No `errdefer` cleanup
- No use-after-free bugs
- No memory leaks from forgotten frees
- Predictable performance (no GC pauses)
### 2. Default Allocator (Long-Lived Data)
The default allocator (typically `context.allocator` at program start) is used for:
- Table metadata
- Table locks (sync.RW_Mutex)
- Engine state
- Items returned from storage layer (copied to request arena when needed)
## Storage Format
### Binary Keys (Varint-Prefixed Segments)
All keys use varint length prefixes for space efficiency:
```
Meta key: [0x01][len][table_name]
Data key: [0x02][len][table_name][len][pk_value][len][sk_value]?
GSI key: [0x03][len][table_name][len][index_name][len][gsi_pk][len][gsi_sk]?
LSI key: [0x04][len][table_name][len][index_name][len][pk][len][lsi_sk]
```
**Example Data Key:**
```
Table: "Users"
PK: "user:123"
SK: "profile"
Encoded:
[0x02] // Entity type (Data)
[0x05] // Table name length (5)
Users // Table name bytes
[0x08] // PK length (8)
user:123 // PK bytes
[0x07] // SK length (7)
profile // SK bytes
```
### Item Encoding (TLV Format)
Items use Tag-Length-Value encoding for space efficiency:
```
Format:
[attr_count:varint]
[name_len:varint][name:bytes][type_tag:u8][value_len:varint][value:bytes]...
Type Tags:
String = 0x01 Number = 0x02 Binary = 0x03
Bool = 0x04 Null = 0x05
SS = 0x10 NS = 0x11 BS = 0x12
List = 0x20 Map = 0x21
```
**Example Item:**
```json
{
"id": {"S": "user123"},
"age": {"N": "30"}
}
```
Encoded as:
```
[0x02] // 2 attributes
[0x02] // name length (2)
id // name bytes
[0x01] // type tag (String)
[0x07] // value length (7)
user123 // value bytes
[0x03] // name length (3)
age // name bytes
[0x02] // type tag (Number)
[0x02] // value length (2)
30 // value bytes (stored as string)
```
## Request Flow
```
1. HTTP POST / arrives
↓
2. Create arena allocator (4MB)
Set context.allocator = arena_allocator
↓
3. Parse HTTP headers
Extract X-Amz-Target → Operation
↓
4. Parse JSON body
Convert DynamoDB JSON → internal types
↓
5. Route to handler (e.g., handle_put_item)
↓
6. Storage engine operation
- Build binary key
- Encode item to TLV
- RocksDB put/get/delete
↓
7. Build response
- Serialize item to DynamoDB JSON
- Format HTTP response
↓
8. Send response
↓
9. Destroy arena
All request memory freed automatically
```
## Concurrency Model
### Table-Level RW Locks
Each table has a reader-writer lock:
```odin
Storage_Engine :: struct {
db: rocksdb.DB,
table_locks: map[string]^sync.RW_Mutex,
table_locks_mutex: sync.Mutex,
}
```
**Read Operations** (GetItem, Query, Scan):
- Acquire shared lock
- Multiple readers can run concurrently
- Writers are blocked
**Write Operations** (PutItem, DeleteItem, UpdateItem):
- Acquire exclusive lock
- Only one writer at a time
- All readers are blocked
### Thread Safety
- RocksDB handles are thread-safe (column family-based)
- Table metadata is protected by locks
- Request arenas are thread-local (no sharing)
## Error Handling
Odin uses explicit error returns via `or_return`:
```odin
// Odin error handling
parse_json :: proc(data: []byte) -> (Item, bool) {
parsed := json.parse(data) or_return
item := json_to_item(parsed) or_return
return item, true
}
// Usage
item := parse_json(request.body) or_else {
return error_response(.ValidationException, "Invalid JSON")
}
```
No exceptions, no panic-recover patterns. Every error path is explicit.
## DynamoDB Wire Protocol
### Request Format
```
POST / HTTP/1.1
X-Amz-Target: DynamoDB_20120810.PutItem
Content-Type: application/x-amz-json-1.0
{
"TableName": "Users",
"Item": {
"id": {"S": "user123"},
"name": {"S": "Alice"}
}
}
```
### Response Format
```
HTTP/1.1 200 OK
Content-Type: application/x-amz-json-1.0
x-amzn-RequestId: local-request-id
{}
```
### Error Format
```json
{
"__type": "com.amazonaws.dynamodb.v20120810#ResourceNotFoundException",
"message": "Table not found"
}
```
## Performance Characteristics
### Time Complexity
| Operation | Complexity | Notes |
|-----------|-----------|-------|
| PutItem | O(log n) | RocksDB LSM tree insert |
| GetItem | O(log n) | RocksDB point lookup |
| DeleteItem | O(log n) | RocksDB deletion |
| Query | O(log n + m) | n = items in table, m = result set |
| Scan | O(n) | Full table scan |
### Space Complexity
- Binary keys: ~20-100 bytes (vs 50-200 bytes JSON)
- Binary items: ~30% smaller than JSON
- Varint encoding saves space on small integers
### Benchmarks (Expected)
Based on Zig version performance:
```
Operation Throughput Latency (p50)
PutItem ~5,000/sec ~0.2ms
GetItem ~7,000/sec ~0.14ms
Query (1 item) ~8,000/sec ~0.12ms
Scan (1000 items) ~20/sec ~50ms
```
## Future Enhancements
### Planned Features
1. **UpdateExpression** - SET/REMOVE/ADD/DELETE operations
2. **FilterExpression** - Post-query filtering
3. **ProjectionExpression** - Return subset of attributes
4. **Global Secondary Indexes** - Query by non-key attributes
5. **Local Secondary Indexes** - Alternate sort keys
6. **BatchWriteItem** - Batch mutations
7. **BatchGetItem** - Batch reads
8. **Transactions** - ACID multi-item operations
### Optimization Opportunities
1. **Connection pooling** - Reuse HTTP connections
2. **Bloom filters** - Faster negative lookups
3. **Compression** - LZ4/Zstd on large items
4. **Caching layer** - Hot item cache
5. **Parallel scan** - Segment-based scanning
## Debugging
### Enable Verbose Logging
```bash
make run VERBOSE=1
```
### Inspect RocksDB
```bash
# Use ldb tool to inspect database
ldb --db=./data scan
ldb --db=./data get <key_hex>
```
### Memory Profiling
Odin's tracking allocator can detect leaks:
```odin
when ODIN_DEBUG {
track: mem.Tracking_Allocator
mem.tracking_allocator_init(&track, context.allocator)
context.allocator = mem.tracking_allocator(&track)
defer {
for _, leak in track.allocation_map {
fmt.printfln("Leaked %d bytes at %p", leak.size, leak.location)
}
}
}
```
## Migration from Zig Version
The Zig version (ZynamoDB) used the same binary storage format, so existing RocksDB databases can be read by JormunDB without migration.
### Compatibility
- ✅ Binary key format (byte-compatible)
- ✅ Binary item format (byte-compatible)
- ✅ Table metadata (JSON, compatible)
- ✅ HTTP wire protocol (identical)
### Breaking Changes
None - JormunDB can open ZynamoDB databases directly.
---
## Contributing
When contributing to JormunDB:
1. **Use the context allocator** - All request-scoped allocations should use `context.allocator`
2. **Avoid manual frees** - Let the arena handle it
3. **Long-lived data** - Use the default allocator explicitly
4. **Test thoroughly** - Run `make test` before committing
5. **Format code** - Run `make fmt` before committing
## References
- [Odin Language](https://odin-lang.org/)
- [RocksDB Wiki](https://github.com/facebook/rocksdb/wiki)
- [DynamoDB API Reference](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/)
- [Varint Encoding](https://developers.google.com/protocol-buffers/docs/encoding#varints)

8
dynamodb/expression.odin
View File

@@ -440,7 +440,13 @@ parse_expression_attribute_values :: proc(request_body: []byte) -> (map[string]A
for key, val in values_obj { for key, val in values_obj {
attr, attr_ok := parse_attribute_value(val) attr, attr_ok := parse_attribute_value(val)
if !attr_ok { if !attr_ok {
continue // Clean up already-parsed values before returning error
for k, &v in result {
attr_value_destroy(&v)
delete(k)
}
delete(result)
return make(map[string]Attribute_Value), false
} }
result[strings.clone(key)] = attr result[strings.clone(key)] = attr
} }

11
dynamodb/filter.odin
View File

@@ -170,6 +170,9 @@ filter_node_destroy :: proc(node: ^Filter_Node) {
if node.child != nil { if node.child != nil {
filter_node_destroy(node.child) filter_node_destroy(node.child)
} }
// Free the node itself (allocated with new(Filter_Node))
free(node)
} }
// ============================================================================ // ============================================================================
@@ -735,10 +738,10 @@ evaluate_contains :: proc(attr: Attribute_Value, val: Attribute_Value) -> bool {
} }
} }
case Number_Set: case DDB_Number_Set:
if v, ok := val.(Number); ok { if v, ok := val.(DDB_Number); ok {
for n in a { for num in a {
if n == string(v) { if compare_ddb_numbers(num, v) == 0 {
return true return true
} }
} }

4
dynamodb/gsi.odin
View File

@@ -76,8 +76,8 @@ attr_value_to_bytes :: proc(attr: Attribute_Value) -> ([]byte, bool) {
#partial switch v in attr { #partial switch v in attr {
case String: case String:
return transmute([]byte)string(v), true return transmute([]byte)string(v), true
case Number: case DDB_Number:
return transmute([]byte)string(v), true return encode_ddb_number_for_sort(v), true
case Binary: case Binary:
return transmute([]byte)string(v), true return transmute([]byte)string(v), true
} }

13
dynamodb/item_codec.odin
View File

@@ -83,11 +83,6 @@ encode_attribute_value :: proc(buf: ^bytes.Buffer, attr: Attribute_Value) -> boo
encode_varint(buf, len(num_str)) encode_varint(buf, len(num_str))
bytes.buffer_write_string(buf, num_str) bytes.buffer_write_string(buf, num_str)
case Number:
bytes.buffer_write_byte(buf, u8(Type_Tag.Number))
encode_varint(buf, len(v))
bytes.buffer_write_string(buf, string(v))
case Binary: case Binary:
bytes.buffer_write_byte(buf, u8(Type_Tag.Binary)) bytes.buffer_write_byte(buf, u8(Type_Tag.Binary))
encode_varint(buf, len(v)) encode_varint(buf, len(v))
@@ -119,14 +114,6 @@ encode_attribute_value :: proc(buf: ^bytes.Buffer, attr: Attribute_Value) -> boo
bytes.buffer_write_string(buf, s) bytes.buffer_write_string(buf, s)
} }
case Number_Set:
bytes.buffer_write_byte(buf, u8(Type_Tag.Number_Set))
encode_varint(buf, len(v))
for n in v {
encode_varint(buf, len(n))
bytes.buffer_write_string(buf, n)
}
case Binary_Set: case Binary_Set:
bytes.buffer_write_byte(buf, u8(Type_Tag.Binary_Set)) bytes.buffer_write_byte(buf, u8(Type_Tag.Binary_Set))
encode_varint(buf, len(v)) encode_varint(buf, len(v))

13
dynamodb/json.odin
View File

@@ -324,9 +324,6 @@ serialize_attribute_value :: proc(b: ^strings.Builder, attr: Attribute_Value) {
case String: case String:
fmt.sbprintf(b, `{"S":"%s"}`, string(v)) fmt.sbprintf(b, `{"S":"%s"}`, string(v))
case Number:
fmt.sbprintf(b, `{"N":"%s"}`, string(v))
case DDB_Number: case DDB_Number:
num_str := format_ddb_number(v) num_str := format_ddb_number(v)
fmt.sbprintf(b, `{"N":"%s"}`, num_str) fmt.sbprintf(b, `{"N":"%s"}`, num_str)
@@ -350,16 +347,6 @@ serialize_attribute_value :: proc(b: ^strings.Builder, attr: Attribute_Value) {
} }
strings.write_string(b, "]}") strings.write_string(b, "]}")
case Number_Set:
strings.write_string(b, `{"NS":[`)
for n, i in v {
if i > 0 {
strings.write_string(b, ",")
}
fmt.sbprintf(b, `"%s"`, n)
}
strings.write_string(b, "]}")
case DDB_Number_Set: case DDB_Number_Set:
strings.write_string(b, `{"NS":[`) strings.write_string(b, `{"NS":[`)
for num, i in v { for num, i in v {

2
dynamodb/number.odin
View File

@@ -494,7 +494,7 @@ f64_to_ddb_number :: proc(val: f64) -> (DDB_Number, bool) {
return parse_ddb_number(str) return parse_ddb_number(str)
} }
// Format a DDB_Number for display (like format_number but preserves precision) // Format a DDB_Number for display
format_ddb_number :: proc(num: DDB_Number) -> string { format_ddb_number :: proc(num: DDB_Number) -> string {
// Normalize first // Normalize first
norm := normalize_ddb_number(num) norm := normalize_ddb_number(num)

59
dynamodb/storage.odin
View File

@@ -243,7 +243,13 @@ serialize_table_metadata :: proc(metadata: ^Table_Metadata) -> ([]byte, bool) {
// Add other metadata // Add other metadata
meta_item["TableStatus"] = String(strings.clone(table_status_to_string(metadata.table_status))) meta_item["TableStatus"] = String(strings.clone(table_status_to_string(metadata.table_status)))
meta_item["CreationDateTime"] = Number(fmt.aprint(metadata.creation_date_time)) ts_str := fmt.aprint(metadata.creation_date_time)
ts_num, ts_ok := parse_ddb_number(ts_str)
if ts_ok {
meta_item["CreationDateTime"] = ts_num
} else {
meta_item["CreationDateTime"] = String(strings.clone(ts_str))
}
// Encode GSI definitions as JSON string // Encode GSI definitions as JSON string
if gsis, has_gsis := metadata.global_secondary_indexes.?; has_gsis && len(gsis) > 0 { if gsis, has_gsis := metadata.global_secondary_indexes.?; has_gsis && len(gsis) > 0 {
@@ -314,8 +320,9 @@ deserialize_table_metadata :: proc(data: []byte, allocator: mem.Allocator) -> (T
// Parse creation date time // Parse creation date time
if time_val, found := meta_item["CreationDateTime"]; found { if time_val, found := meta_item["CreationDateTime"]; found {
#partial switch v in time_val { #partial switch v in time_val {
case Number: case DDB_Number:
val, parse_ok := strconv.parse_i64(string(v)) num_str := format_ddb_number(v)
val, parse_ok := strconv.parse_i64(num_str)
metadata.creation_date_time = val if parse_ok else 0 metadata.creation_date_time = val if parse_ok else 0
} }
} }
@@ -1226,6 +1233,44 @@ evaluate_sort_key_condition :: proc(item: Item, skc: ^Sort_Key_Condition) -> boo
return false return false
} }
// Use numeric comparison if both sides are DDB_Number
item_num, item_is_num := attr.(DDB_Number)
cond_num, cond_is_num := skc.value.(DDB_Number)
if item_is_num && cond_is_num {
cmp := compare_ddb_numbers(item_num, cond_num)
switch skc.operator {
case .EQ:
return cmp == 0
case .LT:
return cmp < 0
case .LE:
return cmp <= 0
case .GT:
return cmp > 0
case .GE:
return cmp >= 0
case .BETWEEN:
if v2, has_v2 := skc.value2.?; has_v2 {
upper_num, upper_ok := v2.(DDB_Number)
if !upper_ok {
return false
}
cmp2 := compare_ddb_numbers(item_num, upper_num)
return cmp >= 0 && cmp2 <= 0
}
return false
case .BEGINS_WITH:
// BEGINS_WITH on numbers: fall through to string comparison
item_str := format_ddb_number(item_num)
cond_str := format_ddb_number(cond_num)
return strings.has_prefix(item_str, cond_str)
}
return false
}
// Fallback: string comparison for S/B types
item_sk_str, ok1 := attr_value_to_string_for_compare(attr) item_sk_str, ok1 := attr_value_to_string_for_compare(attr)
if !ok1 { if !ok1 {
return false return false
@@ -1272,8 +1317,10 @@ attr_value_to_string_for_compare :: proc(attr: Attribute_Value) -> (string, bool
#partial switch v in attr { #partial switch v in attr {
case String: case String:
return string(v), true return string(v), true
case Number: case DDB_Number:
return string(v), true // Return formatted string for fallback string comparison
// (actual numeric comparison is handled in compare_attribute_values)
return format_ddb_number(v), true
case Binary: case Binary:
return string(v), true return string(v), true
} }
@@ -1318,7 +1365,7 @@ validate_item_key_types :: proc(
#partial switch _ in attr { #partial switch _ in attr {
case String: case String:
match = (et == .S) match = (et == .S)
case Number: case DDB_Number:
match = (et == .N) match = (et == .N)
case Binary: case Binary:
match = (et == .B) match = (et == .B)

76
dynamodb/types.odin
View File

@@ -5,28 +5,24 @@ import "core:strings"
// DynamoDB AttributeValue - the core data type // DynamoDB AttributeValue - the core data type
Attribute_Value :: union { Attribute_Value :: union {
String, // S String, // S
Number, // N = stored as string, I'm keeping this so we can still do the parsing/serialization DDB_Number, // N — decimal-preserving numeric type
DDB_Number, // N Dynamo uses whole numbers and not floats or strings so we'll make that its own type Binary, // B (base64)
Binary, // B (base64) Bool, // BOOL
Bool, // BOOL Null, // NULL
Null, // NULL String_Set, // SS
String_Set, // SS DDB_Number_Set, // NS
Number_Set, // NS Binary_Set, // BS
Binary_Set, // BS List, // L
DDB_Number_Set,// BS Map, // M
List, // L
Map, // M
} }
String :: distinct string String :: distinct string
Number :: distinct string
Binary :: distinct string Binary :: distinct string
Bool :: distinct bool Bool :: distinct bool
Null :: distinct bool Null :: distinct bool
String_Set :: distinct []string String_Set :: distinct []string
Number_Set :: distinct []string
DDB_Number_Set :: distinct []DDB_Number DDB_Number_Set :: distinct []DDB_Number
Binary_Set :: distinct []string Binary_Set :: distinct []string
List :: distinct []Attribute_Value List :: distinct []Attribute_Value
@@ -63,7 +59,7 @@ key_from_item :: proc(item: Item, key_schema: []Key_Schema_Element) -> (Key, boo
// Validate that key is a scalar type (S, N, or B) // Validate that key is a scalar type (S, N, or B)
#partial switch _ in attr { #partial switch _ in attr {
case String, Number, Binary: case String, DDB_Number, Binary:
// Valid key type // Valid key type
case: case:
return {}, false return {}, false
@@ -119,12 +115,11 @@ key_get_values :: proc(key: ^Key) -> (Key_Values, bool) {
#partial switch v in key.pk { #partial switch v in key.pk {
case String: case String:
pk_bytes = transmute([]byte)string(v) pk_bytes = transmute([]byte)string(v)
case Number: case DDB_Number:
pk_bytes = transmute([]byte)string(v) pk_bytes = encode_ddb_number_for_sort(v)
case Binary: case Binary:
pk_bytes = transmute([]byte)string(v) pk_bytes = transmute([]byte)string(v)
case: case:
// Keys should only be scalar types (S, N, or B)
return {}, false return {}, false
} }
@@ -133,12 +128,11 @@ key_get_values :: proc(key: ^Key) -> (Key_Values, bool) {
#partial switch v in sk { #partial switch v in sk {
case String: case String:
sk_bytes = transmute([]byte)string(v) sk_bytes = transmute([]byte)string(v)
case Number: case DDB_Number:
sk_bytes = transmute([]byte)string(v) sk_bytes = encode_ddb_number_for_sort(v)
case Binary: case Binary:
sk_bytes = transmute([]byte)string(v) sk_bytes = transmute([]byte)string(v)
case: case:
// Keys should only be scalar types
return {}, false return {}, false
} }
} }
@@ -369,13 +363,27 @@ error_to_response :: proc(err_type: DynamoDB_Error_Type, message: string) -> str
// Build an Attribute_Value with the correct scalar type from raw bytes // Build an Attribute_Value with the correct scalar type from raw bytes
build_attribute_value_with_type :: proc(raw_bytes: []byte, attr_type: Scalar_Attribute_Type) -> Attribute_Value { build_attribute_value_with_type :: proc(raw_bytes: []byte, attr_type: Scalar_Attribute_Type) -> Attribute_Value {
owned := strings.clone(string(raw_bytes))
switch attr_type { switch attr_type {
case .S: return String(owned) case .S:
case .N: return Number(owned) return String(strings.clone(string(raw_bytes)))
case .B: return Binary(owned) case .N:
// Key bytes are canonical-encoded via encode_ddb_number_for_sort.
// Decode them back to a DDB_Number.
ddb_num, ok := decode_ddb_number_from_sort(raw_bytes)
if ok {
return clone_ddb_number(ddb_num)
}
// Fallback: try interpreting as a plain numeric string
fallback_num, fb_ok := parse_ddb_number(string(raw_bytes))
if fb_ok {
return fallback_num
}
// Last resort — return as string (shouldn't happen)
return String(strings.clone(string(raw_bytes)))
case .B:
return Binary(strings.clone(string(raw_bytes)))
} }
return String(owned) return String(strings.clone(string(raw_bytes)))
} }
// Deep copy an attribute value // Deep copy an attribute value
@@ -383,8 +391,6 @@ attr_value_deep_copy :: proc(attr: Attribute_Value) -> Attribute_Value {
switch v in attr { switch v in attr {
case String: case String:
return String(strings.clone(string(v))) return String(strings.clone(string(v)))
case Number:
return Number(strings.clone(string(v)))
case DDB_Number: case DDB_Number:
return clone_ddb_number(v) return clone_ddb_number(v)
case Binary: case Binary:
@@ -399,12 +405,6 @@ attr_value_deep_copy :: proc(attr: Attribute_Value) -> Attribute_Value {
ss[i] = strings.clone(s) ss[i] = strings.clone(s)
} }
return String_Set(ss) return String_Set(ss)
case Number_Set:
ns := make([]string, len(v))
for n, i in v {
ns[i] = strings.clone(n)
}
return Number_Set(ns)
case DDB_Number_Set: case DDB_Number_Set:
ddb_ns := make([]DDB_Number, len(v)) ddb_ns := make([]DDB_Number, len(v))
for num, i in v { for num, i in v {
@@ -441,8 +441,6 @@ attr_value_destroy :: proc(attr: ^Attribute_Value) {
case DDB_Number: case DDB_Number:
delete(v.integer_part) delete(v.integer_part)
delete(v.fractional_part) delete(v.fractional_part)
case Number:
delete(string(v))
case Binary: case Binary:
delete(string(v)) delete(string(v))
case String_Set: case String_Set:
@@ -451,12 +449,6 @@ attr_value_destroy :: proc(attr: ^Attribute_Value) {
} }
slice := v slice := v
delete(slice) delete(slice)
case Number_Set:
for n in v {
delete(n)
}
slice := v
delete(slice)
case DDB_Number_Set: case DDB_Number_Set:
for num in v { for num in v {
delete(num.integer_part) delete(num.integer_part)

89
dynamodb/update.odin
View File

@@ -13,8 +13,6 @@
package dynamodb package dynamodb
import "core:encoding/json" import "core:encoding/json"
import "core:fmt"
import "core:strconv"
import "core:strings" import "core:strings"
// ============================================================================ // ============================================================================
@@ -710,7 +708,7 @@ execute_update_plan :: proc(item: ^Item, plan: ^Update_Plan) -> bool {
if existing, found := item[action.path]; found { if existing, found := item[action.path]; found {
// If existing is a number, add numerically // If existing is a number, add numerically
#partial switch v in existing { #partial switch v in existing {
case Number: case DDB_Number:
result, add_ok := numeric_add(existing, action.value) result, add_ok := numeric_add(existing, action.value)
if !add_ok { if !add_ok {
return false return false
@@ -732,13 +730,13 @@ execute_update_plan :: proc(item: ^Item, plan: ^Update_Plan) -> bool {
return false return false
} }
case Number_Set: case DDB_Number_Set:
if new_ns, is_ns := action.value.(Number_Set); is_ns { if new_ns, is_ns := action.value.(DDB_Number_Set); is_ns {
merged := set_union_strings(([]string)(v), ([]string)(new_ns)) merged := set_union_ddb_numbers(([]DDB_Number)(v), ([]DDB_Number)(new_ns))
old_copy := existing old_copy := existing
attr_value_destroy(&old_copy) attr_value_destroy(&old_copy)
delete_key(item, action.path) delete_key(item, action.path)
item[strings.clone(action.path)] = Number_Set(merged) item[strings.clone(action.path)] = DDB_Number_Set(merged)
} else { } else {
return false return false
} }
@@ -769,14 +767,14 @@ execute_update_plan :: proc(item: ^Item, plan: ^Update_Plan) -> bool {
} }
} }
case Number_Set: case DDB_Number_Set:
if del_ns, is_ns := action.value.(Number_Set); is_ns { if del_ns, is_ns := action.value.(DDB_Number_Set); is_ns {
result := set_difference_strings(([]string)(v), ([]string)(del_ns)) result := set_difference_ddb_numbers(([]DDB_Number)(v), ([]DDB_Number)(del_ns))
old_copy := existing old_copy := existing
attr_value_destroy(&old_copy) attr_value_destroy(&old_copy)
delete_key(item, action.path) delete_key(item, action.path)
if len(result) > 0 { if len(result) > 0 {
item[strings.clone(action.path)] = Number_Set(result) item[strings.clone(action.path)] = DDB_Number_Set(result)
} else { } else {
delete(result) delete(result)
} }
@@ -810,30 +808,17 @@ numeric_add :: proc(a: Attribute_Value, b: Attribute_Value) -> (Attribute_Value,
} }
numeric_subtract :: proc(a: Attribute_Value, b: Attribute_Value) -> (Attribute_Value, bool) { numeric_subtract :: proc(a: Attribute_Value, b: Attribute_Value) -> (Attribute_Value, bool) {
a_num, a_ok := a.(Number) a_num, a_ok := a.(DDB_Number)
b_num, b_ok := b.(Number) b_num, b_ok := b.(DDB_Number)
if !a_ok || !b_ok { if !a_ok || !b_ok {
return nil, false return nil, false
} }
a_val, a_parse := strconv.parse_f64(string(a_num)) result, result_ok := subtract_ddb_numbers(a_num, b_num)
b_val, b_parse := strconv.parse_f64(string(b_num)) if !result_ok {
if !a_parse || !b_parse {
return nil, false return nil, false
} }
return result, true
result := a_val - b_val
result_str := format_number(result)
return Number(result_str), true
}
format_number :: proc(val: f64) -> string {
// If it's an integer, format without decimal point
int_val := i64(val)
if f64(int_val) == val {
return fmt.aprintf("%d", int_val)
}
return fmt.aprintf("%g", val)
} }
// ============================================================================ // ============================================================================
@@ -873,6 +858,52 @@ set_difference_strings :: proc(a: []string, b: []string) -> []string {
return result[:] return result[:]
} }
// Union of two DDB_Number slices (dedup by numeric equality)
set_union_ddb_numbers :: proc(a: []DDB_Number, b: []DDB_Number) -> []DDB_Number {
result := make([dynamic]DDB_Number)
// Add all from a
for num in a {
append(&result, clone_ddb_number(num))
}
// Add from b if not already present
for num in b {
found := false
for existing in result {
if compare_ddb_numbers(existing, num) == 0 {
found = true
break
}
}
if !found {
append(&result, clone_ddb_number(num))
}
}
return result[:]
}
// Difference: elements in a that are NOT in b
set_difference_ddb_numbers :: proc(a: []DDB_Number, b: []DDB_Number) -> []DDB_Number {
result := make([dynamic]DDB_Number)
for num in a {
in_b := false
for del in b {
if compare_ddb_numbers(num, del) == 0 {
in_b = true
break
}
}
if !in_b {
append(&result, clone_ddb_number(num))
}
}
return result[:]
}
// ============================================================================ // ============================================================================
// Request Parsing Helper // Request Parsing Helper
// ============================================================================ // ============================================================================