Skip to content

feat: optional packed J layout to cut JSON memory on small MCUs - #252

Open
rayozzie wants to merge 2 commits into
masterfrom
feat/j-storage-optimization
Open

feat: optional packed J layout to cut JSON memory on small MCUs#252
rayozzie wants to merge 2 commits into
masterfrom
feat/j-storage-optimization

Conversation

@rayozzie

@rayozzie rayozzie commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

We are running out of RAM on small MCU hosts, and the JSON object model is where it goes. A parsed document costs 5–9× the size of the text it came from, because every object member is three separate heap allocations — the node, the key, and the value string — and each one pays the allocator's per-chunk header and rounding.

NOTE_C_STORAGE_OPTIMIZATION packs a member's key and short string value into the node's own allocation, so that member costs one allocation instead of three. Members are also reordered to remove alignment padding, taking a node from 48 to 40 bytes on a 32-bit target. valueint and valuenumber are unused on a string node, so their bytes carry the value and then the key; anything that doesn't fit still goes to the heap, and a setter that needs those bytes back evacuates first.

Backward compatible. The historical layout is untouched and remains the default, so nothing shifts unless you opt in. Both layouts render byte-identical JSON from one shared parser and printer. CI builds and tests both, crossed with single precision and low memory.

Results

Measured on a 32-bit target with newlib dlmalloc (4 B header, 8 B align, 16 B minimum chunk).

JSON object text objs allocations bytes saved
{"req":"card.version"} 22 2 5 → 2 152 → 96 37%
{"req":"note.add","file":"data.qo","sync":true} 47 4 11 → 4 304 → 200 34%
{"req":"hub.set","product":"com.blues.airnote",…} 93 6 17 → 6 472 → 312 34%
{"req":"note.add","file":"air.qo","body":{6 readings}} 122 10 23 → 10 736 → 536 27%
{"req":"card.location","status":"GPS updated",…} 91 6 17 → 6 472 → 288 39%
{"err":"note: no notes available {note-noexist}"} 49 2 5 → 2 176 → 120 32%
{"req":"env.get","name":"monitor-pump","text":"enabled"} 56 4 13 → 4 328 → 192 41%
{"device":"dev:864475044204278","sn":"pump-A17",…} 75 4 13 → 4 336 → 208 38%
{"a":"b","c":"d","e":"f","g":"h","i":"j","k":"l"} 49 7 25 → 7 584 → 336 42%
TOTAL 604 45 129 → 45 3560 → 2288 36%

Best case is many short string members: 25 allocations become 7. Worst case is numeric bodies, where there is no string value to absorb into the node, but the key still packs.

Over a 50-document corpus the totals are 38% fewer allocations and 26% less heap, dropping expansion from 9.0× to 6.7× the JSON text.

Under NOTE_C_SINGLE_PRECISION the saving is 16%, not 26%: a float JNUMBER already removes the historical layout's tail padding, so both layouts are 40 bytes and the win comes only from packing.

Also fixed

Two pre-existing printer defects, found by comparing output against a pristine build of the base revision:

  • Strings containing more than one escaped control character were truncated and emitted invalid JSON.
  • Formatted + omitempty printing emitted indentation for members it then elided.

The original idea that I gave to Claude to implement

Screenshot 2026-08-04 at 9 26 00 PM

We are running out of RAM on small MCU hosts, and the JSON object model is
where it goes. A parsed document costs 5-9x the size of the text it came
from, because every object member is three separate heap allocations -- the
node, the key, and the value string -- and each one pays the allocator's
per-chunk header and rounding.

NOTE_C_STORAGE_OPTIMIZATION packs a member's key and short string value into
the node's own allocation, so that member costs ONE allocation instead of
three. Members are also reordered to remove alignment padding, taking a node
from 48 to 40 bytes on a 32-bit target. valueint and valuenumber are unused
on a string node, so their bytes carry the value and then the key; anything
that doesn't fit still goes to the heap, and a setter that needs those bytes
back evacuates first.

Backward compatible: the historical layout is untouched and remains the
default, so nothing shifts unless you opt in. Both layouts render
byte-identical JSON from one shared parser and printer. CI builds and tests
both, crossed with single precision and low memory.

Measured on a 32-bit target, newlib dlmalloc (4B header, 8B align, 16B min):

  JSON object                                              txt  obj   allocs      bytes    saved
  -------------------------------------------------------  ---  ---  --------  ----------  -----
  {"req":"card.version"}                                    22    2   5 ->  2   152 ->  96   37%
  {"req":"note.add","file":"data.qo","sync":true}           47    4  11 ->  4   304 -> 200   34%
  {"req":"hub.set","product":"com.blues.airnote",...}       93    6  17 ->  6   472 -> 312   34%
  {"req":"note.add","file":"air.qo","body":{6 readings}}   122   10  23 -> 10   736 -> 536   27%
  {"req":"card.location","status":"GPS updated",...}        91    6  17 ->  6   472 -> 288   39%
  {"err":"note: no notes available {note-noexist}"}         49    2   5 ->  2   176 -> 120   32%
  {"req":"env.get","name":"monitor-pump","text":"enabled"}  56    4  13 ->  4   328 -> 192   41%
  {"device":"dev:864475044204278","sn":"pump-A17",...}      75    4  13 ->  4   336 -> 208   38%
  {"a":"b","c":"d","e":"f","g":"h","i":"j","k":"l"}         49    7  25 ->  7   584 -> 336   42%
  -------------------------------------------------------  ---  ---  --------  ----------  -----
  TOTAL                                                    604   45 129 -> 45  3560 -> 2288   36%

Best case is many short string members: 25 allocations become 7. Worst case
is numeric bodies, where there is no string value to absorb into the node,
but the key still packs. Over a 50-document corpus the totals are 38% fewer
allocations and 26% less heap, dropping expansion from 9.0x to 6.7x.

Under NOTE_C_SINGLE_PRECISION the saving is 16%, not 26%: a float JNUMBER
already removes the historical layout's tail padding, so both layouts are 40
bytes and the win comes only from packing.

Also fixes two pre-existing printer defects found by comparing output against
a pristine build of the base revision. Strings containing more than one
escaped control character were truncated and emitted invalid JSON, and
formatted+omitempty printing emitted indentation for members it then elided.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01Hb7xkJBH7VWvvayVHsNZaA
@rayozzie
rayozzie requested a review from zfields August 5, 2026 01:10
…tion

All three surfaced only under conditions the local macOS runs don't have:
valgrind, and 32-bit i386.

1. JAddIntToObject lost integer precision under NOTE_C_SINGLE_PRECISION.

   It routed its JINTEGER through JSetIntHelper, which takes JNUMBER — a
   float in that build. A Unix timestamp is exact in an int32 but not in a
   float, so it came back rounded. The base revision assigned valueint
   directly; it does again. Verified in the CI container with the exact CI
   flags (-m32 -mfpmath=sse -msse2):

     9a12cfc base  -> {"time":1705699768}   CI passed
     2f13a66       -> {"time":1705699712}   CI failed
     this commit   -> {"time":1705699768}

   This is what broke NoteGetStatus, NoteGetLocation and
   JSON_number_handling — a real library regression, not a test artifact.

2. Two reference scenarios leaked their fixture.

   Catch2 re-enters a GIVEN once per leaf THEN, and five inspect-only leaves
   never freed anything while those two GIVENs had no trailing cleanup. The
   tracked allocator missed it because it resets per entry; valgrind did not.
   Every leaf now releases the fixture and asserts liveBlocks == 0.

3. The packing assertion assumed a 16-byte inline region.

   On i386 long long is 4-byte aligned, unlike ARM and x86-64, so packed +
   single precision yields sizeof(J) = 36 and a 12-byte inline region:

     layout / precision      sizeof(J)  offsetof(valueint)  inline region
     default / double            40             20              20
     default / single            36             20              16
     packed  / double            40             24              16
     packed  / single            36             24              12

   {"aqi_level":"good"} needs 15 bytes of content, so the library correctly
   grew the node past sizeof(J) and the hardcoded expectation failed. No
   library defect. The expectation is now derived from offsetof and sizeof,
   so it holds on any ABI.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01Hb7xkJBH7VWvvayVHsNZaA
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant