Here are a few things you might find interesting:
- This is fully end to end encrypted! We use the WISP protocol for TCP-over-websockets.
- There is a novel WASM->JS JIT for experimental site speedup
- This port cost over 25k in opus/fable tokens for debugging and JIT research
This was just a fun experiment to push the boundaries of WebAssembly. For a more usable "browser in browser" experience, we also built https://github.com/HeyPuter/browser.js that eats a bit less RAM.
It has the following characteristics:
- SOTA decompression throughput in its ratio class
- Decent ratios (comparable to LZ4 at high effort levels)
- Slow compression
Results on the tarred Silesia corpus on Intel x86-64 follow:
codec decode ratio encode
misa77 -0 5219 MB/s 42.64% 54.5 MB/s
misa77 -1 4274 MB/s 39.65% 51.2 MB/s
lz4 2505 MB/s 47.59% 371 MB/s
lz4hc -12 2531 MB/s 36.45% 7.31 MB/sI have zero-allocation on the hot paths, memory segments for prompts and tokens are allocated once inside confined Arenas. Raw pointers pass straight through down to the low C level. This avoids primitive array cloning and heap churn.
I mapped out the native structures from llama.cpp and whisper.cpp while matching the compiler's padding to maintain safe memory access.
I bundle pre-compiled native binaries in the jar for easy deployment.
This execution engine provides the foundation I need for work I'm doing on a spatio-temporal memory layer (L-TABB) to replace RAGs. I'd love to get technical feedback to polish any issues while I continue working on the next layer. Deep-dives from anyone hacking on Project Panama or low-latency systems in modern JDK would be very appreciated!
I'm much better with code than prose, so I'll let the code do most of the talking.
Happy Hacking! /David
Code: https://libargus.cc Project Landing Page: https://projectargus.cc