TinyComputers.io (Posts about language design)

Teaching an LLM a Language It Has Never Seen

A.C. Jokela — Thu, 02 Apr 2026 13:00:00 GMT

Lattice is a programming language I designed. Its central feature is the phase system: every runtime value carries a mutability tag that transitions between states the way matter moves between liquid and solid. You declare a variable with flux (mutable) or fix (immutable). You freeze a value to make it immutable, thaw it to get a mutable copy, and sublimate it to make it permanently frozen. forge blocks let you build something mutably and have the result exit as immutable. None of this exists in any other language.

Lattice does not appear in Claude's training data. I designed the language after the knowledge cutoff. There is no Lattice source code on GitHub (other than my own repository). There are no Stack Overflow answers. There is no tutorial ecosystem, no community blog posts, no textbook chapters. The only documentation that exists is the code itself, a 38-chapter handbook I wrote, and three blog posts on this site.

Claude writes Lattice fluently. It writes correct programs using the phase system, the concurrency primitives, the module system, and the trait/impl pattern. It writes struct definitions with per-field phase annotations. It uses forge blocks and anneal expressions correctly. And it wrote a 4,955-line self-hosted compiler in Lattice, for Lattice: a complete tokenizer, parser, and bytecode code generator that reads .lat source files and emits .latc bytecode binaries.

The question is how any of this is possible when the model has never seen the language before.

The Rust Smell

The answer starts with syntax. Here is a Lattice function:

fn greet(name: String) -> String {
    return "Hello, ${name}!"
}

And here is the Rust equivalent:

fn greet(name: &str) -> String {
    format!("Hello, {name}!")
}

The fn keyword, the colon-separated type annotations, the -> return type, the curly braces: Claude has seen these patterns millions of times in Rust code. When it encounters them in Lattice, it doesn't need to learn a new syntax. It needs to recognize a familiar one.

This extends deep into the language. Lattice structs look like Rust structs:

struct Point {
    x: Float,
    y: Float
}

Lattice enums look like Rust enums:

enum Shape {
    Circle(Float),
    Rectangle(Float, Float)
}

Lattice match expressions look like Rust match expressions:

match shape {
    Shape::Circle(r) => pi() * r * r,
    Shape::Rectangle(w, h) => w * h,
    _ => 0.0
}

Lattice traits and impl blocks look like Rust traits and impl blocks:

trait Printable {
    fn display(self: any) -> String
}

impl Printable for Point {
    fn display(self: any) -> String {
        return "(${self.x}, ${self.y})"
    }
}

Closures use the same |params| body syntax. The .. range operator works the same way. The ? postfix operator propagates errors. for item in collection iterates. let binds variables. The structural similarity is pervasive enough that a model trained on Rust can parse and generate Lattice code without any Lattice-specific training.

I did not design Lattice to be AI-friendly. I designed it because Rust's syntax is good and I wanted to use it for a language with different semantics. But the side effect is that Claude can write Lattice from day one because the syntax activates the same neural pathways that Rust does. The model doesn't know it's writing a different language. It knows it's writing code that looks like Rust, and the structural patterns transfer.

The Phase System: Where Familiarity Ends

The Rust resemblance carries Claude through basic Lattice programs without difficulty. Where it gets interesting is the phase system, because this is where Lattice has no analog in any language Claude has seen.

In Rust, mutability is a static property: let mut x = 5; or let x = 5;. You decide at declaration time and the compiler enforces it. In Lattice, mutability is a runtime state that values transition through:

flux counter = 0          // mutable
counter = counter + 1     // allowed: counter is fluid

freeze(counter)           // transition: fluid → crystal
counter = counter + 1     // runtime error: counter is crystal

flux copy = thaw(counter) // get a mutable copy
copy = copy + 1           // allowed: copy is fluid

Claude handles this correctly. When I describe the phase system and provide examples, Claude generates code that uses flux and fix declarations appropriately, calls freeze() at the right points, and avoids mutating crystal values. The model maps flux to "mutable variable" and fix to "immutable variable" in its internal representation, and the transition functions (freeze, thaw) become explicit state changes that it tracks through the program.

The harder constructs are the ones with no familiar analog.

forge blocks are mutable construction zones whose output exits as immutable:

fix config = forge {
    flux c = {}
    c.host = "localhost"
    c.port = 8080
    c.debug = false
    c   // exits the forge block as crystal
}
// config is now crystal; cannot be modified

Claude gets this right because the pattern (build something mutably, freeze the result) maps to the builder pattern in Rust and other languages. The syntax is novel but the concept isn't.

anneal is harder. It temporarily thaws a crystal value into a mutable binding for the duration of a block, then re-freezes it:

fix settings = forge { flux s = {}; s.theme = "dark"; s }

anneal(settings) |s| {
    s.theme = "light"   // temporarily mutable
}
// settings is crystal again, with theme = "light"

Claude produces correct anneal code when given the semantics, but it occasionally generates patterns that would work in Rust (taking a &mut reference) but don't apply in Lattice (where anneal is the only way to modify a crystal value in place). The model's Rust intuitions are strong enough to produce syntactically valid Lattice but sometimes semantically incorrect programs, because it defaults to Rust's mutation model when the Lattice-specific construct is unfamiliar.

The reactive phase system is where Claude needs the most guidance. react, bond, and seed have no precedent in any mainstream language:

flux temperature = 72.0

react("temperature", fn(name, old_phase, new_phase) {
    print("${name} changed from ${old_phase} to ${new_phase}")
})

freeze(temperature)  // triggers the reaction callback

flux primary = "active"
flux mirror = "active"

bond("mirror", "primary", "sync")  // when primary changes phase, mirror follows

freeze(primary)  // mirror also freezes

Claude can produce these patterns when given the API, but it doesn't intuit them. It never suggests react or bond unprompted, because there's nothing in its training data that would trigger the association. These constructs must be taught explicitly. The Rust smell gets Claude through 80% of Lattice. The last 20% requires actual specification.

The Spectrum of Difficulty

Working with Claude on Lattice code over several months has revealed a clear gradient of difficulty:

Trivial (Rust transfer): Functions, structs, enums, match expressions, closures, for loops, string interpolation, module imports, error propagation with ?. Claude writes these correctly on the first attempt because they're syntactically identical to Rust.

Easy (new vocabulary, familiar concept): flux/fix declarations, freeze()/thaw() calls, basic phase checking. Claude maps these to mutable/immutable patterns it already knows. The vocabulary is new; the concept isn't.

Moderate (new pattern, teachable): forge blocks, anneal expressions, crystallize blocks, struct field-level phase annotations (alloy structs). These require explanation, but once Claude sees one or two examples, it generalizes correctly. The builder pattern and block-scoped mutation are close enough to existing patterns that the model bridges the gap.

Hard (no analog, requires specification): Reactive phase operations (react, bond, seed), phase pattern matching (fluid val =>, crystal val =>), the concurrency constraint that only crystal values can be sent on channels, strict mode's consumption semantics for freeze. Claude can use these but never invents them. They must be explicitly described.

The concurrency constraint is a good example of the "hard" category. In Lattice, data sent on a channel must be crystal:

let ch = Channel::new()
flux data = "mutable"

// ch.send(data)  // runtime error: cannot send fluid value

freeze(data)
ch.send(data)     // works: data is now crystal

This rule exists because crystal values are deeply immutable: they can't be modified by the sender after transmission, which eliminates data races structurally. Claude understands the concept (Rust has Send and Sync traits that serve a similar purpose), but it doesn't automatically apply Lattice's specific rule without being told. Left to its own devices, Claude will try to send fluid values on channels, because that's what you'd do in Go or Python. The constraint must be stated.

Strict mode (#mode strict at the top of a file) is another case where Claude needs explicit guidance. In strict mode, let is banned (you must use flux or fix), freeze() consumes the original binding (Rust-like move semantics), and crystal bindings cannot be assigned to at all, not even as a runtime error. Claude can write strict-mode Lattice, but it defaults to casual-mode patterns unless reminded. The model's prior is "permissive runtime" because that's what most dynamic languages are.

The gradient correlates exactly with how much the construct resembles something in Rust or another mainstream language. When the syntax is familiar, Claude's transfer learning handles it. When the concept is familiar but the syntax is new, one or two examples are enough. When both the syntax and the concept are novel, Claude needs the specification.

The Self-Hosted Compiler

The strongest evidence that Claude can deeply understand a language it was never trained on is latc.lat: a 4,955-line self-hosted compiler written in Lattice, for Lattice.

The compiler reads .lat source files and emits .latc bytecode binaries. It has twelve sections:

Opcode constant definitions (mapping all 100+ VM opcodes to integers)
Token stream and cursor helpers (peek, advance, expect, match_tok)
Compiler state management (save/restore for nested compilation)
Error reporting
Bytecode emit helpers (emit_byte, emit_jump, patch_jump, emit_loop)
Constant pool management (integers, floats, strings, closures)
Scope and variable resolution (begin_scope, end_scope, resolve_local, upvalue tracking)
Expression parsing (precedence climbing, binary/unary ops, calls, field access)
Statement compilation (let/flux/fix, if/while/for, return, match, try/catch)
Declaration compilation (functions, structs, enums, traits, impl blocks)
Binary serialization (writing the LATC file format with magic bytes, version header, chunk data)
Main entry point

Claude wrote this. Not "Claude assisted with this" or "Claude generated boilerplate for this." Claude wrote a recursive descent parser for Lattice's grammar, a bytecode compiler that emits correct opcodes for the phase system, and a binary serializer that produces files the C runtime can load and execute. The compiler bootstraps: you run it with the C-based clat interpreter, and it produces bytecode that the same interpreter executes.

The compiler itself uses Lattice's phase system for its own internal state. The compiler's mutable working data (the bytecode buffer, the constant pool, the local variable tracking arrays) is declared with flux:

flux code = []
flux c_lines = []
flux constants = []
flux local_name_arr = []
flux local_depth_arr = []
flux local_captured_arr = []
flux local_count = 0

This is the compiler eating its own dogfood. The mutable state that the compiler needs to build bytecode is declared using the same phase system that the compiler is compiling. The phase keywords aren't decorative here; they're structurally necessary because the compiler modifies these arrays on every opcode emission and scope transition.

The compiler has 118 functions across 12 sections, with 554 opcode references. It handles every construct in the language: flux/fix declarations, forge blocks, freeze/thaw/sublimate calls, anneal and crystallize expressions, struct and enum definitions with phase annotations, trait/impl blocks, match expressions with phase-aware pattern matching, structured concurrency with scope/spawn, channel operations, try/catch, defer, and the complete expression grammar with correct operator precedence.

Writing a self-hosted compiler requires understanding the language at every level simultaneously. The tokenizer must know every keyword, operator, and delimiter. The parser must handle every grammatical production, including the phase-specific constructs (forge, anneal, crystallize) that exist nowhere in Claude's training data. The code generator must emit the correct opcodes for phase transitions, reactive bindings, and structured concurrency. And the whole thing must be written in the language being compiled, which means Claude is writing Lattice to compile Lattice, using constructs it learned from examples rather than training data.

The compiler's serialization section writes the LATC binary format byte by byte:

fn serialize_latc(ch: any) {
    ser_buf = Buffer::new(0)

    // Header: "LATC" + version(1) + reserved(0)
    write_u8(76)    // 'L'
    write_u8(65)    // 'A'
    write_u8(84)    // 'T'
    write_u8(67)    // 'C'
    write_u16_le(1) // format version
    write_u16_le(0) // reserved

    serialize_chunk(ch)
    return ser_buf
}

This is not pattern matching against compiler source code from the training data. No Lattice compiler exists in the training data. Claude wrote a compiler for a language that has no prior art, in a language that has no prior art, producing a binary format that has no prior art. Every decision (the magic bytes, the chunk serialization order, the upvalue encoding) came from understanding the specification I provided and the runtime behavior of the C-based interpreter.

What I Actually Gave Claude

The teaching process was less structured than you might expect. There was no formal curriculum, no staged introduction of concepts, no carefully sequenced lesson plan. And I should be honest about the recursive nature of what happened: Claude Code was the primary tool for building Lattice itself. The language, the C implementation, the grammar, the runtime, the test suite, the handbook: all of it was built with Claude Code. I designed the language and directed the implementation, but Claude wrote the C, the LaTeX, and the example programs.

So the situation is: Claude wrote Lattice (the implementation), and then Claude wrote in Lattice (the programs and the self-hosted compiler). The model built the language and then learned the language it built. The "teaching material" that Claude uses to write Lattice code is documentation and examples that Claude itself produced in earlier sessions.

The artifacts:

The C implementation: ~80 source files, the parser, the VM, the phase system runtime. Built with Claude Code from my architectural direction.
A handbook: 38 chapters covering every feature, with worked examples. Written in LaTeX with Claude Code. This lives in a repository that Claude can read in subsequent sessions.
Example programs (examples/phase_demo.lat, examples/sorting.lat, examples/state_machine.lat) that demonstrate idiomatic Lattice. Written by Claude Code.
815 test files under AddressSanitizer that exercise every construct. Written by Claude Code.
An EBNF grammar reference as an appendix to the handbook.

When I work with Claude on Lattice code, I don't paste the entire handbook into the context window. Claude has access to the project directory. It reads files as needed. If I ask it to write a function that uses forge, it reads examples/phase_demo.lat or chapters/ch11-phases-explained.tex to see how forge works. If I ask it to add an opcode to the compiler, it reads include/stackopcode.h and src/stackvm.c to understand the existing instruction set.

The key insight: Claude doesn't need to be trained on a language to write it. It needs access to the specification and examples at inference time. And in this case, those specifications and examples were produced by Claude itself in prior sessions. The model's understanding is constructed on the fly from documentation in its context, not retrieved from weights. This is why the Rust resemblance matters so much: the syntax gives Claude a structural scaffold, and the specification (which Claude wrote) fills in the semantics.

This is also why the self-hosted compiler was possible. By the time Claude wrote latc.lat, it had already written the entire language implementation, the handbook, the test suite, and hundreds of example programs. The language had moved from "novel" to "familiar" through accumulated context, not through training. Each session built on the last. Each example reinforced the phase system's rules. By the time the compiler was attempted, Claude's working understanding of Lattice (constructed from its own prior output) was deep enough to write a 5,000-line program that correctly compiles the language. The model taught itself a language by building the language first.

Why Syntax Matters More Than Semantics

The Lattice experience suggests something counterintuitive about how LLMs interact with programming languages: syntax transfer is more powerful than semantic understanding.

Claude can write correct Lattice because Lattice looks like Rust. The semantic differences (phase system vs. ownership, runtime type checking vs. compile-time guarantees, garbage collection vs. RAII) are significant, but they don't prevent Claude from producing working code. The model generates syntactically valid Lattice from Rust patterns and then adjusts the semantics when corrected.

This has implications for language design. If you want AI tooling to support your language from day one, without waiting for it to appear in training data, design your syntax to rhyme with something popular. Lattice's resemblance to Rust wasn't designed for AI, but it is the reason AI can write it. A language with a radically different syntax (APL, Forth, J) would be much harder for Claude to learn from examples alone, even if the semantics were simpler.

The reverse is also true: a language with familiar syntax but deeply unfamiliar semantics (like Lattice's reactive phase system) will produce code that looks correct but occasionally behaves wrong. Claude's Rust intuitions are strong enough to generate valid-looking phase code, but the model sometimes falls back to Rust's mutation model when the Lattice-specific behavior is more constrained. The syntax transfers perfectly. The semantics require teaching.

Implications for Language Designers

If you're designing a new programming language in 2026, the AI tooling question is unavoidable. Your language won't have IDE plugins, autocompleters, or AI coding assistants on day one. The community doesn't exist yet. The training data doesn't include your language. Every other language your users work with has Copilot or Claude support. Yours doesn't.

Lattice suggests a strategy: make your syntax rhyme with something an LLM already knows.

This isn't about copying Rust. Lattice has genuinely novel semantics. The phase system, the reactive bindings, the alloy structs with per-field phase annotations: none of these exist in Rust. But they're expressed through syntax (keywords, braces, type annotations, block expressions) that maps directly to Rust's structural patterns. Claude can parse the syntax without help and learn the semantics from examples.

The alternative is designing a syntax so novel that LLMs can't bootstrap from existing knowledge. This is a legitimate design choice; some ideas genuinely need new notation. But the cost is high: your users won't get AI assistance until your language appears in training data, which requires the language to become popular first, which is harder without AI assistance. It's a chicken-and-egg problem that familiar syntax sidesteps.

The practical recommendation: novel semantics, familiar syntax. Invent the ideas. Borrow the notation. Let the LLM cross the bridge on syntax and learn the semantics on the other side.

What This Means for the "AI Writes Code" Conversation

The Lattice case study complicates the popular narrative about AI code generation in both directions.

For the optimists who say AI can learn anything: Claude cannot invent the reactive phase system. It cannot propose bond or seed or anneal without being told they exist. The novel constructs, the ones that make Lattice a genuinely different language rather than a Rust reskin, are invisible to the model until explicitly specified. AI transfer learning has limits, and those limits are at the boundaries of what the training data contains.

For the pessimists who say AI can only regurgitate training data: Claude wrote a 5,000-line self-hosted compiler for a language it has never seen. That is not regurgitation. The compiler produces correct bytecode for constructs (phase transitions, reactive bonds, per-field phase annotations) that exist in no other language. The model assembled knowledge from its understanding of compilers generally, Rust syntax specifically, and the Lattice specification I provided, and produced something genuinely new. Antirez called this "assembling knowledge" when he observed the same phenomenon with his Z80 emulator project. I think that's the right term.

The truth is somewhere that neither camp wants to occupy. LLMs can go far beyond their training data when the new territory is structurally adjacent to something they know. They cannot go beyond their training data when the new territory is structurally novel. The boundary between "adjacent" and "novel" is syntax. Familiar syntax is a bridge. Novel syntax is a wall. Novel semantics behind familiar syntax is a trap: the model crosses the bridge confidently and then occasionally falls.

Lattice exists in all three zones simultaneously. Its Rust-like surface lets Claude cross the bridge. Its phase system is the novel semantics behind familiar syntax. And the self-hosted compiler is proof that the bridge, once crossed, supports weight that no one expected.

I didn't set out to test the limits of LLM language understanding when I designed Lattice. I set out to build a programming language with a novel approach to mutability. The AI dimension was a side effect: I used Claude Code as my development tool because I use Claude Code for everything, and the language happened to be learnable because it happened to look like Rust. But the result is one of the more complete demonstrations of LLM transfer learning applied to a genuinely novel domain: not just writing programs in an unfamiliar language, but writing a compiler for that language, in that language, from a specification that exists nowhere in the training data.

The 4,955 lines of latc.lat are the proof that LLMs can go further than their training data when the conditions are right. The conditions are: familiar syntax, clear specification, accessible examples, and a human who knows when the model is wrong. Remove any one of those and the compiler doesn't get written. But with all four in place, the model produces something that works, that compiles, and that no human typed by hand.

A Stack-Based Bytecode VM for Lattice: 100 Opcodes, Serialization, and a Self-Hosted Compiler

A.C. Jokela — Fri, 20 Feb 2026 18:00:00 GMT

When I first wrote about Lattice's move from a tree-walking interpreter to a bytecode VM, the instruction set had 62 opcodes, concurrency primitives still delegated to the tree-walker, and programs couldn't be serialized. The VM was a foundation, correct and complete enough to become the default, but clearly a starting point.

That was ten versions ago. The bytecode VM now has 100 opcodes, compiles concurrency primitives into standalone sub-chunks with zero AST dependency at runtime, ships a binary serialization format for ahead-of-time compilation, includes an ephemeral bump arena for short-lived string temporaries, and (perhaps most satisfyingly) has a self-hosted compiler written entirely in Lattice that produces the same .latc bytecode files as the C implementation.

This post walks through what changed and why. The full technical treatment is available as a research paper; this is the practitioner's version.

Why Keep Going

The original bytecode VM solved the immediate problems: it eliminated recursive AST dispatch overhead and gave Lattice a single execution path for file execution, the REPL, and the WASM playground. But three issues remained.

First, OP_SCOPE and OP_SELECT (Lattice's structured concurrency opcodes) still stored AST node pointers in the constant pool and dropped into the tree-walking evaluator at runtime. This meant the AST had to stay alive during concurrent execution, which defeated one of the main motivations for having a bytecode VM in the first place.

Second, the AST dependency made serialization impossible. You can serialize bytecode to a file, but you can't easily serialize an arbitrary C pointer to an AST node. Programs had to be parsed and compiled on every run.

Third, the dispatch loop used a plain switch statement. Not a crisis, but computed goto dispatch is a well-known improvement for bytecode interpreters, and leaving it on the table felt unnecessary.

All three problems are solved now. Let me start with the instruction set, since everything else builds on it.

100 Opcodes

The instruction set grew from 62 to 100 opcodes, organized into 16 functional categories:

Category	Representative opcodes	Count
Stack manipulation	`CONSTANT`, `NIL`, `TRUE`, `FALSE`, `UNIT`, `POP`, `DUP`, `SWAP`	8
Arithmetic/logical	`ADD`, `SUB`, `MUL`, `DIV`, `MOD`, `NEG`, `NOT`	7
Bitwise	`BIT_AND`, `BIT_OR`, `BIT_XOR`, `BIT_NOT`, `LSHIFT`, `RSHIFT`	6
Comparison	`EQ`, `NEQ`, `LT`, `GT`, `LTEQ`, `GTEQ`	6
String	`CONCAT`	1
Variables	`GET/SET_LOCAL`, `GET/SET/DEFINE_GLOBAL`, `GET/SET_UPVALUE`, `CLOSE_UPVALUE`	8
Control flow	`JUMP`, `JUMP_IF_FALSE`, `JUMP_IF_TRUE`, `JUMP_IF_NOT_NIL`, `LOOP`	5
Functions	`CALL`, `CLOSURE`, `RETURN`	3
Iterators	`ITER_INIT`, `ITER_NEXT`	2
Data structures	`BUILD_ARRAY`, `INDEX`, `SET_INDEX`, `GET_FIELD`, `INVOKE`, etc.	15
Exceptions/defer	`PUSH_EXCEPTION_HANDLER`, `THROW`, `DEFER_PUSH`, `DEFER_RUN`, etc.	6
Phase system	`FREEZE`, `THAW`, `CLONE`, `MARK_FLUID`, `REACT`, `BOND`, `SEED`, etc.	14
Builtins/modules	`PRINT`, `IMPORT`	2
Concurrency	`SCOPE`, `SELECT`	2
Integer fast paths	`INC_LOCAL`, `DEC_LOCAL`, `ADD_INT`, `SUB_INT`, `LOAD_INT8`, etc.	8
Wide variants	`CONSTANT_16`, `GET_GLOBAL_16`, `SET_GLOBAL_16`, `DEFINE_GLOBAL_16`, `CLOSURE_16`	5
Special	`RESET_EPHEMERAL`, `HALT`	2
Total		100

The growth came from three directions: the integer fast-path opcodes (8 new), the wide constant variants (5 new), and the concurrency/arena opcodes. Let me explain each.

Integer Fast Paths

Tight loops like for i in 0..1000 spend most of their time incrementing a counter and comparing it to a bound. The generic OP_ADD has to check whether its operands are integers, floats, or strings (for concatenation), which adds branching overhead on every iteration.

The integer fast-path opcodes (OP_ADD_INT, OP_SUB_INT, OP_MUL_INT, OP_LT_INT, OP_LTEQ_INT) skip the type check entirely and operate directly on int64_t values. OP_INC_LOCAL and OP_DEC_LOCAL handle the i += 1 and i -= 1 patterns as single-byte instructions that modify the stack slot in place, no push or pop required.

OP_LOAD_INT8 encodes a signed byte directly in the instruction stream. The integer 42 becomes two bytes (OP_LOAD_INT8, 0x2A) instead of a three-byte OP_CONSTANT plus an eight-byte constant pool entry. Any integer in [-128, 127] gets this treatment.

Wide Constant Variants

The original instruction set used a single byte for constant pool indices, limiting each chunk to 256 constants. This is fine for most functions, but the self-hosted compiler (a 2,000-line Lattice program compiled as a single top-level script) blows past that limit easily.

OP_CONSTANT_16, OP_GET_GLOBAL_16, OP_SET_GLOBAL_16, OP_DEFINE_GLOBAL_16, and OP_CLOSURE_16 use two-byte big-endian indices, supporting up to 65,536 constants per chunk. The compiler automatically switches to wide variants when an index exceeds 255.

The Compiler

The bytecode compiler performs a single-pass walk over the AST. It maintains a chain of Compiler structs linked via enclosing pointers, one per function being compiled. Variable references resolve through three tiers: local (scan the current compiler's locals array), upvalue (recursively check enclosing compilers), and global (fall through to OP_GET_GLOBAL).

Three compilation modes handle different use cases. compile() is the standard file mode: it compiles all declarations and emits an implicit call to main() if one is defined. compile_module() is for imports, identical to compile() but skips the auto-call. compile_repl() preserves the last expression on the stack as the iteration's return value (displayed with => prefix) and keeps the known-enum table alive across REPL iterations so enum declarations persist.

The compiler implements several optimizations during code generation. Binary operations on literal operands are folded at compile time: 3 + 4 emits a single OP_LOAD_INT8 7 rather than two loads and an OP_ADD. The pattern x += 1 is detected and emitted as the single-byte OP_INC_LOCAL, which modifies the stack slot in place. And every statement is wrapped by compile_stmt_reset(), which appends OP_RESET_EPHEMERAL to trigger the ephemeral arena cleanup.

Computed Goto Dispatch

The dispatch loop now uses GCC/Clang's labels-as-values extension for computed goto:

#ifdef VM_USE_COMPUTED_GOTO
  static void *dispatch_table[] = {
    [OP_CONSTANT] = &&lbl_OP_CONSTANT,
    [OP_NIL]      = &&lbl_OP_NIL,
    // ... all 100 entries
  };
#endif

for (;;) {
    uint8_t op = READ_BYTE();
#ifdef VM_USE_COMPUTED_GOTO
    goto *dispatch_table[op];
#endif
    switch (op) {
        // ...
    }
}

Each opcode handler ends with a goto *dispatch_table[READ_BYTE()] rather than breaking back to the top of the loop. This eliminates the switch statement's bounds check and branch table indirection, replacing it with a single indirect jump. The CPU's branch predictor sees different jump sites for different opcodes, which improves prediction accuracy compared to a single switch that all opcodes funnel through.

On platforms without the extension, it falls back to a standard switch. The VM works correctly either way.

Pre-Compiled Concurrency

This is the change I'm most pleased with, because it solves the problem cleanly.

Lattice has three concurrency primitives: scope defines a concurrent region, spawn launches a task within that region, and select multiplexes over channels. In the tree-walker, these work by passing AST node pointers to spawned threads, which then evaluate the subtrees independently. The bytecode VM's original implementation did the same thing: OP_SCOPE stored an Expr* pointer in the constant pool and called the tree-walking evaluator at runtime.

The solution is to compile each concurrent body into a standalone Chunk at compile time. The compiler provides two helpers: compile_sub_body() for statement blocks and compile_sub_expr() for expressions. Each creates a fresh Compiler, compiles the code into a new chunk, emits OP_HALT, and stores the resulting chunk in the parent's constant pool as a VAL_CLOSURE constant.

OP_SCOPE uses variable-length encoding: a spawn count, a sync body chunk index, and one chunk index per spawn body. At runtime, the VM:

Exports locals to the global environment using the local_names debug table, so sub-chunks can access parent variables via OP_GET_GLOBAL
Runs the sync body (if present) via a recursive vm_run() call
Spawns threads for each spawn body, each running on a cloned VM
Joins all threads and propagates errors

OP_SELECT similarly encodes per-arm metadata: flags, channel expression chunk index, body chunk index, and binding name index. The VM evaluates channel expressions, polls for readiness, and executes the winning arm.

The key insight is that sub-chunks run as FUNC_SCRIPT without lexical access to the parent's locals. Since they can't use upvalues to reach into the parent frame, the VM exports the parent's live locals into the global environment before running any sub-chunk, using a pushed scope that gets popped after all sub-chunks complete. This is slightly more expensive than true lexical capture, but it keeps the sub-chunks completely self-contained: no AST, no parent frame dependency, fully serializable.

Bytecode Serialization

With AST dependency eliminated, serialization becomes straightforward. The .latc binary format starts with an 8-byte header:

[4C 41 54 43]  magic: "LATC"
[01 00]        format version: 1
[00 00]        reserved

The rest is a recursive chunk encoding: code length + bytecode bytes, line numbers for source mapping, typed constants (with a one-byte type tag for each), and local name debug info. Constants use seven type tags:

Tag	Type	Encoding
0	Int	8-byte signed LE
1	Float	8-byte IEEE 754
2	Bool	1 byte
3	String	length-prefixed (u32 + bytes)
4	Nil	no payload
5	Unit	no payload
6	Closure	param count + variadic flag + recursive sub-chunk

The Closure tag is what makes this recursive: a function constant contains its parameter metadata followed by a complete serialized sub-chunk. Nested functions serialize naturally to arbitrary depth.

The CLI integrates this cleanly:

# Compile to .latc
clat compile input.lat -o output.latc

# Run pre-compiled bytecode (auto-detects .latc suffix)
clat output.latc

# Or compile and run in one step (the default)
clat input.lat

Loading validates magic bytes, checks the format version, and uses a bounds-checking ByteReader that produces descriptive error messages for truncated or malformed inputs.

The Ephemeral Bump Arena

String concatenation is a common source of short-lived allocations. An expression like "hello " + name + "!" creates intermediate strings that are immediately consumed and discarded. In a language with deep-clone-on-read semantics, these temporaries add up.

The ephemeral bump arena is a simple optimization: string concatenation in OP_ADD and OP_CONCAT allocates into a bump arena (vm->ephemeral) instead of the general-purpose heap. These allocations are tagged with REGION_EPHEMERAL, and OP_RESET_EPHEMERAL (emitted by the compiler at every statement boundary) resets the arena in O(1), reclaiming all temporary strings at once.

The tricky part is escape analysis. If a temporary string gets assigned to a global variable, stored in an array, or passed to a compiled closure, it needs to be promoted out of the ephemeral arena before the arena is reset. The VM handles this at specific escape points: OP_DEFINE_GLOBAL, OP_CALL (for compiled closures), array.push, and OP_SET_INDEX_LOCAL. Each of these calls vm_promote_value(), which deep-clones the string to the regular heap if its region is ephemeral.

The arena uses a page-based allocator with 4 KB pages. Resetting doesn't free pages; it just moves the bump pointer back to zero, so subsequent allocations reuse the same memory without any malloc/free overhead. The full design and safety proof are covered in a companion paper.

Closures and the Storage Hack

The upvalue system hasn't changed architecturally since the first VM post; it's still the Lua-inspired open/closed model where ObjUpvalue structs start pointing into the stack and get closed (deep-cloned to the heap) when variables go out of scope. But the encoding grew to accommodate the wider instruction set.

OP_CLOSURE uses variable-length encoding: a constant pool index for the function's compiled chunk, an upvalue count, and then [is_local, index] byte pairs for each captured variable. OP_CLOSURE_16 uses a two-byte big-endian function index for chunks with more than 256 constants.

The storage hack (repurposing closure.body (NULL), closure.native_fn (Chunk pointer), closure.captured_env (ObjUpvalue** cast), and region_id (upvalue count)) remains unchanged. A sentinel value VM_NATIVE_MARKER distinguishes C-native functions from compiled closures:

#define VM_NATIVE_MARKER ((struct Expr **)(uintptr_t)0x1)

A closure with body == NULL and native_fn != NULL is either a C native (if default_values == VM_NATIVE_MARKER) or a compiled bytecode function (otherwise). This avoids adding VM-specific fields to the LatValue union, which matters when values are deep-cloned frequently.

The Self-Hosted Compiler

The file compiler/latc.lat is a bytecode compiler written entirely in Lattice, approximately 2,060 lines that read .lat source, produce bytecode, and write .latc files using the same binary format as the C implementation:

# Use the self-hosted compiler
clat compiler/latc.lat input.lat output.latc

# Run the result
clat output.latc

The architecture mirrors the C compiler: lexing via the built-in tokenize() function, a recursive-descent parser, single-pass code emission, and scope management with upvalue resolution. But Lattice's value semantics required some creative workarounds.

The biggest constraint is that structs and maps are pass-by-value. In C, the compiler uses a Compiler struct with mutable fields: local arrays, scope depth, a chunk pointer. In Lattice, passing a struct to a function creates a copy, so mutations in the callee don't propagate back. The self-hosted compiler works around this with parallel global arrays: code, constants, c_lines, local_names, local_depths, local_captured. Since array mutations via .push() and index assignment are in-place (via resolve_lvalue), global arrays work where structs don't.

Nested function compilation uses explicit save_compiler() / restore_compiler() functions that copy all global arrays to local temporaries and back. It's verbose but correct. The Buffer type (used for serialization output) is also pass-by-value, so a global ser_buf accumulates serialized bytes across function calls.

Other language constraints: no else if (requires else { if ... } or match), mandatory type annotations on function parameters (fn foo(a: any)), and test is a keyword so you can't use it as an identifier.

The self-hosted compiler currently handles expressions, variables, functions with closures, control flow (if/else, while, loop, for, break, continue, match), structs, enums, exceptions, defer, string interpolation, and imports. Not yet implemented: concurrency primitives and advanced phase operations (react, bond, seed). The bootstrapping chain is:

latc.lat → [C VM interprets] → output.latc → [C VM executes]

Full self-hosting (where latc.lat compiles itself) requires adding concurrency support and closing the remaining feature gaps.

The VM Execution Engine

The VM maintains a 4,096-slot value stack, a 256-frame call stack, an exception handler stack (64 entries), a defer stack (256 entries), a global environment, the open upvalue linked list, the ephemeral arena, and a module cache. A pre-allocated fast_args[16] buffer avoids heap allocation for most native function calls.

The OP_CALL instruction discriminates three callee types. Native C functions (marked with VM_NATIVE_MARKER) get the fast path: arguments are popped into fast_args, the C function pointer is invoked, and the return value is pushed. No call frame allocated. Compiled closures get the full treatment: the VM promotes ephemeral values in the current frame (so the callee's OP_RESET_EPHEMERAL doesn't invalidate the caller's temporaries), then pushes a new CallFrame with the instruction pointer at byte 0 of the callee's chunk. Callable structs look up a constructor-named field and dispatch accordingly.

Exception handling uses a handler stack. OP_PUSH_EXCEPTION_HANDLER records the current IP, chunk, call frame index, and stack top. When OP_THROW executes, the nearest handler is popped, the call frame and value stacks are unwound, the error value is pushed, and execution resumes at the handler's saved IP. Deferred blocks interact correctly: OP_DEFER_RUN executes all defer entries registered at or above the current frame before the frame is popped by OP_RETURN.

Iterators avoid closure allocation entirely. OP_ITER_INIT converts a range or array into an internal iterator occupying two stack slots (collection + cursor index). OP_ITER_NEXT advances the cursor, pushes the next element, or jumps to a specified offset when exhausted. The tree-walker used closure-based iterators for for loops; the bytecode version is simpler and avoids the allocation.

Ref<T>: The Escape Hatch from Value Semantics

Everything described so far operates in a world where values are deep-cloned on every read. Maps are pass-by-value. Structs are pass-by-value. Pass a collection to a function and the function gets its own copy; mutations don't propagate back. This is correct and eliminates aliasing bugs, but it creates a real problem: how do you share mutable state when you actually need to?

Ref<T> is the answer. It's a reference-counted shared mutable wrapper, the one type in Lattice that deliberately breaks value semantics:

struct LatRef {
    LatValue value;      // the wrapped inner value
    size_t   refcount;   // reference count
};

When a Ref is cloned (which happens on every variable read, like everything else), the VM bumps the refcount and copies the pointer. It does not deep-clone the inner value. Multiple copies of a Ref share the same underlying LatRef, so mutations through one are visible through all others. This is the explicit opt-in to reference semantics that the rest of the language avoids.

let r = Ref::new([1, 2, 3])
let r2 = r              // shallow copy — same LatRef
r.push(4)
print(r2.get())          // [1, 2, 3, 4] — shared state

The VM provides transparent proxying: OP_INDEX, OP_SET_INDEX, and OP_INVOKE all check for VAL_REF and delegate to the inner value. Indexing into a Ref<Array> indexes the inner array. Calling .push() on a Ref<Array> mutates the inner array directly. At the language level, a Ref mostly behaves like the value it wraps; you just get shared mutation instead of isolated copies.

Ref has its own methods (get()/deref() to clone the inner value out, set(v) to replace it, inner_type() to inspect the wrapped type) plus proxied methods for whatever the inner value supports (map set/get/keys, array push/pop, etc.).

The phase system applies to Refs too. Freezing a Ref blocks all mutation: set(), push(), index assignment all check obj->phase == VTAG_CRYSTAL and error with "cannot set on a frozen Ref." This makes frozen Refs safe to share across concurrent boundaries; they're immutable handles to immutable data.

This introduces a third memory management strategy alongside the dual-heap (mark-and-sweep for fluid values, arenas for crystal values) and the ephemeral bump arena. Refs use reference counting: ref_retain() on clone, ref_release() on free, with the inner value freed when the count hits zero. It's a deliberate trade-off: reference counting is simple and deterministic, and since Refs are the uncommon case (most Lattice code uses value semantics), the lack of cycle collection hasn't been an issue in practice.

Validation

The VM is validated by 815 tests covering every feature: arithmetic, closures, upvalues, phase transitions, exception handling, defer, iterators, data structures, concurrency, modules, bytecode serialization, and the self-hosted compiler.

All 815 tests pass under both normal compilation and AddressSanitizer builds (make asan), which dynamically checks for heap buffer overflows, use-after-free, stack buffer overflows, and memory leaks. For a VM with manual memory management, upvalue lifetime tracking, and an ephemeral arena that reclaims memory at statement boundaries, ASan validation is essential.

Both execution modes (bytecode VM (default) and tree-walker (--tree-walk)) share the same test suite and produce identical results:

make test                # bytecode VM: 815 passed
make test TREE_WALK=1    # tree-walker: 815 passed

Feature parity is complete:

Feature	Tree-walker	Bytecode VM
Phase system (freeze/thaw/clone/forge)	✓	✓
Closures with upvalues	✓	✓
Exception handling (try/catch/throw)	✓	✓
Defer blocks	✓	✓
Pattern matching	✓	✓
Structs with methods	✓	✓
Enums with payloads	✓	✓
Arrays, maps, tuples, sets, buffers	✓	✓
Iterators (for-in, ranges)	✓	✓
Module imports	✓	✓
Concurrency (scope/spawn/select)	✓	✓
Channels	✓	✓
Phase reactions/bonds/seeds	✓	✓
Contracts (require/ensure)	✓	✓
Variable tracking (history)	✓	✓
Bytecode serialization (.latc)	---	✓
Computed goto dispatch	---	✓
Ephemeral bump arena	---	✓
Specialized integer ops	---	✓

The last four rows are VM-only features that have no tree-walker equivalent.

What's Next

The VM is feature-complete but not performance-optimized. The obvious next steps are register allocation to reduce stack traffic, type-specialized dispatch paths guided by runtime profiling, tail call optimization for recursive patterns, and constant pool deduplication across compilation units. Further out, the bytecode provides a natural intermediate representation for JIT compilation.

On the self-hosting front, adding concurrency primitives to latc.lat would close the gap to full self-compilation, where the Lattice compiler compiles itself, producing a .latc file that can then compile other programs without the C implementation in the loop.

The full technical details (including encoding diagrams, the complete opcode listing, compilation walkthroughs, and references to related work in Lua, CPython, YARV, and WebAssembly) are in the research paper.

The source code is at github.com/ajokela/lattice, and the project site is at lattice-lang.org.

git clone https://github.com/ajokela/lattice.git
cd lattice && make
./clat

Rue: Steve Klabnik's AI-Assisted Experiment in Memory Safety Without the Pain

A.C. Jokela — Fri, 20 Feb 2026 12:00:00 GMT

The Pitch

Every few years, a new programming language appears that promises to fix what its predecessors got wrong. Most quietly vanish. A handful become infrastructure. What makes Rue interesting is not that it promises to replace Rust (its creator explicitly says it won't), but that the person making it is one of the most qualified people alive to attempt the experiment.

Steve Klabnik spent thirteen years in the Rust ecosystem. He co-authored The Rust Programming Language, the official book that has introduced more people to Rust than any other resource. He served on the Rust core team, led its documentation team, and worked at Mozilla during Rust's formative years. He later joined Oxide Computer Company, where he contributed to a scratch-built operating system written in Rust. Before all of that, he was a prolific contributor to Ruby on Rails and authored Rust for Rubyists, one of the earliest attempts to bridge the Ruby and Rust communities.

If anyone has earned the right to look at Rust and say "I think we can do this differently," it's Klabnik. And that's exactly what Rue is: a research project asking whether memory safety without garbage collection can be achieved with less cognitive overhead than Rust demands.

But Rue is also something else entirely: an experiment in whether a single person, assisted by an AI, can build a programming language from scratch without funding, without a team, and without a multi-year timeline. The compiler is written in Rust, designed by Klabnik, and implemented primarily by Claude, Anthropic's AI assistant. The project went from nothing to a working compiler in roughly two weeks, producing approximately 100,000 lines of Rust code across 700+ commits.

The Person Behind It

Understanding Rue requires understanding Klabnik's trajectory through programming language communities. He entered professional programming through Ruby, becoming one of the most prolific open-source contributors to the Rails ecosystem in the early 2010s. His final commits to Rails landed in late 2013, around the time he discovered Rust 0.5 during a Christmas visit to his parents' house in rural Pennsylvania.

What followed was a thirteen-year involvement with Rust that few can match. Beyond the official book (now in its second edition, co-authored with Carol Nichols and published by No Starch Press), Klabnik shaped how the Rust community communicates, documents, and teaches. His work on Rust's documentation established patterns that other language communities later adopted. At Mozilla, he helped shepherd Rust through its 1.0 release. At Oxide, he worked on systems software at the lowest levels the language supports.

Klabnik describes himself as having been an AI skeptic until 2025, when he found that large language models had crossed a threshold of genuine usefulness for programming. The shift was dramatic enough that he now writes most of his code with AI assistance. Rue is the product of that conversion, not just a language experiment but a methodology experiment, testing what happens when a deeply experienced language designer directs an AI to implement his vision.

The name follows a deliberate pattern from his career: Ruby, Rust, Rue. He notes three associations: "rue the day" (the negative connotation, a nod to the skepticism any new language faces), the rue plant (paralleling Rust's fungal connotation), and brevity.

What Rue Is Today

Let's be direct about the current state: Rue is a version 0.1.0 research project. The website prominently warns that it is "not ready for real use" and to "expect bugs, missing features, and breaking changes." Klabnik himself has described the language as "still very janky" and cautions against reading too deeply into current implementation details. The GitHub README puts it plainly: "Not everything in here is good, or accurate, or anything: I'm just messing around."

With that caveat firmly in place, here's what exists.

Rue compiles to native machine code targeting x86-64 and ARM64. There is no virtual machine, no interpreter, and no garbage collector. The compiler is written in Rust (95.8% of the repository) and produces binaries directly. It builds using Buck2, Meta's build system, and the project includes a specification, a ten-chapter tutorial, a blog, and a benchmark dashboard that tracks compilation time, memory usage, and binary size across Linux and macOS.

The language itself is statically typed with type inference. Variables are declared with let (immutable by default) or let mut (mutable), following Rust's convention. The type system includes signed and unsigned integers (i8 through i64, u8 through u64), booleans, fixed-size arrays, structs, and enums. There are no strings. There is no standard library. There are no traits, no closures, no iterators, no modules, no error handling beyond panics, and no heap allocation. Generics exist behind a --preview comptime flag (more on that below), but they are not yet part of the stable language.

Read that list of absences again. It is long. And it is honest.

The Type System and Memory Model

Rue's central technical bet is that affine types with mutable value semantics can deliver memory safety more intuitively than Rust's borrow checker and lifetime annotations.

In practice, this means values in Rue are moved by default. When you assign a struct to a new variable or pass it to a function, the original becomes invalid. The compiler enforces single ownership at the type level. This is Rust's move semantics without the escape hatches that references and borrowing provide. If you want a type to be copied instead of moved, you annotate the struct definition with @copy, which enables value duplication.

Where Rust provides shared references (&T) and mutable references (&mut T) governed by the borrow checker's aliasing rules, Rue provides two simpler mechanisms: borrow and inout.

A borrow parameter grants read-only access to a value without copying it. An inout parameter grants temporary mutable access; the function can modify the value, and changes persist after the function returns. Critically, both are marked at the call site, not just in the function signature. When reading Rue code, you can immediately see which arguments a function will modify:

sort(inout values, borrow config);

The tutorial makes the design motivation explicit: "When you read code, you want to understand what it does without tracing through every function." In Go or Python, a function receiving a slice or object might mutate it invisibly. In Rue, mutation is always syntactically visible where the function is called.

This is genuinely appealing. One of Rust's persistent pain points is that understanding what a function does to its arguments requires reading the signature carefully, and even then, interior mutability patterns like RefCell can make the signature misleading. Rue's approach trades expressiveness for legibility.

The trade-off, however, is severe. Without general references, you cannot build self-referential data structures. Linked lists, trees, and graphs (the bread and butter of systems programming data structures) have no obvious implementation path in current Rue. The Hacker News discussion around the language's announcement surfaced this concern immediately: without references that can be stored in data structures, you cannot implement iterators that borrow from containers. This is not a missing feature that will be added later; it is a fundamental consequence of the design choice.

Klabnik has acknowledged this directly: "There is going to inherently be some expressiveness loss. There is no silver bullet." The question Rue poses is whether the expressiveness that remains is sufficient for a useful class of programs. The answer today is: we don't know yet.

What the Tutorial Reveals

The ten-chapter tutorial walks from installation through a capstone quicksort implementation. It covers variables, types, functions, control flow, arrays, structs, enums, and the borrow/inout system. The final chapter implements partitioning and recursive sorting, demonstrating how the language's features compose.

Working through it reveals a language that feels like a simplified Rust with the hard parts surgically removed. Pattern matching on enums is exhaustive, as in Rust. Structs have named fields and move semantics. Arrays are fixed-size with runtime bounds checking that panics on out-of-bounds access. Integers overflow-check by default.

The syntax is deliberately familiar to anyone who has written Rust, Go, or C. Function signatures look like Rust without lifetimes:

fn partition(arr: inout [i32; 5], low: u64, high: u64) -> u64

Control flow uses if/else and while without parentheses around conditions. There is no for loop; iteration requires manual index management with while, which feels like a notable omission even for an early-stage language.

What's conspicuously absent from the tutorial is any program that does something useful beyond computation. There is no file I/O, no string handling, no network access, no memory allocation. The only output mechanism is @dbg(), a built-in debug print. The actual hello.rue in the examples directory is revealing: it is fn main() -> i32 { 42 }. There are no strings, so there is no "Hello, World." The fizzbuzz example is equally telling; it returns integers (1 for Fizz, 2 for Buzz, 3 for FizzBuzz) because it cannot print the words. The specification explicitly excludes coverage of a standard library "when one exists."

The AI Development Story

The most widely discussed aspect of Rue is not its type system but how it was built. Klabnik's blog posts describe a process where he provided architectural direction and design decisions while Claude wrote the vast majority of the implementation code. The project's blog posts are co-credited to both Klabnik and Claude, with some posts authored solely by the AI.

The timeline is striking. The first commit landed on December 15, 2025. By December 22 (one week later), the compiler could handle basic types, structs, control flow, and had accumulated 130 commits. By early January 2026, the project had 777 specification tests across two platforms and the compiler had grown to handle enums, pattern matching, arrays, and the borrow/inout system.

The repository now contains over 700 commits from 4 contributors, with 1,100+ GitHub stars. For a personal hobby project by a well-known developer, this represents meaningful interest, though not the kind of momentum that suggests organic community adoption.

This development model raises legitimate questions. When Claude writes the compiler and Claude writes the blog posts, what does it mean for a human to have "designed" the language? Klabnik's answer is that he makes all architectural and design decisions (what features to include, how the type system works, what trade-offs to accept) while the AI handles implementation. He compares it to an architect and a construction crew: the architect doesn't lay bricks, but the building reflects the architect's vision.

The analogy is imperfect. A construction crew doesn't suggest design changes mid-build based on patterns learned from every other building ever constructed. But the broader point (that directing implementation is itself a form of authorship) is reasonable, and Klabnik's thirteen years of language implementation experience give him credibility that a less experienced designer directing an AI would lack.

What the Source Code Reveals

The documentation and tutorial tell one story. The GitHub repository tells a more nuanced one. Digging into the examples directory and compiler crates reveals features in various stages of development that the official tutorial has not yet caught up with.

Most notably, generics exist as a preview feature. The examples/generics.rue file demonstrates Zig-style comptime type parameters:

fn max(comptime T: type, a: T, b: T) -> T {
    if a > b { a } else { b }
}

let bigger = max(i32, 10, 20);

The comptime T: type syntax tells the compiler to monomorphize, creating specialized versions (max__i32, max__bool, etc.) for each concrete type used at call sites. This is a meaningful step beyond what the tutorial covers, and it follows the Zig model rather than Rust's trait-bounded generics. Whether this approach will scale to real-world code, and whether it can support the kind of type-level abstraction that traits and interfaces provide, remains to be seen. You must compile with rue --preview comptime to access this feature, signaling that it is experimental even by Rue's standards.

The compiler architecture itself is surprisingly mature for a project of this age. The repository contains 18 separate crates: a lexer, parser, intermediate representation (rue-rir), an abstract intermediate representation (rue-air), control flow graph analysis (rue-cfg), code generation, linking, a fuzzer, a spec test runner, and a VS Code extension. This is not a toy single-file compiler; it is a modular pipeline that reflects real compiler engineering, even if the language it compiles remains minimal.

Beyond generics, the gaps are still significant. There are no traits or interfaces, so there is no polymorphism beyond what enums and comptime monomorphization provide. There are no closures or first-class functions. There is no error handling mechanism; functions can panic, but there is no Result type or equivalent. There are no modules or visibility controls. There are no methods on types. There is no string type.

The unchecked keyword provides an escape hatch similar to Rust's unsafe, allowing low-level memory operations while keeping such code "visibly separate from normal safe code." The specification includes chapters on unchecked code syntax and unchecked intrinsics, suggesting that Klabnik recognizes the eventual need for the kind of low-level access that Rust provides through its unsafe system.

The performance page tracks compilation metrics (time, memory, binary size) across x86-64 and ARM64 on both Linux and macOS, but provides no runtime benchmarks comparing Rue's generated code to C, Rust, or Go. This is understandable for a research project, but it means we cannot evaluate whether Rue's native compilation delivers competitive performance.

The Fundamental Question

Every new systems language must answer the same question: what programs can I write in this language that I couldn't write (or couldn't write as well) in an existing one?

For Rust, the answer was clear from early on: memory-safe systems programs without garbage collection pauses. For Go, it was concurrent network services with fast compilation. For Zig, it was a C replacement with better defaults and comptime.

Rue's answer, as it stands today, is: nothing. Not yet. You cannot write a program in Rue that you couldn't write more easily in any mainstream language, because Rue cannot perform I/O, allocate memory, manipulate strings, or interact with the operating system.

This is not a criticism so much as a statement of development stage. The interesting question is what Rue's answer could become if development continues. The borrow/inout model is genuinely simpler than Rust's borrow checker for the cases it handles. Generics are already emerging through the comptime preview. If Rue can stabilize them, grow a heap allocator, a string type, and enough standard library to write real programs (without reintroducing the complexity it was designed to avoid), it could serve programmers who find Rust's learning curve prohibitive but need more safety than C or Go provide.

That is a big "if." History is littered with languages that simplified an existing language's hard parts and then discovered that the hard parts existed for good reasons. Rust's borrow checker is complex because the problems it solves are complex. Removing it and replacing it with a simpler system necessarily means either accepting less expressiveness or finding a genuinely novel solution that the Rust team somehow missed in over a decade of research.

Klabnik is explicit that he doesn't claim to have found such a solution. Rue is a research project exploring a design space, not a product claiming to have mapped it.

Who Should Pay Attention

If you're looking for a language to write software in today, Rue is not it. The project's own documentation says so clearly and repeatedly.

If you're interested in programming language design, Rue is worth following for two reasons. First, the borrow/inout model is a clean articulation of an alternative to Rust's reference system, and watching it encounter real-world requirements will be educational regardless of whether it succeeds. Second, the AI-assisted development methodology is itself a data point in the ongoing question of what role AI can play in building complex software systems.

If you're a Rust programmer who has ever thought "there must be a simpler way to express this," Rue represents one concrete exploration of what "simpler" might look like, and what it costs. The language makes the trade-offs visible in a way that abstract arguments about borrow checker complexity do not.

And if you're Steve Klabnik, messing around with a hobby project after thirteen years of working on someone else's language, Rue looks like exactly the kind of thing a deeply experienced language person should be doing with their evenings and weekends.

Conclusion

Rue is best understood as three things simultaneously: a technical experiment in memory safety ergonomics, a methodological experiment in AI-assisted language development, and a personal project by someone with unusually deep expertise in the problem space. As a technical experiment, it has articulated a clean alternative to Rust's borrow checker that is worth studying even if it proves too restrictive for general use. As a methodological experiment, the speed of development (700+ commits and a working multi-platform compiler in weeks) is genuinely remarkable, though the long-term maintainability of AI-generated compiler code remains unproven. As a personal project, it is honest about its limitations in a way that many language announcements are not.

The language cannot do anything useful yet. It may never be able to. But the questions it asks (can memory safety be simpler? can one person with an AI build a compiler? what happens when you remove the borrow checker and try something else?) are worth asking. And the person asking them has spent over a decade earning the credibility to make the attempt interesting rather than naive.

Watch the GitHub repository. Read the specification. Try the tutorial if you're curious about what a post-Rust systems language might feel like. Just don't write anything important in it yet.

Review of "Crafting Interpreters" by Robert Nystrom

A.C. Jokela — Thu, 19 Feb 2026 16:30:00 GMT

Introduction

There is a particular category of programming book that transcends its subject matter, becoming not just a reference but an experience. "Crafting Interpreters" by Robert Nystrom belongs firmly in this category. Originally published in 2021 after six years of development, the book tackles what many programmers consider one of the most intimidating topics in computer science (building a programming language from scratch) and makes it not just accessible but genuinely enjoyable.

Nystrom is no stranger to technical writing that connects with practitioners. His earlier book "Game Programming Patterns" demonstrated a talent for explaining complex software concepts through clear prose and practical examples. With "Crafting Interpreters," he applies that same skill to language implementation, a domain traditionally guarded by dense academic texts and formal notation that sends most working programmers running.

The book's central premise is both ambitious and elegant: build the same programming language twice. First as a tree-walk interpreter in Java (called jlox), then as a bytecode virtual machine in C (called clox). This dual implementation strategy isn't just a structural gimmick. It serves a deep pedagogical purpose, allowing readers to first grasp the conceptual architecture of language processing in a high-level language before rebuilding everything from raw memory and pointer arithmetic. The result is a book that manages to teach compiler theory, language design, software architecture, and low-level systems programming simultaneously.

The entire text is freely available at craftinginterpreters.com, which speaks to Nystrom's commitment to making this knowledge widely accessible. The physical edition, published with care and featuring hand-drawn illustrations throughout, is worth owning for anyone who works through the material.

The Language: Lox

Before diving into implementation, Nystrom introduces Lox, the language both interpreters will execute. Lox is a dynamically typed, garbage-collected language with C-family syntax that supports first-class functions, closures, and class-based object orientation with single inheritance. It is deliberately modest in scope (no arrays, no module system, no standard library to speak of), but this restraint is precisely the point.

Every feature in Lox exists because it teaches something important about language implementation. Dynamic typing means building a runtime type system. Garbage collection means understanding memory management at the deepest level. Closures require wrestling with variable capture and lifetime semantics. Classes and inheritance demand method resolution and the vtable-like dispatch mechanisms that underpin most object-oriented languages. Lox is small enough to implement in a book but complex enough that implementing it forces the reader to confront every major challenge in language design.

The choice of a custom language rather than a subset of an existing one is significant. It frees Nystrom from having to explain why certain features are omitted or work differently than readers might expect. Lox is exactly what it needs to be, nothing more.

Part II: The Tree-Walk Interpreter (`jlox`)

The first implementation spans thirteen chapters and builds a complete interpreter in Java. Nystrom begins where every language implementation must: with scanning. The scanner chapter walks through converting raw source text into tokens, handling string literals, numbers, keywords, and the inevitable edge cases that make lexical analysis more interesting than it first appears.

From there, the book moves into parsing, where Nystrom introduces recursive descent parsing with a clarity that makes the technique feel almost obvious in hindsight. Rather than reaching for parser generators like YACC or ANTLR, every line of the parser is written by hand. This decision is characteristic of the book's philosophy: no black boxes, no magic, no dependencies. The reader understands every piece because the reader built every piece.

The chapters on expression evaluation and statement execution establish the runtime model, but the book truly hits its stride in the chapters on scope and environments. Nystrom's explanation of lexical scoping (using a chain of environment objects that form what he calls a "cactus stack") is one of the clearest treatments of this topic in any programming text. The hand-drawn illustration of nested environments, with their parent pointers threading back through enclosing scopes, communicates in a single image what paragraphs of formal specification struggle to convey.

Functions and closures represent the first major conceptual challenge, and Nystrom handles them with characteristic patience. The problem of captured variables (where a closure must hold onto variables from an enclosing scope that may have already returned) is presented as a puzzle to be solved rather than a rule to be memorized. The resolver pass that performs static analysis to determine variable binding is introduced as a natural response to a concrete bug, not as an abstract compiler phase.

The object-oriented chapters add classes, methods, constructors, inheritance, and super expressions. By the time jlox is complete, the reader has built a language implementation capable of running recursive algorithms, managing object hierarchies, and handling the scoping rules that trip up even experienced programmers in production languages.

What makes this section exceptional is Nystrom's willingness to show the design process, not just the final design. When a naive approach creates a bug or performance problem, the reader sees it happen and participates in fixing it. This iterative development style mirrors how real software is built and teaches debugging intuition alongside language implementation.

Part III: The Bytecode Virtual Machine (`clox`)

If Part II is the approachable on-ramp, Part III is where the book reveals its true ambition. Across seventeen chapters, Nystrom rebuilds everything in C, this time compiling Lox to bytecode and executing it on a stack-based virtual machine. The motivation is made concrete early: jlox takes 72 seconds to compute the 40th Fibonacci number recursively, while C can do it in half a second. The bytecode VM will close that gap dramatically.

The transition from Java to C is itself educational. Readers who have grown comfortable with Java's automatic memory management, dynamic arrays, and hash maps must now implement all of these from scratch. Nystrom builds a dynamic array type, a hash table, and ultimately a mark-sweep garbage collector, all in service of the language implementation. These data structures are not taught in isolation; they emerge because the VM needs them.

The chunk and instruction design chapters teach the reader to think about data representation at the byte level. Each bytecode instruction is a single byte, followed by operands that encode constants, variable slots, or jump offsets. The disassembler that Nystrom builds alongside the VM is a thoughtful touch, providing a debugging tool that makes the otherwise invisible bytecode tangible.

The single-pass compiler that replaces jlox's separate parsing and resolution phases is a masterclass in practical compiler construction. Nystrom uses Pratt parsing for expressions, a technique he explains with such clarity that this chapter alone has become a widely referenced resource for anyone implementing expression parsers. The Pratt parser's elegant handling of precedence and associativity through a simple table of parsing functions is one of those ideas that, once understood, feels like it should have been obvious all along.

The chapters on closures in clox deserve special mention. Where jlox could lean on Java's garbage collector and object references to capture variables, clox must solve the "upvalue" problem explicitly. Nystrom introduces the concept of upvalues (runtime objects that represent captured variables) and walks through the mechanism by which stack-allocated locals are "closed over" and moved to the heap when their enclosing function returns. The complexity of this implementation, managed through careful incremental development, demonstrates why closures are considered one of the hardest features to implement correctly in a bytecode VM.

The garbage collection chapter is the book's peak of systems programming depth. Nystrom implements a mark-sweep collector, explaining reachability, root sets, and the tricolor abstraction. The treatment is practical rather than theoretical; the reader sees exactly when collection triggers, how objects are traced, and why the collector must handle the subtle case of the VM itself allocating memory during collection (which could invalidate pointers being traced). The self-adjusting heap threshold that balances collection frequency against memory usage is a detail that separates a textbook GC from one that works in practice.

Writing Style and Presentation

Nystrom's prose is the book's secret weapon. Technical writing about compilers tends toward one of two failure modes: impenetrable formalism or hand-waving oversimplification. Nystrom avoids both. His writing is conversational without being sloppy, precise without being dry. Footnotes contain genuine wit. Asides acknowledge the reader's likely confusion at exactly the moments when confusion is most natural.

The hand-drawn illustrations scattered throughout the book serve a purpose beyond aesthetics. They signal that this is a personal, crafted work rather than a mass-produced textbook. The diagrams of memory layouts, parse trees, and stack states during execution are clearer than their machine-generated equivalents in most compiler texts, partly because they include exactly the detail needed and nothing more.

The "Design Note" sections that appear between chapters are mini-essays on language design philosophy: why dynamic typing exists, what makes a feature "elegant," how language designers balance expressiveness against implementation complexity. These sections transform the book from a pure implementation guide into something closer to a meditation on programming language design as a creative discipline.

Strengths

The book's greatest achievement is making compiler construction feel like a natural extension of everyday programming rather than a specialized academic pursuit. By avoiding formal grammars, automata theory, and the mathematical notation that dominates traditional compiler texts, Nystrom demonstrates that you don't need a PhD to build a working language implementation.

The dual-implementation approach pays dividends throughout. Concepts that are murky in one implementation become clear in the other. The tree-walk interpreter makes the abstract concepts tangible; the bytecode VM reveals the performance and engineering considerations that production language implementations face. Together, they provide a stereoscopic view of language implementation that neither could achieve alone.

The no-dependency philosophy deserves praise. There is no lexer generator, no parser generator, no framework, no library. Every line of code in both implementations is written in the book and understood by the reader. This means that upon completion, the reader owns their understanding completely; there is no mysterious tool doing critical work behind the scenes.

The incremental development style produces a book that is remarkably difficult to get lost in. Each chapter begins with working code and ends with working code. The reader is never more than a few pages from being able to compile and run something. For a topic as complex as language implementation, this steady cadence of progress is essential for maintaining motivation.

Limitations

The book is not without its shortcomings. The choice of dynamic typing for Lox means that static type systems (one of the most active and important areas of modern language design) receive no coverage. Type inference, generics, algebraic data types, and pattern matching are absent. A reader completing both implementations still would not know how to add a type checker, which is arguably the most practically relevant compiler phase for working programmers today.

Optimization is largely unexplored. The clox VM is faster than jlox by virtue of being a bytecode interpreter written in C, but Nystrom does not cover constant folding, dead code elimination, register allocation, or any of the optimization passes that distinguish a teaching compiler from a production one. JIT compilation, increasingly the standard for high-performance language runtimes, is mentioned only in passing.

The error handling and recovery throughout both implementations is minimal. Production parsers need sophisticated error recovery to provide useful diagnostics. Nystrom acknowledges this gap but does not address it, leaving readers who want to build user-facing tools with significant work ahead of them.

Lox's deliberate simplicity means that several common language features (arrays, iterators, modules, pattern matching, exception handling) are left as exercises. While this keeps the book focused, it means that readers must figure out on their own how to implement the features that most real languages require. The gap between Lox and a practical language is significant.

Who Should Read This Book

"Crafting Interpreters" is ideal for working programmers who have always been curious about how languages work but have been intimidated by the traditional compiler literature. Comfortable familiarity with Java and C is assumed; this is not a book for learning either language. But the reader need not have any prior knowledge of compilers, formal languages, or automata theory.

Computer science students will find it an excellent companion to a formal compilers course, providing the practical intuition that textbooks like Aho's "Dragon Book" deliberately omit. Conversely, self-taught programmers who never took a compilers course will find this book fills a significant gap in their education.

Language enthusiasts who have tinkered with toy interpreters but never built anything with closures, classes, or garbage collection will find exactly the guidance they need to level up. And anyone who simply enjoys beautifully crafted technical writing will find the book rewarding even as a pure reading experience.

Conclusion

"Crafting Interpreters" is one of the best programming books published in recent years. It takes a subject that most programmers consider forbiddingly complex and renders it not just comprehensible but engaging. Nystrom's combination of clear writing, thoughtful pedagogy, practical focus, and genuine craft produces a book that teaches far more than its nominal subject. Beyond scanning, parsing, and code generation, the reader learns how to approach complex software design, how to build systems incrementally, and how to think about the tools they use every day at a deeper level.

The book will not make you a compiler engineer. It will not teach you how to build a production language runtime, optimize generated code, or implement a sophisticated type system. What it will do is demystify the machinery that powers every programming language you have ever used, and give you the confidence and foundation to explore further. For most programmers, that is more than enough. It is, in fact, exactly what was needed.

From Tree-Walker to Bytecode VM: Compiling Lattice

A.C. Jokela — Tue, 17 Feb 2026 18:00:00 GMT

Lattice is a programming language built around a crystallization-based phase system: values start as mutable "flux" and can be frozen into immutable "fix," with the runtime enforcing the transition and providing reactions, bonds, contracts, and temporal tracking around it. It's implemented in C with no external dependencies.

When I started building Lattice, a tree-walking interpreter was the obvious first move. You parse source into an AST, walk the nodes recursively, and evaluate as you go. It's straightforward, easy to debug, and lets you iterate on language semantics quickly without worrying about a second representation. Crafting Interpreters calls this approach "the simplest way to build an interpreter," and it's right.

But tree-walkers have well-known limitations. Every expression evaluation descends through function calls: eval_expr calling eval_binary calling eval_expr twice more. The overhead compounds. You're chasing pointers through heap-allocated AST nodes with poor cache locality. And the call stack of the host language (C, in Lattice's case) becomes tangled with the call stack of the guest language, making it harder to implement features like error recovery and coroutines cleanly.

Lattice v0.3.0 shipped a bytecode compiler and stack-based virtual machine alongside the tree-walker. In v0.3.1, the bytecode VM became the default for file execution, the interactive REPL, and the browser-based playground. The tree-walker is still available via --tree-walk, but the VM now handles everything. This post walks through the architecture of that VM, some design decisions that turned out to matter, and a mutation bug that only surfaces when you combine deep-clone-on-read semantics with in-place method dispatch.

Architecture Overview

The bytecode pipeline has three stages: lexing and parsing (shared with the tree-walker), compilation from AST to bytecode chunks, and execution on a stack-based VM. The compiler and VM together add about 8,200 lines of C to the codebase, bringing the total to around 33,000 lines.

A Chunk is the compilation unit: a dynamic array of bytecode instructions, a constant pool, and debug metadata mapping instructions back to source line numbers. The compiler walks the AST and emits bytes into a chunk. The VM reads bytes from the chunk and executes them against a value stack.

typedef struct {
    uint8_t  *code;          // bytecode array
    size_t    len, cap;
    LatValue *constants;     // constant pool
    size_t    const_len, const_cap;
    int      *lines;         // source line per instruction
    char    **local_names;   // slot → variable name (debug)
    size_t    local_name_cap;
} Chunk;

The VM itself is a for(;;) loop with a switch on the current opcode byte, the textbook approach. No computed gotos, no threaded dispatch, no JIT. Just a switch. On modern hardware with branch prediction, a well-organized switch over 62 opcodes is fast enough that the overhead is negligible compared to the cost of actual operations (string allocation, hash table lookups, deep cloning).

for (;;) {
    uint8_t op = READ_BYTE();
    switch (op) {
        case OP_CONSTANT: { ... }
        case OP_ADD:      { ... }
        case OP_CALL:     { ... }
        // 59 more cases
    }
}

The value stack holds 4,096 slots. The call frame stack holds 256 frames. Each CallFrame tracks its own instruction pointer, a base pointer into the value stack for its local variables, and an array of captured upvalues for closures. When you call a function, the VM pushes a new frame pointing at the callee's chunk. When the function returns, the frame pops and execution resumes in the caller.

The Instruction Set

Lattice's instruction set has 62 opcodes. Some are standard (OP_ADD, OP_JUMP_IF_FALSE, OP_RETURN). Others exist because of Lattice-specific semantics.

The phase system needs dedicated opcodes. OP_FREEZE pops a value, deep-clones it into a crystal region with VTAG_CRYSTAL tags, and pushes the frozen result. OP_THAW does the reverse. OP_MARK_FLUID sets the phase tag to VTAG_FLUID; this is what flux bindings emit after their initializer. OP_FREEZE_VAR and OP_THAW_VAR handle the case where freeze(x) targets a named variable and needs to write back the result, carrying extra operands to identify the variable's location (local slot, upvalue, or global name).

Phase reactions and bonds each have their own opcodes: OP_REACT, OP_UNREACT, OP_BOND, OP_UNBOND, OP_SEED, OP_UNSEED. These could theoretically be implemented as native function calls, but making them opcodes lets the compiler emit the variable name as a constant operand, and the VM needs the name to look up the correct reaction/bond registration in its tracking tables, and encoding it in the bytecode avoids a runtime string lookup.

Structured concurrency uses an interesting hybrid. OP_SCOPE and OP_SELECT each carry a constant-pool index that stores a pointer to the original AST Expr* node. When the VM hits one of these opcodes, it invokes the tree-walking evaluator on that subtree. This is a deliberate design choice; the concurrency primitives involve spawning threads and managing channels, which requires the evaluator's full environment machinery. Rather than reimplement all of that in the VM, the bytecode compiler punts to the tree-walker for these specific constructs. The rest of the program runs on the VM; only scope and select blocks briefly drop into interpretation.

Closures and Upvalues

Closures are where bytecode VMs get interesting, and Lattice follows the upvalue model that Lua pioneered and Crafting Interpreters popularized.

When a function is defined inside another function and references variables from the enclosing scope, those variables need to outlive their original stack frame. The solution is upvalues, indirection objects that start pointing into the stack and get "closed over" when the variable goes out of scope.

typedef struct ObjUpvalue {
    LatValue *location;        // points to stack slot or &closed
    LatValue  closed;          // holds value after scope exit
    struct ObjUpvalue *next;   // linked list for open upvalues
} ObjUpvalue;

While the enclosing function is still executing, location points directly at the stack slot. When the enclosing function returns, OP_CLOSE_UPVALUE copies the stack value into the closed field and repoints location to &closed. The closure doesn't know or care about the switch; it always dereferences location. This is why upvalues work: they're a level of indirection that transparently survives stack frame destruction.

The compiler resolves variable references in three stages: first it checks local scope (resolve_local), then upvalues (resolve_upvalue, which walks the compiler chain recursively), then falls back to globals via OP_GET_GLOBAL. The OP_CLOSURE instruction is followed by a series of (is_local, index) byte pairs, one per upvalue, telling the VM whether to capture from the current frame's stack or from the parent frame's upvalue array.

A concrete example makes this clearer. Consider a counter factory:

fn make_counter() {
    flux count = 0
    return |n| { count += n; count }
}

let c = make_counter()
print(c(5))   // 5
print(c(3))   // 8

When make_counter returns, its stack frame is destroyed, but count needs to survive, because the returned closure references it. During compilation, the compiler sees that the closure's body references count, which is local to the enclosing make_counter. It emits an (is_local=true, index=1) upvalue descriptor. At runtime, OP_CLOSURE calls capture_upvalue(), which either reuses an existing ObjUpvalue pointing at that stack slot or creates a new one. When make_counter returns, OP_CLOSE_UPVALUE copies the stack value of count into the upvalue's closed field and repoints location. The closure keeps working, oblivious to the frame being gone.

One implementation detail worth noting: Lattice stores the upvalue array by repurposing the closure's captured_env field (normally an Env* in the tree-walker) and the upvalue count in the region_id field. This avoids adding new fields to the LatValue union, which matters when values are deep-cloned frequently, since every field adds to the clone cost.

Compiling for the REPL

A REPL that runs on a bytecode VM needs different compilation from file execution. The difference is small but important.

In file mode, compile_module() compiles a complete program and terminates with OP_UNIT; OP_RETURN; the module returns unit, and any expression results along the way are discarded with OP_POP. This is the right behavior for scripts: you don't want every intermediate expression to accumulate on the stack.

In REPL mode, compile_repl() needs the opposite behavior for the last expression. When you type 42 at the REPL prompt, you want to see => 42. So if the last item in the compiled chunk is a bare expression statement, compile_repl() compiles the expression but skips the OP_POP, leaving the value on the stack. Then it emits OP_RETURN, and the VM receives the value as the chunk's return value.

bool last_is_expr = (prog->item_count > 0 &&
    prog->items[prog->item_count - 1].tag == ITEM_STMT &&
    prog->items[prog->item_count - 1].as.stmt->tag == STMT_EXPR);

if (last_is_expr)
    emit_byte(OP_RETURN, 0);       // value already on stack
else {
    emit_byte(OP_UNIT, 0);         // no expression — return unit
    emit_byte(OP_RETURN, 0);
}

For function definitions, struct declarations, and enum definitions, the result is unit, and the REPL silently suppresses the => output. This matches user expectations: defining a function shouldn't print anything. The effect in practice:

lattice> 42
=> 42
lattice> "hello" + " world"
=> "hello world"
lattice> flux x = 10
lattice> x * 2
=> 20
lattice> fn square(n: Int) -> Int { n * n }
lattice> square(7)
=> 49

Each line is independently compiled and executed on the persistent VM. Globals defined in one line (flux x = 10) are visible in subsequent lines because they're stored in the VM's environment, which persists across iterations. The Chunk for each line is freed after execution; constants that matter (like global variable values) have already been deep-cloned into the environment.

The other critical difference is enum persistence. compile_module() frees its known-enum registry after compilation, because the compiler is done. compile_repl() must not, because enums defined in REPL iteration N need to be visible in iteration N+1. The REPL calls compiler_free_known_enums() only on exit. The same lifetime concern applies to parsed programs; struct and function declarations store Expr* pointers that compiled chunks reference at runtime. The REPL accumulates all parsed programs in a dynamic array and frees them only when the session ends.

The Global Mutation Bug

This is the story I find most instructive, because it reveals a subtle interaction between two independently reasonable design decisions.

Lattice has deep-clone-on-read semantics. When you access a variable, the environment doesn't hand you a reference to the stored value; it hands you a fresh deep clone. This eliminates aliasing entirely: two variables never share underlying memory, passing a map to a function gives the function its own copy, and there's no way to create spooky action at a distance through shared mutable state.

bool env_get(const Env *env, const char *name, LatValue *out) {
    for (size_t i = env->count; i > 0; i--) {
        LatValue *v = lat_map_get(&env->scopes[i-1], name);
        if (v) {
            *out = value_deep_clone(v);  // always a fresh copy
            return true;
        }
    }
    return false;
}

This is expensive but correct. It gives Lattice pure value semantics without needing a borrow checker or persistent data structures.

The tree-walking evaluator handles in-place mutation (like array.push()) with a separate resolve_lvalue() mechanism that obtains a direct mutable pointer into the environment's storage, bypassing the deep clone. Push, pop, index assignment: these all go through resolve_lvalue and mutate the stored value directly.

The bytecode VM needed the same distinction. For local variables, this is straightforward: locals live on the value stack, and the VM has a direct pointer to them via frame->slots[slot]. I added OP_INVOKE_LOCAL, which takes a stack slot index as an operand and passes a pointer to vm_invoke_builtin():

case OP_INVOKE_LOCAL: {
    uint8_t slot = READ_BYTE();
    uint8_t method_idx = READ_BYTE();
    uint8_t arg_count = READ_BYTE();
    const char *method_name = frame->chunk->constants[method_idx].as.str_val;
    LatValue *obj = &frame->slots[slot];  // direct pointer

    if (vm_invoke_builtin(vm, obj, method_name, arg_count, local_var_name)) {
        // builtin mutated obj in-place — mutation persists
        break;
    }
    // ... fall through to closure/method dispatch
}

When .push() grows the array by reallocating obj->as.array.elems and incrementing obj->as.array.len, it's directly modifying the stack slot. The mutation persists because obj is the variable.

For globals, the situation is different. Globals live in the environment (a scope-chain of hash maps), and env_get() deep-clones. The generic OP_INVOKE opcode works by evaluating the receiver expression onto the stack (which, for a global variable, means emitting OP_GET_GLOBAL, which calls env_get(), which deep-clones) and then dispatching the method on the cloned value. After the builtin mutates the clone, OP_INVOKE pops and frees it. The mutation vanishes.

flux nums = [1, 2, 3]
nums.push(4)
print(nums)  // still [1, 2, 3] — the push mutated a clone

This is the kind of bug that's obvious in retrospect but invisible when you're implementing things one piece at a time. env_get() deep-cloning is correct. OP_INVOKE popping the receiver after dispatch is correct. Each piece behaves correctly in isolation. The bug emerges from their composition.

The fix is OP_INVOKE_GLOBAL, a new opcode that knows the receiver is a global variable and writes back after mutation:

case OP_INVOKE_GLOBAL: {
    uint8_t name_idx = READ_BYTE();
    uint8_t method_idx = READ_BYTE();
    uint8_t arg_count = READ_BYTE();
    const char *global_name = frame->chunk->constants[name_idx].as.str_val;
    const char *method_name = frame->chunk->constants[method_idx].as.str_val;

    LatValue obj_val;
    if (!env_get(vm->env, global_name, &obj_val)) {
        VM_ERROR("undefined variable '%s'", global_name); break;
    }

    if (vm_invoke_builtin(vm, &obj_val, method_name, arg_count, global_name)) {
        if (vm->error) { /* handle error */ }
        // Write back the mutated clone to the environment
        env_set(vm->env, global_name, obj_val);
        break;
    }
    // ... fall through for non-builtin methods
}

The compiler emits OP_INVOKE_GLOBAL when it sees a method call on an identifier that isn't a local variable or an upvalue:

case EXPR_METHOD_CALL: {
    if (e->as.method_call.object->tag == EXPR_IDENT) {
        int slot = resolve_local(current, e->as.method_call.object->as.str_val);
        if (slot >= 0) {
            // ... emit OP_INVOKE_LOCAL
        }
        int upvalue = resolve_upvalue(current, e->as.method_call.object->as.str_val);
        if (upvalue < 0) {
            // Not local, not upvalue — must be global
            // ... emit OP_INVOKE_GLOBAL
        }
    }
    // ... fall through to generic OP_INVOKE
}

This gives us three tiers of method dispatch: OP_INVOKE_LOCAL for locals (direct pointer, no clone), OP_INVOKE_GLOBAL for globals (clone + write-back), and OP_INVOKE for everything else (computed receivers like get_array().push(x), where there's nothing to write back to). With the fix:

flux nums = [1, 2, 3]
nums.push(4)
nums.push(5)
print(nums)  // [1, 2, 3, 4, 5] — mutations persist

All mutating builtins (push, pop, set, remove, insert, remove_at) now work correctly on global variables. The same pattern applies to maps, sets, and any other type with in-place methods.

The broader lesson is that deep-clone-on-read semantics create an impedance mismatch with in-place mutation. In a reference-based language, obj.push(x) just works; obj is a reference, and the mutation happens wherever the reference points. In a value-based language, you need to explicitly handle the write-back for every level of variable storage. The tree-walker's resolve_lvalue is one solution. The VM's tiered invoke opcodes are another. Both exist because of the same underlying tension.

The WASM Playground

Lattice's browser-based playground compiles the entire VM to WebAssembly via Emscripten. The WASM API exposes four functions: lat_init(), lat_run_line(), lat_is_complete(), and lat_destroy().

The playground runs the same bytecode VM as the native binary. Each line of input goes through the same pipeline: lex, parse, compile_repl(), vm_run(). The lat_is_complete() function checks bracket depth to determine whether the user is mid-expression, enabling multi-line input by waiting for balanced braces before compiling.

Previously the playground used the tree-walking evaluator, which meant code could behave differently in the browser than on the command line. Switching the WASM build to the bytecode VM eliminates that inconsistency; the playground, the REPL, and file execution all use the same compilation and execution path.

What Didn't Change

It's worth noting what the bytecode VM doesn't change about Lattice.

The value representation is identical. A LatValue is still a tagged union with a type tag, phase tag, and payload. Phase transitions still deep-clone data across heap regions. The dual-heap architecture (mark-and-sweep for fluid data, arena-based regions for crystal data) is unchanged. Global variables still live in a scope-chain environment.

The parser and AST are completely shared. The compiler reads the same Program structure that the tree-walker reads. A single set of test programs validates both execution paths, and all 771 tests pass on both.

The phase system compiles one-to-one. freeze() becomes OP_FREEZE. thaw() becomes OP_THAW. Bonds, reactions, seeds, pressure constraints: each has a corresponding opcode that does exactly what the tree-walker's evaluator function did, just driven by bytecode dispatch instead of recursive AST traversal.

Performance

I haven't done rigorous benchmarking, and I'm deliberately not making performance claims. The motivation for the bytecode VM wasn't speed; it was consistency (one execution path everywhere) and architectural cleanliness (the VM is easier to extend than the tree-walker's deeply nested switch statements).

That said, bytecode VMs are generally faster than tree-walkers for the structural reasons mentioned earlier: better cache locality (sequential byte array vs. pointer-chasing through AST nodes), less call overhead (one switch dispatch vs. recursive function calls), and a compact representation that fits more of the program in cache. Whether this matters for Lattice programs depends on the workload. For a language whose core runtime cost is dominated by deep cloning, the dispatch overhead is rarely the bottleneck.

Looking Forward

The VM is feature-complete but not optimized. There's no constant folding, no dead code elimination, no register allocation (it's a pure stack machine). The OP_SCOPE and OP_SELECT concurrency opcodes still delegate to the tree-walker. The dispatch loop is a plain switch rather than computed gotos.

These are all well-understood optimizations with clear implementation paths. The point of v0.3.1 is that the bytecode VM is now the default, passes all tests, and handles the full language surface including the phase system. Optimization is a separate project.

The source code is at github.com/ajokela/lattice, and you can try it in the browser at lattice-lang.web.app. The bytecode VM, compiler, REPL, and all 62 opcodes are in four files: compiler.c, vm.c, chunk.c, and opcode.c.

git clone https://github.com/ajokela/lattice.git
cd lattice && make
./clat

Recommended Resources

Crafting Interpreters by Robert Nystrom - The definitive guide to building interpreters and bytecode VMs, and a major influence on Lattice's upvalue implementation
Writing A Compiler In Go by Thorsten Ball - Practical companion covering bytecode compilation and stack-based VMs
Engineering a Compiler by Cooper & Torczon - Comprehensive treatment of compiler internals from front-end to optimization
Compilers: Principles, Techniques, and Tools by Aho, Lam, Sethi, Ullman - The classic Dragon Book covering parsing, code generation, and optimization theory

Mutability as a First-Class Concept: The Lattice Phase System

A.C. Jokela — Sat, 14 Feb 2026 23:00:00 GMT

Mutability as a First-Class Concept: The Lattice Phase System

Most programming languages treat mutability as a binary annotation. You write const or let, final or var, and the compiler enforces it statically. Rust goes further with its borrow checker, enforcing exclusive mutable access at compile time. JavaScript offers Object.freeze(), a runtime operation that's shallow by default and provides no mechanism for observation or validation. These are all useful tools, but they share a common limitation: mutability is something you declare, not something you work with.

In Lattice, I've been building something different. Mutability (what Lattice calls phase) is a first-class runtime property that can be queried, constrained, validated, coordinated across variables, observed reactively, and even tracked historically. Over the last several releases (v0.2.3 through v0.2.6), this system has grown from simple freeze/thaw semantics into a full lifecycle framework. This post walks through that progression and the design decisions behind it.

The Metaphor: Crystallization

Lattice is built around the metaphor of crystallization. Values begin in a fluid state (mutable) and can be frozen into a crystal state (immutable). The thaw() operation creates a mutable copy of a crystal value, and clone() performs a deep copy regardless of phase. This vocabulary isn't just cosmetic; it shapes how you think about data lifecycle in a Lattice program.

flux temperature = 72.5       // fluid: mutable
temperature = 68.0             // allowed

freeze(temperature)            // now crystal: immutable
// temperature = 70.0          // ERROR: cannot mutate crystal value

flux copy = thaw(temperature)  // new fluid copy
copy = 70.0                    // allowed

The flux keyword declares a fluid (mutable) binding. The fix keyword declares a crystal (immutable) binding. And let infers phase from context: fluid if the value is fluid, crystal if crystal. This alone isn't novel. What makes Lattice's approach interesting is everything that builds on top of it.

Phase Constraints: Mutability in Your Type Signatures

The first major addition (v0.2.3) was phase constraints on function parameters. In most languages, a function that receives data has no way to express whether it expects mutable or immutable input. You might document it, or rely on convention, but the language doesn't help. In Lattice, you can annotate parameters with their expected phase:

fn mutate(data: flux Map) {
    data.set("modified", true)
}

fn inspect(data: fix Map) {
    print(data.get("name"))
}

The runtime checks phase at call time. Pass a crystal value to mutate() and you get an error. Pass a fluid value to inspect() and it works fine. Fluid is compatible with fix because it can be read. The constraint is about what the function needs, not what the caller has.

The shorthand syntax uses ~ for flux and * for fix:

fn process(data: ~Map) { ... }  // needs mutable
fn display(data: *Map) { ... }  // needs immutable

Phase-Dependent Dispatch

Phase constraints enable something more powerful: dispatch based on runtime phase. You can define multiple implementations of the same function with different phase signatures, and the runtime selects the best match:

fn serialize(data: ~Map) {
    // mutable path: can add metadata before serializing
    data.set("serialized_at", time_now())
    print(json_stringify(data))
}

fn serialize(data: *Map) {
    // immutable path: serialize directly, no side effects
    print(json_stringify(data))
}

flux config = Map::new()
config["host"] = "localhost"
serialize(config)      // calls flux overload, adds timestamp

freeze(config)
serialize(config)      // calls fix overload, no mutation

The overload resolution uses a scoring system. An exact phase match (fluid argument to flux parameter) scores highest. A compatible match (fluid to unphased) scores lower. An incompatible match (crystal to flux) is rejected entirely. When multiple overloads exist, the best-scoring one wins.

This is genuinely useful in practice. A caching layer might have one implementation that updates a cache (requires mutable data) and another that reads through (works with immutable data). A serialization function might add metadata to mutable structures but serialize immutable ones directly. The caller doesn't need to know; the runtime dispatches based on what the data actually is.

Crystallization Contracts: Validation at the Phase Boundary

The next question was: when data freezes, how do you ensure it's in a valid state? In real systems, immutable data often represents finalized configuration, committed transactions, or published records. You want to validate before that transition happens.

Version 0.2.5 introduced crystallization contracts, validation closures attached to freeze() with the where keyword:

flux config = Map::new()
config["host"] = "localhost"
config["port"] = 8080
config["workers"] = 4

freeze(config) where |v| {
    if !v.has("host") { throw("config missing 'host'") }
    if !v.has("port") { throw("config missing 'port'") }
    if v["workers"] < 1 { throw("need at least 1 worker") }
}

The contract receives a deep clone of the value (so the validation can't accidentally mutate the original), runs the closure, and if the closure throws, the freeze is aborted and the value remains fluid. If validation passes, the value transitions to crystal.

This maps cleanly to real-world patterns. Database ORMs validate before persisting. Configuration systems validate before applying. Form submissions validate before accepting. The difference is that in Lattice, this validation is attached to the phase transition itself, not to a separate method you have to remember to call.

Contracts compose naturally with the rest of the phase system. You can use them with phase bonds (discussed next) or with phase-dependent dispatch. A function that accepts fix Map knows its argument passed whatever contract was attached at freeze time.

Phase Bonds: Coordinated Freezing

Individual freeze/thaw operations work well for isolated values, but real programs have related data that should transition together. A web request's headers, body, and metadata should probably all be immutable before you send it. A transaction's debit and credit entries should freeze atomically.

Phase bonds (also v0.2.5) let you declare these relationships:

flux header = Map::new()
flux body = Map::new()
flux footer = Map::new()

header["content-type"] = "text/html"
body["content"] = "<h1>Hello</h1>"
footer["timestamp"] = time_now()

bond(header, body, footer)

freeze(header)              // cascades to body AND footer
print(phase_of(body))       // "crystal"
print(phase_of(footer))     // "crystal"

The bond(target, ...deps) call links dependencies to a target. When the target freezes, all its dependencies freeze too. Bonds are also transitive: if A is bonded to B and B is bonded to C, freezing A cascades through B to C.

flux a = 1
flux b = 2
flux c = 3

bond(a, b)    // b depends on a
bond(b, c)    // c depends on b

freeze(a)     // freezes a → b → c
print(phase_of(c))  // "crystal"

You can remove bonds with unbond() before the freeze happens:

bond(header, body, footer)
unbond(header, footer)    // footer no longer cascades

freeze(header)            // freezes header and body, NOT footer
print(phase_of(footer))   // "fluid"

Bonds solve a coordination problem that most languages leave to discipline. In a typical codebase, you'd need to remember to freeze all related values, or wrap them in a container and freeze that. Bonds make the relationship explicit and enforced.

Phase Reactions: Observing State Transitions

With constraints, contracts, and bonds, you can control how and when phase transitions happen. But sometimes you also need to know that they happened. Logging, cache invalidation, UI updates, audit trails: these are all responses to state changes.

Version 0.2.6 adds phase reactions: callbacks that fire automatically when a variable's phase changes.

flux data = [1, 2, 3]

react(data, |phase, val| {
    print("data is now " + phase + ": " + to_string(val))
})

freeze(data)   // prints: "data is now crystal: [1, 2, 3]"
thaw(data)     // prints: "data is now fluid: [1, 2, 3]"

The callback receives two arguments: the new phase name (as a string, "crystal" or "fluid") and a deep clone of the current value. Multiple callbacks can be registered on the same variable, and they fire in registration order:

flux counter = 0

react(counter, |phase, val| {
    print("logger: counter is now " + phase)
})

react(counter, |phase, val| {
    if phase == "crystal" {
        print("audit: counter finalized at " + to_string(val))
    }
})

counter = 42
freeze(counter)
// prints:
//   logger: counter is now crystal
//   audit: counter finalized at 42

Reactions also fire during bond cascades. If variable B is bonded to A and has a reaction registered, freezing A will cascade to B and trigger B's reaction:

flux primary = Map::new()
flux replica = Map::new()

bond(primary, replica)

react(replica, |phase, val| {
    print("replica transitioned to " + phase)
})

freeze(primary)
// prints: "replica transitioned to crystal"

This is a powerful combination. Bonds handle the coordination of transitions, and reactions handle the observation. Together they let you build systems where phase changes propagate and trigger side effects in a predictable, declarative way.

Use unreact() to remove all reactions from a variable:

react(data, |phase, val| { print("fired") })
unreact(data)
freeze(data)  // no output — reaction was removed

If a reaction callback throws an error, it propagates as a reaction error, giving you a clean way to handle failures in the observation chain.

Temporal Values: Phase History and Time Travel

The last piece of the phase system (also v0.2.5) is temporal values: the ability to track a variable's phase transitions and value changes over time.

flux counter = 0
track("counter")

counter = 10
counter = 20
freeze(counter)

let history = phases("counter")
// [{phase: "fluid", value: 0},
//  {phase: "fluid", value: 10},
//  {phase: "fluid", value: 20},
//  {phase: "crystal", value: 20}]

The track() function enables recording for a named variable. Every assignment and phase transition creates a snapshot. The phases() function returns the full history as an array of maps, and rewind() lets you retrieve past values by offset:

flux x = 100
track("x")
x = 200
x = 300

print(rewind("x", 0))  // 300 (current)
print(rewind("x", 1))  // 200 (one step back)
print(rewind("x", 2))  // 100 (two steps back)

Temporal values serve primarily as a debugging and auditing tool. When something goes wrong with a frozen value, you can inspect its history to see what mutations happened before the freeze. When testing phase-dependent dispatch, you can verify that the right transitions occurred. In production systems, you can use temporal tracking for audit logs or undo functionality.

The Bigger Picture: Why This Matters

Most programming languages treat mutability as a compiler concern, something to check at build time and forget about. Lattice treats it as a runtime property with the same richness as types or values. This opens up patterns that are difficult or impossible in other languages:

Gradual freezing. Data starts fluid, accumulates state through a pipeline, and freezes when it's complete. Contracts validate at the boundary. Bonds ensure related data transitions together. This maps naturally to request processing, form building, transaction assembly, and configuration loading.

Observable state transitions. Reactions let you attach behavior to phase changes without coupling the code that freezes with the code that responds. A module can register a reaction on shared data without knowing who or when the freeze will happen.

Phase-aware APIs. Functions can express their mutability requirements in their signatures and dispatch based on the caller's data. Libraries can offer mutable and immutable code paths transparently.

Auditability. Temporal tracking provides a built-in mechanism for understanding how data evolved, without external logging infrastructure.

None of these features require abandoning the simple mental model. At its core, Lattice still has fluid and crystal, mutable and immutable. Everything else is opt-in machinery for programs that need more control.

Comparison with Other Approaches

It's worth comparing this to how other languages handle mutability:

Feature	Rust	JavaScript	Lattice
Mutability declaration	`let` / `let mut`	`const` / `let`	`fix` / `flux` / `let`
Enforcement	Compile-time	Runtime (shallow)	Runtime (deep)
Phase transitions	N/A	`Object.freeze()`	`freeze()` / `thaw()` / `clone()`
Validation on freeze	N/A	N/A	Crystallization contracts
Coordinated freezing	N/A	N/A	Phase bonds
Transition observation	N/A	N/A	Phase reactions
Phase-dependent dispatch	N/A	N/A	Overload resolution by phase
History tracking	N/A	N/A	Temporal values

Rust's borrow checker is more powerful for preventing data races at compile time; Lattice doesn't attempt that. JavaScript's Object.freeze() is more pragmatic but also more limited: it's shallow, provides no observation, and offers no coordination. Lattice occupies a different point in the design space: mutability as a domain concept rather than a compiler constraint.

Implementation Notes

The phase system is implemented in C as part of Lattice's tree-walking interpreter. Phase tags are stored directly on values (VTAG_FLUID, VTAG_CRYSTAL, VTAG_UNPHASED), so phase checks are single comparisons. Bonds are stored as a dynamic array of BondEntry structs on the evaluator, each mapping a target variable name to its dependencies. Reactions use a similar structure; ReactionEntry maps a variable name to an array of callback closures. Temporal tracking stores HistorySnapshot arrays containing phase names and deep-cloned values.

The deep cloning is important throughout. Contract validation receives a clone so it can't mutate the original. Reaction callbacks receive clones so observers can't interfere with each other. Temporal snapshots are clones so history is independent of current state. This means the phase system has allocation costs proportional to value size, but it also means the invariants are strong, with no spooky action at a distance.

Freeze cascading through bonds is recursive, and reactions fire during cascading, so a single freeze() call can trigger an arbitrary chain of transitions and callbacks. Error propagation is straightforward: if any reaction throws, the error surfaces immediately with context about which reaction failed.

What's Next

The phase system is reaching a natural plateau in terms of core features. There are a few directions I'm considering:

Partial freezing already exists in a basic form: you can freeze individual struct fields or map keys while leaving the container mutable. Expanding this to support more granular control (freeze all fields matching a pattern, freeze a subtree) could be useful for large data structures.

Phase-aware pattern matching lets you match on phase in match expressions using ~ and * qualifiers. This is already implemented but could be extended with more complex phase patterns.

Compile-time phase inference is a longer-term goal. If the interpreter can prove that a value is always crystal by a certain point, it could skip runtime checks. This would bring some of Rust's static guarantees to Lattice without requiring explicit lifetime annotations.

For now, the phase system provides a cohesive set of tools for working with mutability as a first-class concept. Whether you're building a configuration loader that validates before committing, a pipeline that coordinates related state transitions, or a reactive system that responds to phase changes, Lattice gives you the vocabulary and the enforcement to do it declaratively.

Lattice is open source and available at lattice-lang.org. The language compiles and runs on macOS and Linux with no dependencies beyond a C11 compiler. You can try it in your browser via the playground, or clone the repo and run make && ./clat to start the REPL.

Introducing Lattice: A Crystallization-Based Programming Language

A.C. Jokela — Tue, 10 Feb 2026 18:00:00 GMT

Most programming languages treat mutability as a binary property. A variable is either mutable or it's not. You declare it one way, and that's the end of the story. Rust adds nuance with its ownership and borrowing model, and functional languages sidestep the question by making everything immutable by default, but the fundamental framing remains the same: mutability is a static attribute decided at declaration time.

Lattice takes a different approach. In Lattice, mutability is a phase, a state that a value passes through over its lifetime, like matter transitioning between liquid and solid. A value starts as mutable flux, and when you're done shaping it, you freeze it into immutable fix. Need to modify it again? Thaw it back to flux. Want to build something complex and immutable in one shot? Use a forge block, a controlled mutation zone whose output automatically crystallizes.

This isn't just a metaphor. The phase system is woven through Lattice's entire runtime, from its type representation to its memory management architecture. This post is a deep dive into what that means, how it works at the implementation level, and why it represents a genuinely different way of thinking about the relationship between mutability and memory.

The Problem Lattice Solves

Every language designer eventually confronts the same tension: programmers need mutability to build things, but mutability is the source of most bugs. Shared mutable state causes race conditions. Unexpected mutation causes aliasing bugs. Mutable references that outlive their owners cause use-after-free errors.

Different languages resolve this tension in different ways, and each approach carries trade-offs.

Garbage-collected languages (Java, Python, Go, JavaScript) let you mutate freely and use a garbage collector to clean up. This is convenient but pushes the cost to runtime: GC pauses, unpredictable memory usage, and no compile-time guarantees about who can modify what. You gain ease of use but lose control.

Rust's ownership model provides compile-time guarantees through a sophisticated borrow checker. You can have either one mutable reference or many immutable references, but not both. This eliminates data races at compile time, but the cost is complexity: the borrow checker is notoriously difficult for newcomers, lifetime annotations add syntactic weight, and certain patterns (like self-referential structs or graph structures) require unsafe escape hatches.

Functional languages (Haskell, Erlang, Clojure) default to immutability and model mutation through controlled mechanisms like monads, processes, or atoms. This produces correct programs but can feel unnatural for inherently stateful problems, and persistent data structures carry performance overhead.

C and C++ give you full manual control and zero overhead, at the cost of memory safety. const in C is advisory at best; you can cast it away, and the compiler won't stop you from freeing memory that someone else is still using.

Lattice's phase system is an attempt to find a different point in this design space. The core insight is that in most programs, values have a natural lifecycle: they're constructed (requiring mutation), then used (requiring stability), and occasionally reconstructed (requiring mutation again). The phase system makes this lifecycle explicit and enforceable.

The Phase Model

Lattice has three binding keywords that correspond to mutability phases:

flux declares a mutable binding. A flux variable can be reassigned, and its contents can be modified in place. This is where you do your work: building arrays, populating maps, incrementing counters.

flux counter = 0
counter += 1
counter += 1

fix declares an immutable binding. A fix variable cannot be reassigned, and its contents cannot be modified. Attempting to mutate a fix binding is an error.

fix pi = freeze(3.14159)
// pi = 2.0  -- error: cannot assign to crystal binding

let is the inferred form (available in casual mode). It doesn't enforce a phase; the value keeps whatever phase tag it already has.

The transitions between phases are explicit function calls:

freeze(value) transitions a value from fluid to crystal. In strict mode, this is a consuming operation: the original binding is removed from the environment. You can't accidentally keep a mutable reference to something you've declared immutable.
thaw(value) creates a mutable deep clone of a crystal value. The original remains frozen; you get a completely independent mutable copy.
clone(value) creates a deep copy without changing phase.

And then there's the forge block, which is perhaps the most interesting construct:

fix config = forge {
    flux temp = Map::new()
    temp.set("host", "localhost")
    temp.set("port", "8080")
    temp.set("debug", "true")
    freeze(temp)
}

A forge block is a scoped computation whose result is automatically frozen. Inside the forge, you can use flux variables and mutate freely. But whatever value the block produces comes out crystallized. The temporary mutable state is gone; only the finished, immutable result survives.

This addresses a real pain point. In functional languages, building a complex immutable data structure often requires awkward chains of constructor calls or builder patterns. In Lattice, you just... build it, mutably, in a forge block, and it comes out frozen. The forge acknowledges that construction is inherently a mutable process, while insisting that the result of construction should be stable.

Under the Hood: How the Phase System Maps to Memory

Lattice is implemented as a tree-walking interpreter in C, roughly 6,000 lines across the lexer, parser, phase checker, and evaluator. The implementation reveals some interesting design decisions about how phase semantics interact with memory management.

Value Representation

Every runtime value in Lattice is a LatValue struct, a tagged union carrying a type tag, a phase tag, and the value payload:

struct LatValue {
    ValueType type;   // VAL_INT, VAL_STR, VAL_ARRAY, VAL_MAP, ...
    PhaseTag  phase;  // VTAG_FLUID, VTAG_CRYSTAL, VTAG_UNPHASED
    union { ... } as;
};

Primitive values (integers, floats, booleans) live inline in the union, with no heap allocation. Compound values (strings, arrays, structs, maps, closures) own heap-allocated payloads. A string holds a heap-allocated character buffer. An array holds a malloc'd element buffer. A map holds a pointer to an open-addressing hash table.

Deep-Clone-on-Read: Value Semantics Without a Compiler

The most consequential design decision in Lattice's runtime is that every variable read produces a deep clone. When you access a variable, the environment doesn't hand you a reference to the stored value. It hands you a complete, independent copy.

bool env_get(const Env *env, const char *name, LatValue *out) {
    for (size_t i = env->count; i > 0; i--) {
        LatValue *v = lat_map_get(&env->scopes[i-1], name);
        if (v) {
            *out = value_deep_clone(v);  // always a fresh copy
            return true;
        }
    }
    return false;
}

This is expensive. Every array access clones the entire array. Every map read clones every key-value pair. But it eliminates an entire class of bugs. There is no aliasing in Lattice. Two variables never point to the same underlying memory. When you pass a map to a function, the function gets its own copy, and mutations inside the function don't leak back to the caller. When you assign an array to a new variable, you get two independent arrays.

This is the implementation strategy that makes Lattice's maps value types. In most languages, objects and collections are reference types, and assigning them to a new variable creates a new reference to the same data. In Lattice, assignment means duplication. This is closer to how values work in mathematics than how they work in most programming languages.

For in-place mutation within a scope (like array.push() or map.set()), Lattice uses a separate resolve_lvalue() mechanism that obtains a direct mutable pointer into the environment's storage, bypassing the deep clone. This means local mutations are efficient; it's only cross-scope communication that pays the cloning cost.

The Dual Heap Architecture

Lattice's memory subsystem uses what the implementation calls a DualHeap: two separate allocation regions with different management strategies:

The FluidHeap manages mutable data using a mark-and-sweep garbage collector. It maintains a linked list of all heap allocations, with a mark bit on each. When memory pressure crosses a threshold (1 MB by default), the GC walks all reachable values from the environment and a shadow root stack, marks what's alive, and sweeps everything else.

The RegionManager manages immutable data using arena-based regions. Each freeze creates a new region backed by a page-based arena, a linked list of 4 KB pages with bump allocation. When a value is frozen, it is deep-cloned entirely into the region's arena, giving crystal data cache locality and enabling O(1) bulk deallocation when the region becomes unreachable. Regions are collected during GC cycles based on reachability analysis.

The key insight here is that immutable and mutable data have different lifecycle characteristics and benefit from different management strategies. Mutable data changes frequently and has unpredictable lifetimes, and mark-and-sweep handles this well. Immutable data, once created, never changes and tends to be long-lived, so arena-based region allocation is more efficient for this pattern, as it enables bulk deallocation and better cache locality.

This is conceptually similar to generational garbage collection (where young objects are collected differently from old objects), but the split is based on mutability rather than age. Lattice's phase tags provide the runtime with information that generational GCs have to infer statistically.

The following chart shows how this plays out in practice across several benchmark programs. Fluid peak memory represents the high-water mark of the GC-managed heap, while crystal arena data shows how much data has been frozen into arena-backed regions:

Freeze and Thaw at the Memory Level

When you call freeze() on a value, the runtime creates a new crystal region with a fresh arena, deep-clones the entire value tree into it, sets the phase field to VTAG_CRYSTAL on every node, and frees the original fluid heap pointers. The data physically migrates from the fluid heap into arena pages. Freeze is a move operation, not just a metadata flip. This gives frozen data cache locality within contiguous arena pages and completely separates it from the garbage-collected fluid heap.

But in strict mode, freeze() is also a consuming operation. It removes the original binding from the environment and returns the frozen value. This is effectively a move: after freeze(x), there is no x anymore. You can bind the result to a new name (fix y = freeze(x)), but the mutable original is gone. This prevents a common bug pattern where you freeze a value but accidentally keep mutating the original through a still-live reference.

thaw() is more expensive: it performs a complete deep clone of the crystal value and then recursively sets all phase tags to VTAG_FLUID. The original crystal value is untouched; you get a completely independent mutable copy. This is consistent with the principle that crystal values are permanent. Thawing doesn't melt the original; it creates a new fluid copy.

In practice, both operations are fast. Across the benchmark suite, freeze and thaw costs stay well under a millisecond even for complex data structures:

The number and type of phase transitions varies by workload. Some benchmarks are freeze-heavy (building immutable snapshots), others are thaw-heavy (repeatedly modifying frozen state), and some use deep clones for full value duplication:

How This Compares to Existing Systems

vs. Rust's Ownership and Borrowing

Rust solves the mutability problem at compile time through static analysis. The borrow checker ensures that mutable references are unique and that immutable references don't coexist with mutable ones. This gives Rust zero-runtime-cost safety guarantees that Lattice can't match.

But Rust's approach operates at the reference level. It tracks who has access to data, not the data's intrinsic state. You can have an &mut to data that is conceptually "done being built," or an & to data that you wish you could modify. The permission model and the data lifecycle are orthogonal.

Lattice's phase system operates on the data itself. A frozen value is immutable, not because the type system prevents you from obtaining a mutable reference, but because the value has transitioned to a state where mutation doesn't apply. This is a simpler mental model at the cost of runtime enforcement rather than compile-time proof.

The consuming freeze() in strict mode is reminiscent of Rust's move semantics, where using a value after moving it is a compile error. Lattice achieves a similar effect at runtime: freeze consumes the binding, preventing further mutable access. It's not as strong a guarantee (runtime vs. compile time), but it's the same intuition: once you've declared something immutable, the mutable version shouldn't exist anymore.

vs. Garbage Collection

Traditional garbage collectors (Java, Go, Python) are phase-agnostic. They track reachability, not mutability. A final field in Java prevents reassignment but doesn't inform the GC. An immutable object in Python is collected the same way as a mutable one.

Lattice's dual-heap architecture uses phase information to make better allocation decisions. Crystal values go into arena-managed memory with reachability-based collection. Fluid values go into a mark-and-sweep heap. The GC can reason about immutable data more efficiently because it knows the data won't change, so it doesn't need to re-scan crystal regions for updated references.

This is a form of phase-informed memory management that, to my knowledge, doesn't have a direct precedent in mainstream languages. The closest analogy might be Clojure's persistent data structures, which are structurally shared and immutable, but Clojure doesn't use this information to drive its garbage collection strategy differently.

vs. Functional Immutability

Haskell and other pure functional languages are immutable by default, with mutation confined to monads (IORef, STRef) or similar controlled mechanisms. This is elegant but can be awkward for imperative algorithms where you need to build something up step by step.

Lattice's forge blocks address this directly. Instead of threading a builder through a chain of pure function calls, you write imperative mutation inside a forge and get an immutable result. This acknowledges that construction and consumption are different activities that benefit from different mutability guarantees.

The philosophical difference is that functional languages treat immutability as the default and mutation as the exception. Lattice treats mutability as a phase that values pass through, and both flux and fix are natural, expected states, and the language provides explicit tools for transitioning between them.

vs. C/C++ Manual Memory Management

C gives you malloc and free and wishes you the best. C++ adds RAII, smart pointers, and const correctness, but const in both languages is fundamentally a compiler hint. It can be cast away, and the runtime has no awareness of it. A const pointer in C doesn't prevent someone else from modifying the data through a non-const pointer to the same memory. The const is a property of the reference, not the data.

Lattice's phase tags live on the data itself. When a value is crystal, it's crystal regardless of how you access it. There's no way to "cast away" a freeze; the only path back to mutability is thaw(), which creates a new independent copy. This is a stronger guarantee than const provides, because it operates on values rather than references.

C++ move semantics share DNA with Lattice's consuming freeze() in strict mode. A std::move in C++ transfers ownership of resources, leaving the source in a valid-but-unspecified state. Lattice's strict freeze does something similar: it removes the binding entirely, ensuring the mutable version ceases to exist. But where C++ moves are primarily about avoiding copies for performance, Lattice's consuming freeze is about semantic correctness, ensuring that the transition from mutable to immutable is clean and total. Scott Meyers' Effective Modern C++ remains the best guide to understanding these move semantics and other modern C++ patterns that Lattice's design draws from.

The Static Phase Checker

It's worth noting that Lattice doesn't rely solely on runtime enforcement. Before any code executes, a static phase checker walks the AST and catches phase violations at analysis time. This checker maintains its own scope stack mapping variable names to their declared phases and rejects programs that attempt to reassign crystal bindings, freeze already-frozen values, thaw already-fluid values, or use let in strict mode where an explicit phase declaration is required.

The static checker also enforces spawn boundaries. If Lattice's concurrency model (spawn) is used, fluid bindings from the enclosing scope cannot be captured across the spawn point. Only crystal values can be shared into spawned computations. This is checked before evaluation begins, catching potential data races at parse time rather than at runtime.

This two-layer approach (static checking before evaluation, runtime enforcement during) provides confidence without requiring a full type system or borrow checker. It catches the obvious mistakes early and enforces the subtle invariants at runtime. For the theoretical foundations behind this kind of phase-based type analysis, Benjamin Pierce's Types and Programming Languages is the standard reference.

The Language Beyond Phases

While the phase system is Lattice's defining feature, the language has other characteristics worth noting.

Structs in Lattice can hold closures as fields, enabling object-like patterns without a class system. A struct with function fields and a self parameter in each closure behaves much like an object with methods, but the data flow is explicit, and there's no hidden this pointer or vtable dispatch. When a closure captures self, it receives a deep clone, ensuring that method calls don't produce spooky action at a distance.

Control flow is expression-based: if/else blocks, match expressions, and bare blocks all return values. This reduces the need for temporary variables and makes code more compositional. Error handling uses try/catch blocks with explicit error values rather than exception hierarchies.

The self-hosted REPL is particularly notable. Written entirely in Lattice, it demonstrates that the language is expressive enough to implement its own interactive environment, parsing multi-line input, evaluating expressions, and managing session state. Running ./clat without arguments drops into this REPL, while ./clat file.lat executes a program directly.

Lattice is implemented in C with no external dependencies. The entire codebase (roughly 6,000 lines across the lexer, parser, phase checker, evaluator, and data structures) compiles with a single make invocation. This is a deliberate choice. The language is meant to be small, understandable, and self-contained. You can read the entire implementation in an afternoon. If you're interested in this kind of work, Robert Nystrom's Crafting Interpreters is the best practical guide to building language implementations from scratch. It covers both tree-walking interpreters and bytecode VMs, and Lattice's architecture shares several design decisions with Nystrom's Lox language. For the C implementation side, Kernighan and Ritchie's The C Programming Language remains the definitive reference for writing the kind of clean, minimal C that Lattice targets.

Runtime Characteristics

To understand how the dual-heap architecture behaves in practice, Lattice includes a benchmark suite that exercises different memory patterns: allocation churn, closure-heavy computation, event sourcing, freeze/thaw cycles, game state rollback, long-lived crystal data, persistent tree construction, and undo/redo stacks.

The overview below shows peak RSS (resident set size) alongside the number of live crystal regions at program exit. Benchmarks that use the phase system heavily (like freeze/thaw cycles and persistent trees) maintain more live regions, while purely fluid workloads like allocation churn and closure-heavy computation have none:

The memory churn ratio (total bytes allocated divided by peak live bytes) reveals how aggressively each benchmark recycles memory. A high ratio means the program allocates and discards data rapidly, relying on the GC to keep the working set small. Benchmarks using crystal regions (shown in purple) tend to have lower churn because frozen data is long-lived by design:

Research Papers

For readers interested in the formal foundations and empirical analysis, two companion papers are available:

The Lattice Phase System: First-Class Immutability with Dual-Heap Memory Management: The full research paper covering the language design, formal operational semantics, six proved safety properties (phase monotonicity, value isolation, consuming freeze, forge soundness, heap separation, and thaw independence), implementation details of the dual-heap architecture, and empirical evaluation across eight benchmarks.
Formal Semantics of the Lattice Phase System: A standalone formal treatment containing the complete semantic domains, static phase-checking rules, big-step operational semantics, memory model, and full proofs of all six safety theorems.

Looking Forward

Lattice is at version 0.1.3, which means it's early. The dual-heap architecture is fully wired into the evaluator. Freeze operations physically migrate data into arena-backed crystal regions, providing cache locality and O(1) bulk deallocation for immutable data. The mark-and-sweep GC handles fluid values, while crystal regions are collected through reachability analysis during GC cycles.

The deep-clone-on-read strategy is correct but expensive. Future versions may introduce structural sharing for crystal values (since they can't be modified, sharing is safe) or copy-on-write semantics for fluid values that haven't actually been mutated. The phase tags provide the runtime with exactly the information needed to make these optimizations: which values can be shared safely, and which might change.

There's also the question of concurrency. The phase system provides a natural foundation for safe concurrent programming: crystal values can be freely shared across threads (they're immutable), while fluid values are confined to their owning scope. The spawn keyword exists in the parser and phase checker, with static analysis already preventing fluid bindings from crossing spawn boundaries, though concurrent execution isn't yet implemented.

The source code is available on GitHub under the BSD 3-Clause license, and the project site is at lattice-lang.org. If you're interested in language design, memory management, or just want to play with a language that treats mutability as a physical process rather than a type annotation, it's worth a look.

git clone https://github.com/ajokela/lattice.git
cd lattice && make
./clat

TinyComputers.io (Posts about language design)

Teaching an LLM a Language It Has Never Seen

The Rust Smell

The Phase System: Where Familiarity Ends

The Spectrum of Difficulty

The Self-Hosted Compiler

What I Actually Gave Claude

Why Syntax Matters More Than Semantics

Implications for Language Designers

What This Means for the "AI Writes Code" Conversation

A Stack-Based Bytecode VM for Lattice: 100 Opcodes, Serialization, and a Self-Hosted Compiler

Why Keep Going

100 Opcodes

Integer Fast Paths

Wide Constant Variants

The Compiler

Computed Goto Dispatch

Pre-Compiled Concurrency

Bytecode Serialization

The Ephemeral Bump Arena

Closures and the Storage Hack

The Self-Hosted Compiler

The VM Execution Engine

Ref<T>: The Escape Hatch from Value Semantics

Validation

What's Next

Rue: Steve Klabnik's AI-Assisted Experiment in Memory Safety Without the Pain

The Pitch

The Person Behind It

What Rue Is Today

The Type System and Memory Model

What the Tutorial Reveals

The AI Development Story

What the Source Code Reveals

The Fundamental Question

Who Should Pay Attention

Conclusion

Review of "Crafting Interpreters" by Robert Nystrom

Introduction

The Language: Lox

Part II: The Tree-Walk Interpreter (jlox)

Part III: The Bytecode Virtual Machine (clox)

Writing Style and Presentation

Strengths

Limitations

Who Should Read This Book

Conclusion

From Tree-Walker to Bytecode VM: Compiling Lattice

Architecture Overview

The Instruction Set

Closures and Upvalues

Compiling for the REPL

The Global Mutation Bug

The WASM Playground

What Didn't Change

Performance

Looking Forward

Recommended Resources

Mutability as a First-Class Concept: The Lattice Phase System

Mutability as a First-Class Concept: The Lattice Phase System

The Metaphor: Crystallization

Phase Constraints: Mutability in Your Type Signatures

Phase-Dependent Dispatch

Crystallization Contracts: Validation at the Phase Boundary

Phase Bonds: Coordinated Freezing

Phase Reactions: Observing State Transitions

Temporal Values: Phase History and Time Travel

The Bigger Picture: Why This Matters

Comparison with Other Approaches

Implementation Notes

What's Next

Introducing Lattice: A Crystallization-Based Programming Language

The Problem Lattice Solves

The Phase Model

Under the Hood: How the Phase System Maps to Memory

Value Representation

Deep-Clone-on-Read: Value Semantics Without a Compiler

The Dual Heap Architecture

Freeze and Thaw at the Memory Level

How This Compares to Existing Systems

Part II: The Tree-Walk Interpreter (`jlox`)

Part III: The Bytecode Virtual Machine (`clox`)