In my previous post, I documented building an LLVM backend for the Z80 processor. The backend worked — simple LLVM IR compiled to valid Z80 assembly. But that post ended with a sobering admission: Rust's core library remained out of reach, its abstractions overwhelming the constraints of 1976 hardware.

This post picks up where that one left off. The question nagging at me was simple: can we actually compile real Rust code into Z80 assembly? Not just hand-crafted LLVM IR, but genuine Rust source files with functions and variables and all the conveniences we expect from a modern language?

The answer is yes. But getting there required more RAM than any Z80 system ever had, a creative workaround that sidesteps Rust's build system entirely, and a willingness to accept that sometimes the elegant solution isn't the one that works.

The Hardware Reality Check

Before diving into the technical details, I need to address something that caught me off guard: the sheer computational resources required to compile code for an 8-bit processor.

My first attempt was on my M3 Max MacBook Pro. The machine is no slouch — 64GB of unified memory, fast SSD, Apple's impressive silicon. Building LLVM with the Z80 backend worked fine. Building stage 1 of the Rust compiler worked, albeit slowly. But when I tried to build Rust's core library for the Z80 target, the process crawled. After watching it churn for hours with no end in sight, I gave up.

The next attempt used a Linux workstation with 32GB of RAM. This seemed reasonable — surely 32GB is enough to compile code for a processor with a 64KB address space? It wasn't. The build process hit out-of-memory errors during the compilation of compiler_builtins, a Rust crate that provides low-level runtime functions.

To understand why, you need to know what compiler_builtins actually does. When you write code like let x: u64 = a * b;, and your target processor doesn't have native 64-bit multiplication (the Z80 doesn't even have 8-bit multiplication), something has to implement that operation in software. That something is compiler_builtins. It contains hundreds of functions: software implementations of multiplication, division, floating-point operations, and various other primitives that high-level languages take for granted. Each of these functions gets compiled, optimized, and linked into your final binary.

For the Z80, every one of these functions presents a challenge. 64-bit division on an 8-bit processor expands into an enormous sequence of instructions. The LLVM optimizer works hard to improve this code, and that optimization process consumes memory — lots of it.

The machine that finally worked was a dedicated build server:

OS: Ubuntu 24.04.3 LTS x86_64
Host: Gigabyte G250-G51 Server
CPU: Intel Xeon E5-2697A v4 (64 cores) @ 3.600GHz
Memory: 252GB DDR4
GPU: 4x NVIDIA Tesla P40 (unused for compilation)

With 252GB of RAM and 64 cores, the build finally had room to breathe. LLVM with Z80 support built in about 45 minutes. The Rust stage 1 compiler built in 11 minutes. And when we attempted to build compiler_builtins for Z80, memory usage peaked at 169GB.

Let that sink in: compiling runtime support code for a processor with 64KB of addressable memory required 169GB of RAM. The ratio is absurd — we needed 2.6 million times more memory to compile the code than the target system could ever access. This is what happens when modern software toolchains, designed for 64-bit systems with gigabytes of RAM, encounter hardware from an era when 16KB was a luxury.

The Naive Approach and Why It Fails

With our beefy build server ready, the obvious approach was to build Rust's core library for the Z80 target. The core library is Rust's foundation — it provides basic types like Option and Result, fundamental traits like Copy and Clone, and essential operations like memory manipulation and panicking. Unlike std, which requires an operating system, core is designed for bare-metal embedded systems. If anything could work on a Z80, surely core could.

The first obstacle was unexpected. Rust's build system uses a crate called cc to compile C code and detect target properties. When we ran the build, it immediately failed:

error occurred in cc-rs: target `z80-unknown-none-elf` had an unknown architecture

The cc crate maintains a list of known CPU architectures, and Z80 wasn't on it. The fix was simple — a one-line patch to add "z80" => "z80" to the architecture matching code — but we had to apply it to every version of cc in the cargo registry cache. Not elegant, but effective.

With that patched, the build progressed further before hitting a more fundamental problem:

rustc-LLVM ERROR: unable to legalize instruction: %35:_(s16) = nneg G_UITOFP %10:_(s64)

This error comes from LLVM's GlobalISel pipeline, specifically the Legalizer. To understand it, I need to explain how LLVM actually turns high-level code into machine instructions.

What is GlobalISel and Why Does It Matter?

When you compile code with LLVM, there's a critical step called "instruction selection" — the process of converting LLVM's abstract intermediate representation (IR) into concrete machine instructions for your target CPU. This is harder than it sounds. LLVM IR might say "add these two 32-bit integers," but your CPU might only have 8-bit addition, or it might have three different add instructions depending on whether the operands are in registers or memory.

Historically, LLVM used a framework called SelectionDAG for this task. SelectionDAG works, but it operates on individual basic blocks (straight-line code between branches) and makes decisions that are hard to undo later. For well-established targets like x86 and ARM, SelectionDAG is mature and produces excellent code. But for new or unusual targets, it's difficult to work with.

GlobalISel (Global Instruction Selection) is LLVM's modern replacement. The "Global" in the name refers to its ability to see across basic block boundaries, making better optimization decisions. More importantly for our purposes, GlobalISel breaks instruction selection into distinct, understandable phases:

1. IRTranslator: Converts LLVM IR into generic machine instructions. These instructions have names like G_ADD (generic add), G_LOAD (generic load), and G_UITOFP (generic unsigned integer to floating-point conversion). At this stage, the code is still target-independent — G_ADD doesn't know if it'll become an x86 ADD, an ARM add, or a Z80 ADD A,B.

2. Legalizer: This is where target constraints enter the picture. The Legalizer transforms operations that the target can't handle into sequences it can. If your target doesn't support 64-bit addition directly, the Legalizer breaks it into multiple 32-bit or 16-bit additions. If your target lacks a multiply instruction (hello, Z80), the Legalizer replaces multiplication with a function call to a software implementation.

3. RegBankSelect: Assigns each value to a register bank. For the Z80, this means deciding whether something lives in 8-bit registers (A, B, C, D, E, H, L) or 16-bit register pairs (BC, DE, HL). This phase is crucial for the Z80 because using the wrong register bank means extra move instructions.

4. InstructionSelector: Finally converts the now-legal, register-bank-assigned generic instructions into actual target-specific instructions. G_ADD becomes ADD A,B or ADD HL,DE depending on the operand types.

For the Z80 backend, GlobalISel was the right choice. It gave us fine-grained control over how operations get lowered on extremely constrained hardware. The downside is that every operation needs explicit handling — if the Legalizer doesn't know how to transform a particular instruction for Z80, compilation fails.

The error we hit was in the Legalizer. The G_UITOFP instruction converts an unsigned integer to floating-point. In this case, it was trying to convert a 64-bit integer to a 16-bit half-precision float. This operation appears deep in Rust's core library, in the decimal number parsing code used for floating-point literals.

The Z80 has no floating-point hardware whatsoever. It can't even do integer multiplication in a single instruction. Teaching LLVM to "legalize" 64-bit-to-float conversions on such constrained hardware would require implementing software floating-point operations — a significant undertaking that would generate hundreds of Z80 instructions for a single high-level operation.

Even setting aside the floating-point issue, we encountered another class of failures: LLVM assertion errors in the GlobalISel pipeline when handling complex operations. These manifested as crashes with messages about register operand sizes not matching expectations. The Z80 backend is experimental, and its GlobalISel support doesn't cover every edge case that Rust's core library exercises.

The fundamental problem became clear: Rust's core library, while designed for embedded systems, assumes a level of hardware capability that the Z80 simply doesn't have. It assumes 32-bit integers work efficiently. It assumes floating-point parsing is reasonable. It assumes the register allocator can handle moderately complex control flow.

The Workaround: Cross-Compile and Retarget

When the direct path is blocked, you find another way around.

The key insight is that LLVM IR (Intermediate Representation) is largely target-agnostic. When Rust compiles your code, it first generates LLVM IR, and then LLVM transforms that IR into target-specific assembly. The IR describes your program's logic — additions, function calls, memory accesses — without committing to a specific instruction set.

This suggests a workaround: compile Rust code to LLVM IR using a different target that Rust fully supports, then manually retarget that IR to Z80 and run it through our Z80 LLVM backend.

For the donor target, I chose thumbv6m-none-eabi — the ARM Cortex-M0, a 32-bit embedded processor. This target is well-supported in Rust's ecosystem, and crucially, it's a no_std target designed for resource-constrained embedded systems. The generated IR would be reasonably close to what we'd want for Z80, minus the data layout differences.

The workflow looks like this:

1. Write Rust code with #![no_std] and #![no_main]
2. Compile for ARM: cargo +nightly build --target thumbv6m-none-eabi -Zbuild-std=core
3. Extract the LLVM IR from the build artifacts (the .ll files)
4. Modify the IR's target triple and data layout for Z80
5. Compile to Z80 assembly: llc -march=z80 -O2 input.ll -o output.s

The data layout change is important. ARM uses 32-bit pointers; Z80 uses 16-bit pointers. The Z80 data layout string is:

e-m:e-p:16:8-i16:8-i32:8-i64:8-n8:16

This tells LLVM: little-endian, ELF mangling, 16-bit pointers with 8-bit alignment, native types are 8-bit and 16-bit. When we retarget the IR, we need to update this layout and the target triple to z80-unknown-unknown.

Is this elegant? No. It's a hack that bypasses Rust's proper build system. But it works, and sometimes working beats elegant.

Hello Z80 World

Let's put this into practice with the classic first program.

Here's the Rust source code:

#![no_std]
#![no_main]

use core::panic::PanicInfo;

// Memory-mapped serial output at address 0x8000
const SERIAL_OUT: *mut u8 = 0x8000 as *mut u8;

#[inline(never)]
#[no_mangle]
pub extern "C" fn putchar(c: u8) {
    unsafe {
        core::ptr::write_volatile(SERIAL_OUT, c);
    }
}

#[no_mangle]
pub extern "C" fn hello_z80() {
    putchar(b'H'); putchar(b'e'); putchar(b'l'); putchar(b'l'); putchar(b'o');
    putchar(b' '); putchar(b'Z'); putchar(b'8'); putchar(b'0'); putchar(b'!');
    putchar(b'\r'); putchar(b'\n');
}

#[panic_handler]
fn panic(_: &PanicInfo) -> ! { loop {} }

This is genuine Rust code. We're using core::ptr::write_volatile for memory-mapped I/O, the extern "C" calling convention for predictable symbol names, and #[no_mangle] to preserve function names in the output. The #[inline(never)] on putchar ensures it remains a separate function rather than being inlined into the caller.

After compiling to ARM IR and retargeting to Z80, we run it through llc. The output is real Z80 assembly:

    .globl  putchar
putchar:
    ld   de,32768        ; Load address 0x8000
    push de
    pop  hl              ; DE -> HL (address now in HL)
    ld   (hl),a          ; Store A register to memory
    ret

    .globl  hello_z80
hello_z80:
    push ix              ; Save frame pointer
    ld   ix,0
    add  ix,sp           ; Set up stack frame
    dec  sp              ; Allocate 1 byte on stack
    ld   a,72            ; 'H'
    call putchar
    ld   a,101           ; 'e'
    call putchar
    ld   a,108           ; 'l'
    ld   (ix+-1),a       ; Save 'l' to stack (optimization!)
    call putchar
    ld   a,(ix+-1)       ; Reload 'l' for second use
    call putchar
    ld   a,111           ; 'o'
    call putchar
    ld   a,32            ; ' '
    call putchar
    ld   a,90            ; 'Z'
    call putchar
    ld   a,56            ; '8'
    call putchar
    ld   a,48            ; '0'
    call putchar
    ld   a,33            ; '!'
    call putchar
    ld   a,13            ; '\r'
    call putchar
    ld   a,10            ; '\n'
    call putchar
    ld   sp,ix           ; Restore stack
    pop  ix              ; Restore frame pointer
    ret

This is valid Z80 assembly that would run on real hardware. The putchar function loads the serial port address into the HL register pair and stores the character from the A register. The hello_z80 function calls putchar twelve times, once for each character in "Hello Z80!\r\n".

Notice something interesting: the compiler optimized the duplicate 'l' character. Instead of loading 108 into the A register twice, it saves the value to the stack after the first use and reloads it for the second. This is LLVM's register allocator at work, recognizing that reusing a value from the stack is cheaper than reloading an immediate. The Z80 backend is generating genuinely optimized code.

Running on (Emulated) Hardware

Generating assembly is satisfying, but seeing it actually execute closes the loop. I have a Rust-based Z80 emulator that I use for testing RetroShield firmware. It emulates the Z80 CPU along with common peripheral chips, including the MC6850 ACIA serial chip that my physical hardware uses.

To run our Hello World, we need to adapt the memory-mapped I/O to use the ACIA's port-based I/O instead. The MC6850 uses port $80 for status and port $81 for data. A proper implementation waits for the Transmit Data Register Empty (TDRE) bit before sending each character:

; Hello Z80 World - Compiled from Rust via LLVM
; Adapted for MC6850 ACIA serial output

ACIA_STATUS:    equ     $80
ACIA_DATA:      equ     $81

        org     $0000

_start:
        ld      hl, MESSAGE
        ld      b, MESSAGE_END - MESSAGE

print_loop:
wait_ready:
        in      a, (ACIA_STATUS)
        and     $02                ; Check TDRE bit
        jr      z, wait_ready

        ld      a, (hl)
        out     (ACIA_DATA), a
        inc     hl
        djnz    print_loop

halt_loop:
        halt
        jr      halt_loop

MESSAGE:
        defb    "Hello, Z80 World!", $0D, $0A
MESSAGE_END:

This is the essence of what our Rust code does, translated to the actual hardware interface. The infinite loop at the end mirrors Rust's loop {} — on bare metal, there's nowhere to return to.

Assembling with z80asm produces a 39-byte binary. Running it in the emulator:

$ ./retroshield -d -c 10000 hello_rust.bin
Loaded 39 bytes from hello_rust.bin
Starting Z80 emulation...
Hello, Z80 World!

CPU halted at PC=0011 after 1194 cycles

The program executes in 1,194 Z80 cycles — roughly 300 microseconds at the original 4MHz clock speed. The complete pipeline works:

1. Rust source code → compiled to LLVM IR via rustc
2. LLVM IR → retargeted to Z80 and compiled to assembly
3. Z80 assembly → assembled to binary with z80asm
4. Binary → executed in the Z80 emulator

The 39-byte binary breaks down to about 20 bytes of executable code and 19 bytes for the message string. This is exactly what bare-metal #![no_std] Rust should produce — tight, efficient code with zero runtime overhead.

What Works and What Doesn't

Through experimentation, we've mapped out the boundaries of what the Z80 backend handles well.

Works reliably:

• 8-bit arithmetic: addition, subtraction, bitwise operations. These map directly to Z80 instructions like ADD A,B and AND B.
• 16-bit arithmetic: addition and subtraction use the Z80's 16-bit register pairs (HL, DE, BC) efficiently.
• Memory operations: loads and stores generate clean LD (HL),A and LD A,(HL) sequences.
• Function calls: the calling convention uses registers efficiently, avoiding unnecessary stack operations for simple cases.
• Simple control flow: conditional branches and unconditional jumps work as expected.

Works but generates bulky code:

• 32-bit arithmetic: every 32-bit operation expands into multiple 16-bit operations with careful carry flag handling. A 32-bit addition becomes a sequence that would make a Z80 programmer wince.
• Multiplication: even 8-bit multiplication requires a library call to __mulhi3 since the Z80 lacks a multiply instruction.

Breaks the register allocator:

• Loops with phi nodes: in LLVM IR, loops use phi nodes to represent values that differ depending on which path entered the loop. Complex phi nodes exhaust the Z80's seven registers, causing "ran out of registers" errors.
• Functions with many live variables: if you need more than a handful of values alive simultaneously, the backend can't handle it.

Not supported:

• Floating-point operations: no legalization rules exist for converting the Z80's lack of FPU into software equivalents.
• Complex core library features: iterators, formatters, and most of the standard library infrastructure trigger unsupported operations.

The Calling Convention

Through testing, we've empirically determined how our Z80 backend passes arguments and returns values:

Additional arguments go on the stack. The stack frame uses the IX register as a frame pointer when needed. This convention minimizes register shuffling for common cases — a function taking two 16-bit arguments and returning one uses HL and DE for input and HL for output, requiring no setup at all.

This differs from traditional Z80 calling conventions used by C compilers, which typically pass all arguments on the stack. Our approach is more register-heavy, which suits the short functions typical of embedded code.

Practical Implications

Let me be clear about what we've achieved and what remains out of reach.

What you can realistically build:

• Simple embedded routines: LED patterns, sensor reading, basic I/O handling
• Mathematical functions: integer arithmetic, lookup tables, state machines
• Protocol handlers: parsing simple data formats, generating responses
• Anything that would fit in a few kilobytes of hand-written assembly

What you cannot build:

• Anything requiring heap allocation: no Vec, no String, no dynamic data structures
• Code using iterators or closures: these generate complex LLVM IR that overwhelms the register allocator
• Formatted output: Rust's write! macro and formatting infrastructure are far too heavy
• Floating-point calculations: not without significant backend work

The path to making this more capable is visible but non-trivial. A custom minimal core implementation that avoids floating-point entirely would help. Improving the register allocator's handling of phi nodes would enable loops. Adding software floating-point legalization would unlock numerical code. Each of these is a substantial project.

Reflections

Building a compiler backend for a 50-year-old processor using a 21st-century language toolchain is an exercise in contrasts. Modern software assumes abundant resources. The Z80 was designed when resources were precious. Making them meet requires translation across decades of computing evolution.

The fact that we needed 252GB of RAM to compile code for a processor with a 64KB address space is almost poetic. It captures something essential about how far computing has come and how much we've traded simplicity for capability.

But here's what satisfies me: the generated Z80 code is good. It's not bloated or obviously inefficient. When we compile a simple function, we get a simple result. The LLVM optimization passes do their job, and our backend translates the result into idiomatic Z80 assembly. The 'l' character optimization in our Hello World example isn't something I would have thought to do by hand, but the compiler found it automatically.

Rust on Z80 isn't practical for production use. The core library is too heavy, the workarounds are too fragile, and the resulting code size would exceed most Z80 systems' capacity. But as a demonstration that modern toolchains can target ancient hardware? As an exploration of what compilers actually do? As an answer to "I wonder if this is possible?"

Yes. It's possible. And the journey to get here taught me more about LLVM, register allocation, and instruction selection than any tutorial ever could.

What's Next

With emulation working, the obvious next step is running this code on actual hardware. My RetroShield Z80 sits waiting on my workbench, ready to execute whatever binary we load into it. The emulator uses the same ACIA interface as the physical hardware, so the transition should be straightforward — load the binary, connect a terminal, and watch "Hello, Z80 World!" appear on genuine 8-bit silicon.

Beyond hardware validation, the Z80 backend needs work on loop handling. Phi nodes are the enemy. There may be ways to lower them earlier in the pipeline, before they reach the register-hungry instruction selector. That's a project for another day, another blog post, and probably another round of pair programming with Claude.

The projects are available on GitHub for anyone curious enough to try them:

• LLVM with Z80 backend: github.com/ajokela/llvm-z80
• Rust with Z80 target: github.com/ajokela/rust-z80 (see the z80-backend branch)
• Z80 Emulator: github.com/ajokela/retro-z80-emulator

Be warned: you'll need more RAM than seems reasonable. But if you've read this far, you probably already suspected that.

Resources

If you want to dive deeper into any of the topics covered here, these resources might help:

Books:

• Programming the Z80 by Rodnay Zaks — The definitive Z80 reference, covering every instruction and addressing mode in detail
• The Rust Programming Language by Klabnik and Nichols — The official Rust book, essential for understanding no_std embedded development
• Engineering a Compiler by Cooper and Torczon — Comprehensive compiler textbook covering instruction selection, register allocation, and code generation
• Crafting Interpreters by Robert Nystrom — Excellent practical guide to building language implementations

Hardware:

• Z80 CPU chips — Original Zilog Z80 processors, still available new and vintage
• RetroShield Z80 — Arduino shield that lets you run a real Z80 with modern conveniences
• USB to Serial adapters — Essential for connecting to vintage hardware
• Logic Analyzer — Invaluable for debugging Z80 bus timing and signals

There's something magical about watching a 45-year-old operating system boot on modern hardware. CP/M 2.2, the operating system that launched a thousand microcomputers and paved the way for MS-DOS, still has lessons to teach us about elegant system design.

This post documents my journey getting CP/M 2.2 running on the RetroShield Z80 emulator, a Rust-based Z80 emulator I've been developing. The result is a fully functional CP/M system that can run classic software like Zork and WordStar.

What is CP/M?

CP/M (Control Program for Microcomputers) was created by Gary Kildall at Digital Research in 1974. It became the dominant operating system for 8-bit microcomputers in the late 1970s and early 1980s, running on machines like the Altair 8800, IMSAI 8080, Osborne 1, and Kaypro.

CP/M's genius was its portability. The system separated into three layers:

• CCP (Console Command Processor) - The command line interface
• BDOS (Basic Disk Operating System) - File and I/O services
• BIOS (Basic Input/Output System) - Hardware abstraction

Only the BIOS needed to be rewritten for each machine. This architecture directly influenced MS-DOS and, by extension, every PC operating system that followed.

The RetroShield Z80 Emulator

The RetroShield is a hardware shield that lets you run vintage CPUs on modern microcontrollers. My emulator takes this concept further by providing a complete software simulation of the Z80 and its peripherals.

The emulator includes:

• Full Z80 CPU emulation (via the rz80 crate)
• MC6850 ACIA serial port (console I/O)
• SD card emulation with DMA block transfers
• TUI debugger with memory viewer, disassembly, and single-stepping

The Challenge: Disk I/O

Getting CP/M's console I/O working was straightforward. The real challenge was disk I/O. CP/M expects to read and write 128-byte sectors from floppy disks. I needed to emulate this using files on the host system.

The standard 8" single-sided, single-density floppy format that CP/M uses:

• 77 tracks
• 26 sectors per track
• 128 bytes per sector
• 256KB total capacity

DMA Block Transfers

Rather than transferring bytes one at a time through I/O ports (which would be painfully slow), I implemented DMA block transfers. The BIOS sets up a DMA address and issues a single command to transfer an entire 128-byte sector:

; Set DMA address
ld      hl, (DMAADR)
ld      a, l
out     (SD_DMA_LO), a
ld      a, h
out     (SD_DMA_HI), a

; Issue block read
xor     a
out     (SD_BLOCK), a

; Check status
in      a, (SD_BLOCK)
ret                     ; A = 0 if OK

On the emulator side, this triggers a direct memory copy from the disk image file into emulated RAM.

The Bug That Almost Defeated Me

After implementing everything, CP/M would boot and print its banner, but then hang or show garbage. The debug output revealed the BDOS was requesting insane track numbers like 0x0083 instead of track 2.

The culprit? A classic use-after-move bug in the BIOS:

SELDSK:
    ; Calculate DPH address in HL
    ld      l, c
    ld      h, 0
    add     hl, hl          ; *16
    add     hl, hl
    add     hl, hl
    add     hl, hl
    ld      de, DPH0
    add     hl, de

    call    OPENDISK        ; BUG: This overwrites HL!
    ret                     ; Returns garbage instead of DPH

The OPENDISK subroutine was using HL internally, destroying the Disk Parameter Header address that SELDSK was supposed to return. The BDOS would then read garbage from the wrong memory location for its disk parameters.

The fix was simple:

    push    hl
    call    OPENDISK
    pop     hl              ; Restore DPH address
    ret

24-bit Seek Positions

Another issue: the disk images are 256KB, but I initially only supported 16-bit seek positions (64KB max). I added an extended seek port for the high byte:

pub const SD_SEEK_LO: u8 = 0x14;    // Bits 0-7
pub const SD_SEEK_HI: u8 = 0x15;    // Bits 8-15
pub const SD_SEEK_EX: u8 = 0x19;    // Bits 16-23

The Memory Map

CP/M's memory layout for a 56KB TPA (Transient Program Area):

0000-00FF   Page Zero (jump vectors, FCB, command buffer)
0100-DFFF   TPA - User programs load here (56KB)
E000-E7FF   CCP - Console Command Processor
E800-F5FF   BDOS - Basic Disk Operating System
F600-FFFF   BIOS - Hardware abstraction layer

The BIOS is only about 1KB of Z80 assembly, handling:

• Console I/O via the MC6850 ACIA
• Disk I/O via SD card emulation
• Drive selection and track/sector positioning

Running Classic Software

With CP/M booting successfully, I could run classic software:

Zork I - Infocom's legendary text adventure runs perfectly:

WordStar 3.3 and SuperCalc also run, though they need terminal escape codes configured properly (the Kaypro version uses ADM-3A codes).

Try It Yourself

The code is available on GitHub:

• RetroShield Z80 Emulator - The Rust emulator with SD card DMA support
• RetroShield CP/M - BIOS, boot loader, and CP/M system files

To run:

cd emulator/rust
cargo build --release
./target/release/retroshield_tui -s storage path/to/boot.bin

Press F5 to run, then type zork1 at the A> prompt.

Lessons Learned

Building this system reinforced some timeless principles:

1. Abstraction layers work. CP/M's BIOS/BDOS/CCP split made porting trivial. Only 1KB of code needed to be written for a completely new "hardware" platform.

2. Debug output is essential. Adding hex dumps of track/sector values immediately revealed the SELDSK bug.

3. Read the documentation. The CP/M 2.2 System Alteration Guide is remarkably well-written and explained exactly what the BIOS functions needed to do.

4. Old code still runs. With the right emulation layer, 45-year-old binaries execute flawlessly. The Z80 instruction set is eternal.

There's a certain satisfaction in seeing that A> prompt appear. It's the same prompt that greeted users in 1977, now running on code I wrote in 2025. The machines change, but the software endures.

In Finnish mythology, the Sampo is a magical artifact from the epic poem Kalevala, compiled by Elias Lönnrot in 1835. According to legend, the Sampo was forged by Ilmarinen, a legendary blacksmith and sky god, from a swan's feather, a grain of barley, a ball of wool, a drop of milk, and a shaft of a distaff. The resulting creation took the form of a magical mill that could produce flour, salt, and gold endlessly—bringing riches and good fortune to its holder.

The exact nature of the Sampo has been debated by scholars since 1818, with over 30 theories proposed—ranging from a world pillar to an astrolabe to a decorated shield. This mystery makes it a fitting namesake for a CPU architecture: something that transforms simple inputs into useful outputs, whose inner workings invite exploration and understanding.

This is the first part of a two-part series exploring the Sampo CPU architecture. In this article, we'll dive deep into the theory, design philosophy, and architectural decisions that shaped Sampo. In Part 2, we'll get our hands dirty with an actual FPGA implementation using Amaranth HDL, bringing this processor to life on real silicon.

The Problem Space: Why Another CPU?

Before diving into Sampo's architecture, it's worth asking: why design a new CPU at all? The retrocomputing community has no shortage of classic processors to explore—the Z80, 6502, 68000—and modern RISC architectures like RISC-V offer clean, well-documented designs for educational purposes.

The answer lies in a specific niche that existing architectures don't quite fill. Consider the typical workloads of classic 8-bit systems: interpreters for languages like BASIC and Forth, operating systems like CP/M, text editors, and simple games. These workloads have distinct characteristics:

1. Heavy use of memory operations: Block copies, string manipulation, memory fills
2. Port-based I/O: Serial terminals, disk controllers, sound chips accessed via dedicated I/O instructions
3. Context switching: Interrupt handlers that need to save and restore register state quickly
4. BCD arithmetic: Calculator applications, financial software

The Z80 excels at these tasks through specialized instructions (LDIR, LDDR, IN, OUT) and its alternate register set. But the Z80 is an 8-bit CISC processor with irregular encoding, complex addressing modes, and over 300 instruction variants. This makes it challenging to implement efficiently in modern hardware or to target with optimizing compilers.

Modern RISC architectures like RISC-V take the opposite approach: clean, orthogonal instruction sets optimized for pipelining and compiler code generation. But they typically use memory-mapped I/O (no dedicated I/O instructions), lack block operations, and provide no alternate register sets for fast context switching.

Sampo occupies the middle ground—a "Z80 programmer's RISC" that combines the regularity and simplicity of RISC design with the specialized capabilities that made the Z80 so effective for its target workloads.

Design Goals

Sampo was designed with five primary goals:

1. RISC-inspired instruction set: Clean, orthogonal design with predictable encoding
2. 16-bit native word size: Registers, ALU, and memory addressing all 16-bit
3. Efficient for interpreters and compilers: Stack operations, indirect addressing, hardware multiply/divide
4. Simple to implement: Suitable for FPGA synthesis or software emulation
5. Z80-workload compatible: Port-based I/O, BCD support, block operations, alternate registers

These goals create natural tensions. RISC purity would eliminate block operations and port-based I/O. Maximum Z80 compatibility would preserve its irregular encoding. Sampo resolves these tensions by borrowing selectively from multiple architectural traditions.

Architectural Lineage

Sampo's design draws from four distinct sources, each contributing specific elements:

From RISC-V

RISC-V's influence is most visible in Sampo's register conventions:

• Zero register (R0): A register that always reads as zero and ignores writes. This eliminates the need for separate "clear" or "load zero" instructions—ADD R4, R0, R0 clears R4, ADD R4, R5, R0 copies R5 to R4.
• Register naming conventions: Return address (RA), stack pointer (SP), global pointer (GP), argument registers (A0-A3), temporaries (T0-T3), and saved registers (S0-S3).
• Load/store architecture: Only load and store instructions access memory; all computation occurs between registers.

From MIPS

MIPS contributed Sampo's approach to instruction encoding:

• Simple, orthogonal formats: A small number of instruction formats (R, I, S, B, J) with consistent field positions
• 4-bit primary opcode: Sixteen instruction categories, each with function codes for variants
• PC-relative branching: Branch targets specified as signed offsets from the program counter

From ARM Thumb/Thumb-2

ARM's Thumb instruction set inspired Sampo's hybrid encoding strategy:

• 16-bit base instruction width: Most common operations fit in 16 bits for improved code density
• 32-bit extended forms: Operations requiring larger immediates use a two-word format
• Prefix-based extension: The 0xF opcode prefix indicates a 32-bit instruction, simplifying decode

From the Z80

The Z80 provides Sampo's "personality"—the features that make it feel familiar to retrocomputing enthusiasts:

• Port-based I/O: IN and OUT instructions with 8-bit port addresses, separate from the memory address space
• Alternate register set: The EXX instruction swaps working registers with shadow copies for fast interrupt handling
• Block operations: LDIR, LDDR, FILL, and CPIR for efficient memory manipulation
• BCD support: The DAA (Decimal Adjust Accumulator) instruction for binary-coded decimal arithmetic
• 64KB address space: 16-bit addresses, matching the Z80's memory model

The Register File

Sampo provides 16 general-purpose 16-bit registers, organized with RISC-V-style conventions:

The zero register deserves special attention. Having a register that always contains zero eliminates entire classes of instructions found in other architectures:

• MOV Rd, Rs becomes ADD Rd, Rs, R0
• CLR Rd becomes ADD Rd, R0, R0
• NEG Rd, Rs can use R0 as the implicit minuend
• CMP Rs, #0 becomes SUB R0, Rs, R0 (result discarded, flags set)

This technique, pioneered by MIPS and refined by RISC-V, dramatically simplifies the instruction set while maintaining expressiveness.

Alternate Registers

Unlike the Z80, which swaps all main registers with EXX, Sampo is selective. Only registers R4-R11 (the arguments and temporaries) have shadow copies. The critical system registers—R0 (zero), R1 (return address), R2 (stack pointer), R3 (global pointer), and R12-R15 (saved registers)—are never swapped.

This design decision serves interrupt handling. When an interrupt occurs, the handler can execute EXX to gain a fresh set of working registers without corrupting the interrupted code's arguments or temporaries. The stack pointer remains valid (no need to establish a new stack), and the return address register can be used to save the interrupted PC.

irq_handler:
    EXX                     ; Swap to alternate R4-R11
    ; ... handle interrupt using R4'-R11' ...
    ; Primary registers preserved automatically
    EXX                     ; Swap back
    RETI                    ; Return from interrupt

The Flags Register

Sampo uses an 8-bit flags register with six defined flags:

The N, Z, C, and V flags follow standard conventions and support the full range of conditional branches. The H (half-carry) flag exists specifically for the DAA instruction, enabling correct BCD arithmetic. The I flag controls interrupt recognition.

Notably, Sampo provides explicit GETF and SETF instructions to read and write the flags register, unlike many RISC architectures that treat flags as implicit state. This supports context switching and debugging.

Memory Model

Sampo uses a straightforward memory model:

• Address space: 64KB (16-bit addresses)
• Byte-addressable: Individual bytes can be loaded and stored
• Little-endian: Multi-byte values stored with LSB at lower address
• Word alignment: 16-bit words should be aligned on even addresses (optional enforcement)

A suggested memory map divides the 64KB space:

0x0000-0x00FF   Interrupt vectors / Reset
0x0100-0x7FFF   Program ROM (~32KB)
0x8000-0xFEFF   RAM (~32KB)
0xFF00-0xFFFF   Memory-mapped I/O (256 bytes)

This layout provides a clean separation between code, data, and I/O while leaving room for customization. The interrupt vector area at the bottom of memory follows Z80 conventions, with the reset vector at 0x0000 and interrupt vector at 0x0004.

Port-Based I/O

In addition to memory, Sampo provides a separate 256-port I/O address space accessed via IN and OUT instructions. This design directly mirrors the Z80 and enables straightforward porting of code that interacts with serial ports, disk controllers, sound chips, and other peripherals.

The I/O instructions come in two forms:

INI  R4, 0x80       ; Read from port 0x80 (immediate port number)
IN   R4, (R5)       ; Read from port specified in R5
OUTI 0x81, R4       ; Write R4 to port 0x81 (immediate)
OUT  (R5), R4       ; Write R4 to port specified in R5

Extended 32-bit forms (INX, OUTX) allow the full 8-bit port range to be specified in immediate form.

Instruction Encoding

Sampo uses a clean, regular encoding scheme with 16-bit base instructions and 32-bit extended forms. The 4-bit primary opcode in bits 15:12 determines the instruction category:

Instruction Formats

Six formats cover all instruction types:

Format R (Register-Register):

15       12 11     8 7      4 3      0
+----------+--------+--------+--------+
|  opcode  |   Rd   |  Rs1   |  Rs2   |
+----------+--------+--------+--------+

Used for three-register operations like ADD R4, R5, R6.

Format I (Immediate):

15       12 11     8 7                0
+----------+--------+------------------+
|  opcode  |   Rd   |      imm8        |
+----------+--------+------------------+

Used for operations with 8-bit immediates like ADDI R4, 42.

Format S (Store):

15       12 11     8 7      4 3      0
+----------+--------+--------+--------+
|  opcode  |  imm4  |  Rs1   |  Rs2   |
+----------+--------+--------+--------+

Used for stores where the destination register field holds an offset.

Format B (Branch):

15       12 11     8 7                0
+----------+--------+------------------+
|  opcode  |  cond  |     offset8      |
+----------+--------+------------------+

Used for conditional branches with PC-relative offsets.

Format J (Jump):

15       12 11                       0
+----------+--------------------------+
|  opcode  |        offset12          |
+----------+--------------------------+

Used for unconditional jumps with 12-bit PC-relative offsets.

Format X (Extended):

Word 0:
15       12 11     8 7      4 3      0
+----------+--------+--------+--------+
|   0xF    |   Rd   |  Rs1   |  sub   |
+----------+--------+--------+--------+

Word 1:
15                                   0
+-------------------------------------+
|              imm16                  |
+-------------------------------------+

Used for operations requiring 16-bit immediates or absolute addresses.

Encoding Examples

To illustrate the encoding scheme, let's examine several instructions:

ADD R4, R5, R6 (R4 = R5 + R6):

Opcode = 0x0, Rd = 4, Rs1 = 5, Rs2 = 6
Binary: 0000 0100 0101 0110 = 0x0456

ADDI R4, 10 (R4 = R4 + 10):

Opcode = 0x5, Rd = 4, imm8 = 10
Binary: 0101 0100 0000 1010 = 0x540A

BEQ +8 (branch forward 8 bytes if equal):

Opcode = 0x8, cond = 0 (BEQ), offset = 4 words
Binary: 1000 0000 0000 0100 = 0x8004

LIX R4, 0x1234 (load 16-bit immediate):

Word 0: 0xF (extended), Rd = 4, Rs = 0, sub = 7 (LIX)
Word 1: 0x1234
Binary: 1111 0100 0000 0111 0001 0010 0011 0100 = 0xF407 0x1234

The regularity of this encoding makes instruction decode straightforward—the first nibble determines the instruction category, and subsequent fields are in consistent positions across formats.

The Instruction Set

Sampo provides approximately 66 distinct instructions, organized into ten categories.

Arithmetic (15 instructions)

The arithmetic category includes standard operations (ADD, SUB, ADDI) plus multiply/divide support:

• MUL: 16×16 multiplication, low 16 bits of result
• MULH/MULHU: High 16 bits of 32-bit product (signed/unsigned)
• DIV/DIVU: Integer division (signed/unsigned)
• REM/REMU: Remainder (signed/unsigned)
• DAA: Decimal adjust for BCD arithmetic
• NEG: Two's complement negation
• CMP: Compare (subtract without storing result)

Hardware multiply and divide are essential for interpreter performance—dividing a 32-bit value by 10 for number formatting would be prohibitively slow without hardware support.

Logic (6 instructions)

Standard bitwise operations: AND, OR, XOR, NOT, plus immediate forms ANDI and ORI.

Shift and Rotate (16 variants)

Sampo provides an unusually rich set of shift operations:

• SLL/SRL/SRA: Shift left/right logical/arithmetic
• ROL/ROR: Rotate left/right
• RCL/RCR: Rotate through carry (17-bit rotation)
• SWAP: Swap high and low bytes

Each shift type comes in three shift amounts: 1, 4, and 8 bits. The 4-bit shift is particularly useful for hexadecimal digit extraction and insertion. Variable shifts use the extended format with the shift amount in the second register or immediate field.

Load/Store (6 instructions)

Memory access instructions include word and byte loads (with sign or zero extension), word and byte stores, and LUI (Load Upper Immediate) for constructing 16-bit constants:

LUI  R4, 0x12       ; R4 = 0x1200
ORI  R4, R4, 0x34   ; R4 = 0x1234

Branch (16 conditions)

Sampo supports a comprehensive set of branch conditions:

• BEQ/BNE: Equal/not equal
• BLT/BGE/BGT/BLE: Signed comparisons
• BLTU/BGEU/BHI/BLS: Unsigned comparisons
• BMI/BPL: Negative/positive
• BVS/BVC: Overflow set/clear
• BCS/BCC: Carry set/clear

This covers all reasonable comparison outcomes for both signed and unsigned arithmetic.

Jump/Call (4 instructions)

• J: PC-relative unconditional jump
• JAL: Jump and link (save return address in RA)
• JR: Jump to address in register
• JALR: Jump and link to register address

Block Operations (6 instructions)

The block operations use a fixed register convention (R4=count, R5=source, R6=destination):

• LDI/LDD: Load single byte, increment/decrement pointers and count
• LDIR/LDDR: Repeat until count reaches zero
• FILL: Fill memory region with value
• CPIR: Compare and search forward

These instructions are decidedly un-RISC—they're multi-cycle operations that modify multiple registers. But they're implemented with predictable behavior (always the same registers, always the same algorithm) and provide enormous speedups for common memory operations.

Stack (4 instructions)

• PUSH/POP: Single register push/pop
• PUSHM/POPM: Push/pop multiple registers (via bitmask)

I/O (4 instructions)

• INI/OUTI: Immediate port address
• IN/OUT: Register port address

System (9 instructions)

• NOP: No operation
• HALT: Stop processor
• DI/EI: Disable/enable interrupts
• EXX: Exchange alternate registers
• RETI: Return from interrupt
• SWI: Software interrupt
• SCF/CCF: Set/complement carry flag
• GETF/SETF: Read/write flags register

Comparison with Other Architectures

To put Sampo in context, consider how it compares with related processors:

Sampo sits in an interesting position: more regular than the Z80 but with Z80-friendly features, smaller and simpler than 32-bit RISC but still cleanly orthogonal.

Code Examples

To demonstrate how Sampo assembly looks in practice, here's a "Hello World" program that outputs text via a serial port:

        .org 0x0100

.equ    ACIA_STATUS 0x80
.equ    ACIA_DATA   0x81
.equ    TX_READY    0x02

start:
        LIX  R4, message        ; Load address of string

loop:
        LBU  R5, (R4)           ; Load byte from string
        CMP  R5, R0             ; Compare with zero
        BEQ  done               ; If null terminator, done

wait_tx:
        INI  R6, ACIA_STATUS    ; Read serial status port
        ANDI R6, R6, TX_READY   ; Check transmit ready bit
        BEQ  wait_tx            ; Wait if not ready

        OUTI ACIA_DATA, R5      ; Write character to data port
        ADDI R4, 1              ; Next character
        J    loop
done:
        HALT

message:
        .asciz "Hello, Sampo!\n"

And here's a Fibonacci function demonstrating the calling convention:

; fib(n) - compute nth Fibonacci number
; Input: R4 (A0) = n
; Output: R4 (A0) = fib(n)

fib:
        ADDI R5, R0, 0      ; a = 0
        ADDI R6, R0, 1      ; b = 1
        CMP  R4, R0
        BEQ  fib_done

fib_loop:
        ADD  R7, R5, R6     ; temp = a + b
        ADD  R5, R6, R0     ; a = b
        ADD  R6, R7, R0     ; b = temp
        ADDI R4, R4, -1     ; n--
        BNE  fib_loop

fib_done:
        ADD  R4, R5, R0     ; return a
        JR   RA

The code reads naturally to anyone familiar with RISC assembly, while the I/O instructions and register conventions provide the Z80-like feel that makes porting classic software straightforward.

Looking Ahead: FPGA Implementation

With the architecture defined, the next step is implementation. In Part 2 of this series, we'll build a working Sampo processor using Amaranth HDL, a modern Python-based hardware description language. We'll cover:

• The ALU module: Implementing all arithmetic and logic operations
• The register file: Including the alternate register set and zero register
• The instruction decoder: Parsing the various instruction formats
• The control unit: Managing the fetch-decode-execute cycle
• The memory interface: Connecting to block RAM
• The I/O subsystem: Implementing the port-based I/O model
• Integration: Putting it all together into a working system-on-chip

We'll synthesize the design for an affordable FPGA board and run actual Sampo programs, demonstrating that this architecture isn't just a paper exercise but a real, working processor.

The Sampo project on GitHub includes a complete Rust-based assembler (sasm) and emulator (semu) with a TUI debugger, so you can start writing and testing Sampo programs today. The FPGA implementation will let you run those same programs on real hardware, completing the journey from mythological artifact to silicon reality.

Stay tuned for Part 2, where we'll forge our own Sampo—not from swan feathers and barley, but from lookup tables and flip-flops.

I was born in the back half of 1980, which means I missed the revolution.

By the time I sat down at an Apple IIe in 1986, with its green phosphor glow and chunky 5.25-inch floppies, the war was already over. The Altair 8800 was a museum piece. CP/M was fading into obscurity. The TRS-80 and Commodore PET were yesterday's news. I arrived just in time for the Apple II's twilight years, blissfully unaware that the machine in front of me represented the victory lap of a decade-long transformation.

I never experienced CP/M. I never loaded WordStar from an 8-inch floppy. I never watched VisiCalc recalculate a spreadsheet and felt the shock of a machine doing in seconds what had taken hours by hand. These were foundational moments in computing history, and I missed them entirely.

Now, decades later, I find myself building Z80 emulators and writing compilers for a processor that had already ceded the PC spotlight by the time I could read — though it quietly lived on in the TI graphing calculators that would later get me through high school math, and still powers them today. It's a form of technological archaeology — reconstructing a world I never lived in, trying to understand the texture of an era I only know through documentation and nostalgia. And from this vantage point, watching the current panic over artificial intelligence, I can't help but notice something: we've been here before.

As 2025 quickly fades. ChatGPT writes code. Midjourney creates art. Claude analyzes documents. The headlines scream that knowledge workers are doomed, that white-collar jobs will evaporate, that "this time it's different."

But it's not different. It's never different. And VisiCalc can prove it.

The World Before the Spreadsheet

To understand why VisiCalc mattered, you need to understand what "spreadsheet" meant in 1978. It wasn't software. It was paper.

Accountants, analysts, and financial planners worked with literal sheets of paper, ruled into rows and columns. They called them spreadsheets because the worksheets spread across multiple pages, sometimes taped together into unwieldy grids that covered entire desks. Every number was written by hand. Every calculation was performed with a mechanical adding machine or, if you were modern, an electronic calculator.

Here's what financial planning looked like: You'd spend hours, maybe days, building a projection. Revenue assumptions in one column, cost structures in another, profit margins calculated cell by cell. Then your boss would ask a simple question: "What if we increase prices by 5%?"

And you'd start over.

Not from the pricing cell — from every cell that pricing touched. The cascade of recalculations could take hours. A complex model might require a full day to revise. And if you made an error somewhere in the middle? Good luck finding it in a forest of pencil marks and eraser smudges.

Word processing was no better. Before WordStar and its competitors, documents were produced on typewriters. The IBM Selectric was the gold standard — a marvel of engineering that let you swap font balls and correct single characters with lift-off tape. But if you found a typo on page 47 of a 60-page contract, you had options: live with it, or retype pages 47 through 60.

Typing was a specialized profession. Companies maintained typing pools — rooms full of secretaries whose primary job was converting handwritten drafts and dictation into finished documents. A skilled typist was a valuable employee precisely because the work was so labor-intensive.

And if you needed computing power for serious analysis, you went to the mainframe. You submitted your job to the MIS department, waited in a queue, and paid by the CPU-minute. Time-sharing systems charged hundreds of dollars per hour. Computing was a scarce resource, rationed by bureaucracy.

This was knowledge work in the mid-1970s: manual, slow, expensive, and error-prone.

The Revolution No One Expected

Dan Bricklin was a Harvard MBA student in 1978 when he had the insight that would change everything. Sitting in a classroom, he watched a professor work through a financial model on a blackboard. The professor would write numbers, perform calculations, and fill in cells. Then he'd change an assumption, and the recalculation cascade would begin — erasing, recomputing, rewriting, sometimes running out of blackboard space.

Bricklin's thought was simple: what if the blackboard could recalculate itself?

Working with programmer Bob Frankston, Bricklin built VisiCalc — the "visible calculator." It ran on the Apple II, which was itself a hobbyist curiosity, a machine that enthusiasts bought to tinker with BASIC programs and play primitive games. VisiCalc transformed it into a business tool.

The software shipped in 1979, priced at \$100. Within a year, it was selling 12,000 copies per month. More importantly, it was selling Apple IIs. The \$2,000 computer became justifiable as a business expense because VisiCalc made it productive.

Consider the economics. A financial analyst in 1980 earned perhaps \$25,000 per year. A secretary earned \$12,000 to \$15,000. The Apple II plus VisiCalc cost roughly \$2,500. If the software saved a few weeks of analyst time, or let one analyst do the work that had previously required two, it paid for itself almost immediately.

But the real magic wasn't cost savings — it was capability. Suddenly you could ask "what if?" as many times as you wanted. Change an assumption, watch the spreadsheet ripple with recalculations, and see the answer in seconds. Financial modeling went from a laborious exercise in arithmetic to an exploratory conversation with your data.

WordStar, released a year earlier in 1978, performed the same transformation for documents. Write, edit, revise, move paragraphs, fix typos — all before committing anything to paper. The document existed as a malleable thing, not a fixed artifact produced through irreversible mechanical action.

Together, these applications (and others like dBASE for databases and SuperCalc as a VisiCalc competitor) created the productivity software category. They didn't sell computers to hobbyists; they sold computers to businesses. And they did it by solving mundane problems: arithmetic and typing.

The pundits of the era made predictions. Accountants would become obsolete. Secretaries would be eliminated. The typing pool would vanish. Knowledge work itself was being automated.

What Actually Happened

The predictions were wrong. Or rather, they were right about the transformation but wrong about the outcome.

Typing pools did shrink. The specialized profession of "typist" largely disappeared as word processing became a universal skill. But administrative assistants didn't vanish — their job changed. Instead of spending hours producing documents, they spent hours managing calendars, coordinating logistics, and handling communication. The mechanical work evaporated; the judgment work remained.

Bookkeepers declined as a profession. The person whose job was to maintain ledgers and perform routine calculations found that job automated. But accountants — the people who interpreted the numbers, made recommendations, and exercised judgment — grew in number. The Bureau of Labor Statistics shows steady growth in accounting employment through the 1980s and 1990s, even as the basic arithmetic of accounting was completely automated.

Financial analysts became more valuable, not less. The spreadsheet didn't replace them; it amplified them. An analyst who could build sophisticated models in VisiCalc or Lotus 1-2-3 was worth more than one limited to paper. The ceiling rose.

And here's the crucial point: the total amount of analysis, documentation, and financial modeling exploded. When something becomes cheaper and faster to produce, you produce more of it. Companies that had operated with crude annual budgets started building detailed monthly projections. Reports that had been quarterly became weekly. The volume of knowledge work grew to fill the new capacity.

This pattern — automation making workers more productive, which increases demand for the work, which maintains or increases employment — has a name in economics. It's called the Jevons paradox, originally observed in coal consumption: as steam engines became more efficient, total coal usage increased rather than decreased, because efficiency made steam power economical for more applications.

The same paradox applies to labor. Make an accountant 10x more productive, and you don't need 1/10th as many accountants. You do 10x as much accounting.

The Pattern Repeats

VisiCalc wasn't the first technology to trigger predictions of labor displacement, and it certainly wasn't the last. The pattern repeats with remarkable consistency:

ATMs (1970s-present): Automated Teller Machines were supposed to eliminate bank tellers. The math seemed obvious — why pay a human to dispense cash when a machine could do it? Yet U.S. bank teller employment roughly doubled between 1970 and 2010. The explanation: ATMs made bank branches cheaper to operate, so banks opened more branches, each requiring fewer but still some tellers. And the tellers' jobs shifted from cash handling to sales, complex transactions, and customer relationships.

CAD Software (1980s): Computer-aided design was going to eliminate draftsmen. Instead, it eliminated hand drafting while increasing demand for designers. The ability to iterate quickly, produce more alternatives, and handle more complex designs meant more design work overall.

Desktop Publishing (1980s): PageMaker and QuarkXPress would kill graphic designers by letting anyone create professional documents. Instead, the volume of designed materials exploded, and graphic design became a larger profession. The average quality rose because the floor rose.

Legal Research Databases (1990s): LexisNexis and Westlaw would eliminate paralegals by automating case research. Instead, faster research enabled more litigation, more thorough preparation, and more legal work overall.

Electronic Trading (1990s-2000s): Algorithmic trading would eliminate floor traders and financial professionals. It did eliminate floor traders, but the financial sector's employment grew as new roles emerged: quants, algorithm developers, risk managers, compliance officers.

In every case, the predictions followed the same logic: Technology X automates task Y, therefore workers who do Y are obsolete. And in every case, the predictions missed the second-order effects: automation makes the overall activity more valuable, demand increases, and workers shift to higher-judgment versions of the same work.

The AI Moment

Which brings us to now.

ChatGPT was released in November 2022. Within two months, it had 100 million users. Within a year, AI assistants were embedded in products from Microsoft to Google to Adobe. Large language models could write essays, generate code, summarize documents, answer questions, and produce content that was — on first glance — indistinguishable from human output.

The predictions arrived immediately. Programmers would become obsolete. Writers were doomed. Customer service, legal research, financial analysis, medical diagnosis — all would be automated. Goldman Sachs estimated 300 million jobs would be affected. The World Economic Forum issued reports. Thought leaders proclaimed that "this time it's different."

But is it?

Let's apply the VisiCalc framework. What exactly does AI automate?

First drafts, not final judgment. AI can produce a draft document, a code snippet, an analysis outline. What it cannot do is determine whether that draft serves the actual goal, handles the edge cases that matter, or fits the political context of the organization. The human reviews, revises, and takes responsibility.

Pattern matching, not pattern breaking. Large language models are, at their core, sophisticated pattern matchers trained on existing text. They excel at producing outputs that look like their training data. They struggle with genuine novelty — situations unlike anything in the training corpus, problems that require inventing new approaches rather than recombining old ones.

The middle of the distribution, not the edges. AI handles routine cases well. It struggles with outliers. The customer service bot can resolve common issues; the unusual complaint needs a human. The coding assistant can generate boilerplate; the architectural decision requires judgment.

Production, not accountability. AI can produce outputs, but it cannot be held accountable for them. When the document goes to the client, someone signs it. When the code ships to production, someone owns it. When the decision has consequences, someone faces them. That someone is human, because accountability requires agency, and agency requires humanity.

This is exactly the pattern we saw with spreadsheets. VisiCalc automated arithmetic, not judgment. It automated production, not accountability. It handled the routine middle, not the novel edges. And the humans who learned to use it became more valuable, not less.

The Irreducible Human

Why do humans remain in the loop? Not for sentimental reasons. Not because we want to preserve jobs. But because certain functions cannot be automated, regardless of how sophisticated the technology.

Accountability requires agency. When something goes wrong, someone must be responsible. Legal systems, regulatory frameworks, and social structures all assume a responsible party. AI systems can produce outputs, but they cannot be sued, fired, jailed, or shamed. The human who relies on AI output remains accountable for that output. This isn't a bug; it's a feature of how human society functions.

Context is infinite and local. AI models are trained on general patterns. Your specific situation — your company's politics, your client's unspoken concerns, your industry's unwritten rules — is not in the training data. The model knows what words typically follow other words. It doesn't know that your CFO hates bullet points, that your customer is going through a divorce, or that mentioning the competitor's product is forbidden in this meeting. The human provides context.

Trust requires relationship. Business transactions ultimately rest on trust between humans. You hire the lawyer, not the legal database. You trust your doctor, not the diagnostic algorithm. You buy from salespeople, not recommendation engines. AI can support these relationships, but it cannot replace them, because trust is a human phenomenon.

The feedback loop requires humans. Here's a subtle but critical point: AI systems are trained on human-generated data. If humans stop producing original work, the training data stops improving. The model learns to produce outputs that look like human outputs because it was trained on human outputs. Remove the humans, and you get a system trained on its own outputs — a recursive degradation. We are the curriculum.

Novel situations require genuine understanding. AI excels at interpolation — finding patterns within the space of its training data. It struggles with extrapolation — handling situations outside that space. Genuine novelty, by definition, lies outside the training distribution. The unprecedented situation, the black swan event, the "we've never seen this before" moment — these require human judgment, because no pattern matching can help when there's no pattern to match.

The Reskilling Reality

None of this means AI changes nothing. It changes a lot. The question is what kind of change.

When spreadsheets arrived, certain skills became less valuable. Manual arithmetic, once essential for financial work, became irrelevant. The ability to maintain error-free ledgers through careful penmanship mattered less. Slide rule proficiency joined buggy whip maintenance in the museum of obsolete competencies.

But new skills became essential. Building spreadsheet models, understanding the logic of cell references, knowing how to structure data for analysis — these became core professional competencies. "Computer literacy" emerged as a job requirement. People who learned the new tools thrived; people who refused to adapt struggled.

AI is triggering the same shift. Consider what becomes less valuable:

Writing first drafts from scratch. When AI can produce a competent first draft in seconds, the ability to stare at a blank page and produce prose is less differentiating. The value shifts to editing, directing, and refining.

Routine research and compilation. When AI can summarize documents, extract key points, and synthesize information, the human who only does that work has a problem. The value shifts to evaluating sources, asking the right questions, and interpreting results.

Basic code production. When AI can generate boilerplate, implement standard patterns, and translate requirements into code, the programmer whose main skill is typing syntax is in trouble. The value shifts to architecture, debugging, code review, and understanding what the system should do.

And consider what becomes more valuable:

Judgment and curation. AI produces. Humans evaluate. The ability to look at AI output and quickly determine what's useful, what's wrong, and what's missing becomes essential. This is editing in the broadest sense — not just fixing typos, but directing the creative process.

Domain expertise plus AI fluency. The accountant who understands both accounting and how to leverage AI tools is more valuable than either an accountant who ignores AI or an AI operator who doesn't understand accounting. The combination is the new competency.

Handling exceptions and edge cases. As AI handles the routine middle, humans focus on the exceptions. The unusual customer complaint, the novel legal situation, the unprecedented technical problem — these become the human domain. Expertise in handling weirdness becomes more valuable.

Relationship and trust building. As transactional work becomes automated, relationship work becomes relatively more important. The human who can build trust, navigate politics, and close deals face-to-face has a durable advantage.

This is exactly what happened with spreadsheets. The value shifted from arithmetic to analysis, from production to judgment, from routine to exception. The workers who adapted thrived. The workers who clung to obsolete methods struggled.

The Transition Is Never Painless

I don't want to minimize the disruption. Real people, with real skills, face real challenges when technology shifts beneath them.

The typing pool secretary in 1985 had spent years developing speed and accuracy on the Selectric. She could type 80 words per minute with minimal errors. She knew the quirks of carbon paper, the rhythm of the carriage return, the muscle memory of the key layout. These skills, honed over a decade, became worthless in the span of a few years.

Some of those secretaries learned WordPerfect and became administrative assistants. Some moved into other roles entirely. Some struggled, unable or unwilling to adapt, and found themselves squeezed out of the workforce. The aggregate statistics — employment levels, productivity growth, economic expansion — hide individual stories of dislocation and difficulty.

The same will be true of AI. Some knowledge workers will adapt smoothly, integrating AI tools into their workflow and becoming more productive. Some will resist, clinging to methods that worked in 2020 but feel increasingly obsolete by 2030. Some will find themselves displaced, their particular bundle of skills suddenly less valuable in a market that's moved on.

The historical pattern tells us that the net outcome is positive — that technological transitions create more opportunity than they destroy, that the economy adjusts, that new roles emerge. But history is cold comfort to the individual caught in the transition. The typewriter repairman didn't care that computer technicians were a growing field. He cared that his skills were worthless.

This is why the reskilling conversation matters. Not because AI will eliminate all jobs — it won't — but because the specific jobs, the specific skills, the specific ways of working will change. And navigating that change requires awareness, adaptability, and often institutional support.

The workers who thrived through the spreadsheet revolution weren't necessarily the most skilled at the old methods. They were the ones who recognized the shift and moved with it. The accountant who embraced Lotus 1-2-3, even if she was mediocre at mental arithmetic, outcompeted the brilliant human calculator who refused to touch a keyboard.

The same pattern is emerging now. The programmer who integrates AI assistance, even if she's not the fastest typist, will outcompete the keyboard wizard who insists on writing every character manually. The writer who uses AI for drafts and focuses on editing and judgment will outcompete the prose stylist who spends hours on first drafts. The analyst who lets AI handle data compilation and focuses on interpretation will outcompete the Excel jockey who takes pride in manual formula construction.

Adaptation isn't optional. It wasn't optional in 1980, and it isn't optional now.

The Long View

I spend my weekends building emulators for 50-year-old processors. I write compilers that target the Z80, a chip that was designed when Gerald Ford was president. I run BASIC and FORTH on simulated hardware, watching instructions execute that were first written when disco was young.

From this perspective, the current AI moment looks familiar. Technology extends human capability. It always has. The accountant with VisiCalc wasn't replaced; she was amplified. The writer with WordStar wasn't obsolete; he was leveraged. The analyst with a spreadsheet could do in hours what had taken days, and that made analysis more valuable, not less.

When I run my Z80 emulator — JavaScript interpreting WebAssembly interpreting 1976 machine code — I'm witnessing layers of abstraction that would have seemed like science fiction to the engineers who designed the original chip. But the fundamental relationship remains: humans using tools to extend their capabilities.

The nature of work changes. It always changes. The bookkeeper becomes the accountant. The typist becomes the administrative assistant. The draftsman becomes the designer. The job titles shift, the tools evolve, the skills required transform. But the need for human judgment, human accountability, human creativity, and human relationships remains.

This isn't optimism. It's pattern recognition. The 45-year pattern from VisiCalc to ChatGPT is consistent: technology that automates tasks changes the nature of work without eliminating the need for workers. The "this time it's different" predictions have been wrong every time, not because technology isn't powerful, but because the predictions misunderstand the relationship between automation and human labor.

The spreadsheet didn't eliminate the need for human intelligence. It made human intelligence more valuable by freeing it from arithmetic. AI won't eliminate the need for human judgment. It will make human judgment more valuable by freeing it from production.

We've been here before. And we'll be here again, decades from now, when some new technology triggers the same predictions, and historians look back at our AI panic the way we look back at the VisiCalc panic — as an understandable overreaction that missed the larger pattern.

The work changes. The workers adapt. The need for humans persists.

It's not different this time. It never is.

This is the story of attempting something probably inadvisable: compiling Rust for the Zilog Z80, an 8-bit processor from 1976. It's also a story about using AI as a genuine collaborator on deep systems programming work, and what happens when modern software abstractions collide with hardware constraints from an era when 64 kilobytes was considered generous.

Transparency: Claude Code as Collaborator

I want to be upfront about something: significant portions of this compiler backend were developed in collaboration with Claude Code, Anthropic's AI coding assistant. This isn't a case of "AI wrote the code and I took credit" — it's more nuanced than that. Claude served as an unusually patient pair programmer who happens to have read every LLVM tutorial ever written.

Here's what that collaboration actually looked like:

I would describe a problem: "The instruction selector is failing with cannot select: G_SADDO for signed addition with overflow detection." Claude would analyze the GlobalISel pipeline, identify that the Z80's ADC instruction sets the P/V flag for signed overflow, and propose an implementation. I would review, test, discover edge cases, and we'd iterate.

The debugging sessions were particularly valuable. When compilation hung for seven hours on what should have been a two-minute build, Claude helped trace the issue to an accidental infinite recursion — a replace_all refactoring had changed RBI.constrainGenericRegister(...) to constrainOrSetRegClass(...) inside the constrainOrSetRegClass helper function itself. The function was calling itself forever. Finding that bug manually would have taken hours of printf debugging; with Claude analyzing the code structure, we found it in minutes.

This is what AI-assisted development actually looks like in 2025: not magic code generation, but accelerated iteration with a collaborator who never gets frustrated when you ask "wait, explain register allocation to me again."

Why Z80? Why Rust?

The Z80 powered the TRS-80, ZX Spectrum, MSX computers, and countless embedded systems. It's still manufactured today — you can buy new Z80 chips. I actually did just that, I bought a handful of vintage ceramic Z80 chips off of eBay. There's something appealing about running modern language constructs on hardware designed when ABBA topped the charts.

More practically, I've been building Z80-based projects on the RetroShield platform, which lets you run vintage processors on Arduino-compatible hardware. Having a modern compiler toolchain opens possibilities that hand-written assembly doesn't.

But Rust specifically? Rust's ownership model and zero-cost abstractions are theoretically perfect for resource-constrained systems. The language was designed for systems programming. The question is whether "systems" can stretch back 50 years.

Building LLVM for the Z80

The first step was getting LLVM itself to build with Z80 support. This meant:

1. Adding Z80 to the list of supported targets in the build system
2. Creating the target description files (registers, instruction formats, calling conventions)
3. Implementing the GlobalISel pipeline components
4. Wiring everything together so llc -mtriple=z80-unknown-unknown actually works

The target description files alone span thousands of lines. Here's what defining just the basic registers looks like:

def A : Z80Reg<0, "a">;
def B : Z80Reg<1, "b">;
def C : Z80Reg<2, "c">;
def D : Z80Reg<3, "d">;
def E : Z80Reg<4, "e">;
def H : Z80Reg<5, "h">;
def L : Z80Reg<6, "l">;

// 16-bit register pairs
def BC : Z80RegWithSub<7, "bc", [B, C]>;
def DE : Z80RegWithSub<8, "de", [D, E]>;
def HL : Z80RegWithSub<9, "hl", [H, L]>;

Every instruction needs similar treatment. The Z80 has over 700 documented instruction variants when you count all the addressing modes. Not all are needed for a basic backend, but getting basic arithmetic, loads, stores, branches, and calls working required implementing dozens of instruction patterns.

The build process itself was surprisingly manageable — LLVM's build system is well-designed. A complete build with the Z80 target takes about 20 minutes on modern hardware. The iteration cycle during development was typically: change a few files, rebuild (30 seconds to 2 minutes depending on what changed), test with llc, fix, repeat.

The LLVM Approach

LLVM provides a framework for building compiler backends. You describe your target's registers, instruction set, and calling conventions; LLVM handles optimization, instruction selection, and register allocation. In theory, adding a new target is "just" filling in these descriptions.

In practice, LLVM assumes certain things about targets. It assumes you have a reasonable number of general-purpose registers. It assumes arithmetic operations work on values that fit in registers. It assumes function calls follow conventions that modern ABIs have standardized.

The Z80 violates all of these assumptions.

The Register Poverty Problem

The Z80 has seven 8-bit registers: A, B, C, D, E, H, and L. Some can be paired into 16-bit registers: BC, DE, HL. That's it. Modern architectures have 16 or 32 general-purpose registers; the Z80 has seven that aren't even all general-purpose — A is the accumulator with special arithmetic privileges, HL is the primary memory pointer.

LLVM's register allocator expects to juggle many virtual registers across many physical registers. When you have more virtual registers than physical registers, it spills values to memory. On the Z80, you're spilling constantly. Every 32-bit operation requires careful choreography of the few registers available.

Here's what a simple 16-bit addition looks like in our backend:

define i16 @add16(i16 %a, i16 %b) {
  %result = add i16 %a, %b
  ret i16 %result
}

This compiles to:

add16:
    add hl,de
    ret

That's clean because we designed the calling convention to pass arguments in HL and DE. The backend recognizes that the inputs are already where they need to be and emits just the ADD instruction.

But 32-bit addition? That becomes a multi-instruction sequence juggling values through the stack because we can't hold four 16-bit values in registers simultaneously.

The Width Problem

The Z80 is fundamentally an 8-bit processor with 16-bit addressing. Rust's standard library uses usize for indexing, which on most platforms is 32 or 64 bits. The Z80 cannot directly perform 32-bit arithmetic. Every u32 operation expands into multiple 8-bit or 16-bit operations.

Consider multiplication. The Z80 has no multiply instruction at all. To multiply two 16-bit numbers, we emit a call to a runtime library function (__mulhi3) that implements multiplication through shifts and adds. 32-bit multiplication requires calling a function that orchestrates four 16-bit multiplications with proper carry handling.

Division is worse. Iterative division algorithms on 8-bit hardware are slow. Floating-point arithmetic doesn't exist in hardware — every floating-point operation becomes a library call to software implementations.

GlobalISel: The Modern Approach

We're using LLVM's GlobalISel framework rather than the older SelectionDAG. GlobalISel provides finer control over instruction selection through explicit lowering steps:

1. IRTranslator: Converts LLVM IR to generic machine instructions (G_ADD, G_LOAD, etc.)
2. Legalizer: Transforms operations the target can't handle into sequences it can
3. RegBankSelect: Assigns register banks (8-bit vs 16-bit on Z80)
4. InstructionSelector: Converts generic instructions to target-specific instructions

Each step presented challenges. The Legalizer needed custom rules to break 32-bit operations into 16-bit pieces. RegBankSelect needed to understand that some Z80 instructions only work with specific register pairs. The InstructionSelector needed patterns for every Z80 instruction variant.

One particularly tricky issue: LLVM's overflow-detecting arithmetic. Instructions like G_SADDO (signed add with overflow) return both a result and an overflow flag. The Z80's ADC instruction sets the P/V flag on signed overflow, but capturing that flag to a register requires careful instruction sequencing — you can't just read the flag register arbitrarily.

The Bug That Cost Seven Hours

During development, we hit a bug that perfectly illustrates the challenges of compiler work. After implementing a helper function to handle register class assignment, compilation started hanging. Not crashing — hanging. A simple three-function test file that should compile in milliseconds ran for over seven hours before I killed it.

The issue? During a refactoring pass, we used a global search-and-replace to change all calls from RBI.constrainGenericRegister(...) to our new constrainOrSetRegClass(...) helper. But the helper function itself contained a call to RBI.constrainGenericRegister() as its fallback case. The replace-all changed that too:

// Before (correct):
bool constrainOrSetRegClass(Register Reg, ...) {
  if (!MRI.getRegClassOrNull(Reg)) {
    MRI.setRegClass(Reg, &RC);
    return true;
  }
  return RBI.constrainGenericRegister(Reg, RC, MRI);  // Fallback
}

// After (infinite recursion):
bool constrainOrSetRegClass(Register Reg, ...) {
  if (!MRI.getRegClassOrNull(Reg)) {
    MRI.setRegClass(Reg, &RC);
    return true;
  }
  return constrainOrSetRegClass(Reg, RC, MRI);  // Calls itself forever!
}

The function was calling itself instead of the underlying LLVM function. Every attempt to compile anything would recurse until the stack overflowed or the heat death of the universe, whichever came first.

This is the kind of bug that's obvious in hindsight but insidious during development. There were no compiler errors, no warnings, no crashes with helpful stack traces. Just silence as the process spun forever.

Finding it required adding debug output at each step of the instruction selector, rebuilding, and watching where the output stopped. Claude helped immensely here — recognizing the pattern of "output stops here" and immediately checking what that code path did.

The Calling Convention

We designed a Z80-specific calling convention optimized for the hardware's constraints:

• First 16-bit argument: HL register pair
• Second 16-bit argument: DE register pair
• Return value: HL register pair
• Additional arguments: Stack
• Caller-saved: All registers (callee can clobber anything)
• Callee-saved: None

This convention minimizes register shuffling for simple functions. A function taking two 16-bit values and returning one doesn't need any register setup at all — the arguments arrive exactly where the ADD instruction expects them.

For 8-bit arguments, values arrive in the low byte of HL (L register) or DE (E register). This wastes the high byte but simplifies the calling convention.

This is radically different from typical calling conventions. Modern ABIs specify precise preservation rules, stack alignment requirements, and argument passing in specific registers. On the Z80, with so few registers, we had to make pragmatic choices. Every function saves and restores what it needs; there's no concept of "preserved across calls."

A Working Example

Here's LLVM IR that our backend compiles successfully:

target datalayout = "e-m:e-p:16:8-i16:8-i32:8-i64:8-n8:16"
target triple = "z80-unknown-unknown"

define i16 @add16(i16 %a, i16 %b) {
  %result = add i16 %a, %b
  ret i16 %result
}

define i16 @sub16(i16 %a, i16 %b) {
  %result = sub i16 %a, %b
  ret i16 %result
}

define i8 @add8(i8 %a, i8 %b) {
  %result = add i8 %a, %b
  ret i8 %result
}

Compiled output:

    .text
    .globl  add16
add16:
    add hl,de
    ret

    .globl  sub16
sub16:
    and a           ; clear carry
    sbc hl,de
    ret

    .globl  add8
add8:
    ld  c,l
    ld  b,c
    add a,b
    ret

The 16-bit operations are efficient. The 8-bit addition shows the register shuffling required when values aren't in the accumulator — we have to move values through available registers to get them where the ADD instruction expects.

Compilation time for these three functions: 0.01 seconds. The backend works.

Where We Are Now

The backend compiles simple LLVM IR to working Z80 assembly. Integer arithmetic, control flow, function calls, memory access — the fundamentals work. We've implemented handlers for dozens of generic machine instructions and their various edge cases.

Attempting to compile Rust's core library has been... educational. The core library is massive. It includes:

• All the formatting infrastructure (Display, Debug, write! macros)
• Iterator implementations and adaptors
• Option, Result, and their many combinator methods
• Slice operations, sorting algorithms
• Panic handling infrastructure
• Unicode handling

Each of these generates significant code. The formatting system alone probably exceeds the entire memory capacity of a typical Z80 system.

Current status: compilation of core starts, processes thousands of functions, but eventually hits edge cases we haven't handled yet. The most recent error involves register class assignment in the floating-point decimal formatting code — ironic since the Z80 has no floating-point hardware.

Connecting Rust to the Z80 Backend

Getting Rust to use our LLVM backend required modifying the Rust compiler itself. This involved:

1. Adding a target specification: Defining z80-unknown-none-elf in Rust's target database with the appropriate data layout, pointer width, and feature flags.

2. Pointing Rust at our LLVM: Rust can use an external LLVM rather than its bundled version. We configured the build to use our Z80-enabled LLVM.

3. Disabling C compiler-builtins: Rust's standard library includes some C code from compiler-rt for low-level operations. There's no Z80 C compiler readily available, so we had to disable these and rely on pure Rust implementations.

4. Setting panic=abort: The Z80 can't reasonably support stack unwinding for panic handling.

The Rust target specification looks like this:

Target {
    arch: Arch::Z80,
    data_layout: "e-m:e-p:16:8-i16:8-i32:8-i64:8-n8:16".into(),
    llvm_target: "z80-unknown-unknown".into(),
    pointer_width: 16,
    options: TargetOptions {
        c_int_width: 16,
        panic_strategy: PanicStrategy::Abort,
        max_atomic_width: Some(0),  // No atomics
        atomic_cas: false,
        singlethread: true,
        no_builtins: true,  // No C runtime
        ..TargetOptions::default()
    },
}

The pointer_width: 16 is crucial — this is a 16-bit architecture. The max_atomic_width: Some(0) tells Rust that atomic operations aren't available at all, since the Z80 has no atomic instructions.

When Rust tries to compile core, it invokes rustc, which invokes LLVM, which invokes our Z80 backend. Each function in core goes through this pipeline. The sheer volume is staggering — core contains thousands of generic functions that get monomorphized for every type they're used with.

The Honest Assessment

Will Rust's standard library ever practically run on a Z80? Almost certainly not. The core library alone, compiled for Z80, would likely exceed a megabyte — far beyond the 64KB address space. Even if you could page-swap the code, the runtime overhead of software floating-point, 32-bit arithmetic emulation, and iterator abstractions would make execution glacially slow.

What might actually work:

• #![no_std] #![no_core] programs: Bare-metal Rust with a tiny custom runtime, no standard library, hand-optimized for the hardware. A few kilobytes of carefully written Rust that compiles to tight Z80 assembly.

• Code generation experiments: Using the LLVM backend to study how modern language constructs map to constrained hardware, even if the results aren't practical to run.

• Educational purposes: Understanding compiler internals by working with hardware simple enough to reason about completely.

The value isn't in running production Rust on Z80s. It's in the journey — understanding LLVM's internals, grappling with register allocation on a machine that predates the concept (and myself albeit by only a few years), and seeing how far modern tooling can stretch.

Conclusion

Compiling Rust for the Z80 is somewhere between ambitious and absurd. The hardware constraints are genuinely incompatible with modern language expectations. But the attempt has been valuable — understanding LLVM deeply, exploring what "resource-constrained" really means, and discovering that AI collaboration can work effectively on low-level systems programming.

The Z80 was designed for a world where programmers counted bytes. Rust was designed for a world where programmers trust the compiler to manage complexity. Making them meet is an exercise in translation across decades of computing evolution.

There's something deeply satisfying about watching a nearly 50-year-old CPU execute code you just compiled. The Z80 processor, introduced by Zilog in 1976, powered everything from the TRS-80 to the ZX Spectrum to countless CP/M machines. With roughly 8,500 transistors, it's almost incomprehensibly simple by modern standards; a high-end Intel i9 has around 17 billion. Today, thanks to projects like the RetroShield, you can plug one of these vintage processors into an Arduino and run real 8-bit code.

But here's the thing: actually writing Z80 programs is painful. Traditional approaches involve either hand-assembling hex codes, wrestling with decades-old assemblers that barely run on modern systems, or writing raw bytes into binary files. I wanted something better. What if I could write Z80 programs in Rust, using a fluent API that generates correct machine code without the mental overhead of remembering opcode encodings?

The result is the retroshield-z80-workbench, a Rust crate that powers three substantial retro applications: a dBASE II database clone, a WordStar-compatible text editor, and a VisiCalc-style spreadsheet. The workbench emerged from patterns I discovered while building earlier projects like a C compiler and LISP interpreter. This post explains how it works and what it's enabled.

The Problem with Traditional Z80 Development

I first encountered Z80 assembly in the 1990s, writing programs on a TI-85 graphing calculator. The process was painfully tedious: hand-assemble each instruction to hex using a reference card, type the bytes into the calculator's memory editor, run it, watch it crash, and start over. There was no debugger, no error messages, just a frozen screen or a memory clear if you were unlucky. I spent more time looking up opcodes than thinking about algorithms.

Writing Z80 assembly by hand means memorizing hundreds of opcodes. LD A, B is 0x78. JP NZ, addr is 0xC2 followed by a 16-bit address in little-endian format. Conditional returns, indexed addressing, and the various Z80-specific instructions like LDIR and DJNZ all have their own encodings. One wrong byte and your program jumps into garbage.

Traditional assemblers solve this, but they come with their own problems. Many only run under CP/M or DOS. Modern cross-assemblers exist, but they're another tool to install, another syntax to learn, another build step to manage. And when you're generating code programmatically, like when building a compiler that targets Z80, an external assembler becomes a significant complication.

There are also modern C compilers for the Z80, most notably SDCC (Small Device C Compiler), which is actively maintained and produces decent code. But when your goal is to generate Z80 machine code from Rust, perhaps as the backend of a compiler or code generator, you want something that integrates directly into your Rust toolchain.

What I wanted was the ability to write something like this in Rust:

rom.ld_a(0x42);        // LD A, 0x42
rom.call("print_hex"); // CALL print_hex
rom.ret();             // RET

And have it emit the correct bytes: 0x3E 0x42 0xCD xx xx 0xC9.

The Workbench Architecture

The retroshield-z80-workbench crate is built around three core concepts: emit, label, and fixup.

Emit: The Foundation

At the lowest level, everything is just bytes being appended to a buffer:

pub struct CodeGen {
    rom: Vec<u8>,
    labels: HashMap<String, u16>,
    fixups: Vec<(usize, String)>,
    config: RomConfig,
}

impl CodeGen {
    pub fn emit(&mut self, bytes: &[u8]) {
        self.rom.extend_from_slice(bytes);
    }
}

Every Z80 instruction ultimately calls emit(). The ld_a() method is just:

pub fn ld_a(&mut self, n: u8) {
    self.emit(&[0x3E, n]);  // Opcode 0x3E is LD A, n
}

This pattern scales to cover the entire Z80 instruction set. The crate provides over 80 instruction helpers, from simple register loads to complex block transfer instructions.

Labels: Named Positions

Labels mark positions in the code that can be referenced by jumps and calls:

pub fn label(&mut self, name: &str) {
    let addr = self.config.org + self.rom.len() as u16;
    self.labels.insert(name.to_string(), addr);
}

When you write rom.label("main"), the current position gets recorded. Later, when you write rom.jp("main"), the crate knows exactly where to jump.

Fixups: Forward References

The clever part is handling forward references. When you write rom.call("print_string") before print_string is defined, the crate can't know the address yet. Instead, it records a fixup:

pub fn call(&mut self, label: &str) {
    self.emit(&[0xCD]);  // CALL opcode
    self.fixup(label);   // Record that we need to fill in this address
}

pub fn fixup(&mut self, label: &str) {
    self.fixups.push((self.rom.len(), label.to_string()));
    self.emit_word(0x0000);  // Placeholder
}

At the end, resolve_fixups() walks through all recorded fixups and patches in the correct addresses:

pub fn resolve_fixups(&mut self) {
    for (pos, label) in &self.fixups {
        let addr = self.labels.get(label)
            .expect(&format!("Undefined label: {}", label));
        self.rom[*pos] = *addr as u8;
        self.rom[*pos + 1] = (*addr >> 8) as u8;
    }
}

This simple mechanism enables natural code organization where you can reference routines before defining them.

Building Blocks: The Standard Library

Raw instruction emission is powerful but verbose. The workbench includes pre-built routines for common tasks that any Z80 program needs.

Serial I/O

Our modified RetroShield firmware emulates an MC6850 ACIA for serial communication (the official RetroShield uses an Intel 8251). The standard library provides blocking read/write routines:

pub fn emit_getchar(&mut self) {
    self.label("getchar");
    self.in_a(0x80);           // Read status register
    self.and_a(0x01);          // Test RX ready bit
    self.emit(&[0x28, 0xFA]);  // JR Z, -6 (loop until ready)
    self.in_a(0x81);           // Read data register
    self.ret();
}

This generates a 10-byte routine that any program can call with rom.call("getchar"). The character comes back in the A register, exactly as you'd expect from a standard library function.

Similar routines handle putchar, print_string (for null-terminated strings), and newline (CR+LF).

VT100 Terminal Control

Every program I've written needs cursor positioning, screen clearing, and other terminal operations. The standard library includes VT100 escape sequences:

pub fn emit_clear_screen(&mut self) {
    self.label("clear_screen");
    self.ld_hl_label("_cls_seq");
    self.call("print_string");
    self.ret();
}

// Later, in data section:
rom.label("_cls_seq");
rom.emit_string("\x1B[2J\x1B[H");  // ESC[2J ESC[H

The cursor_pos routine is more complex, converting binary row/column values to the ASCII digits that VT100 expects. It's about 50 bytes of Z80 code that no one wants to write more than once.

Math Routines

The Z80 has limited math capabilities, especially for 16-bit operations. The standard library provides:

• print_byte_dec: Convert and print A register as decimal (000-255)
• div16: 16-bit division with remainder
• negate_hl: Two's complement negation

These become critical building blocks for anything involving numbers.

Pseudo-Assembly as Building Blocks

The real power emerges when you combine these primitives into higher-level constructs. Instead of thinking in individual Z80 instructions, you start thinking in chunks of functionality.

Consider implementing a text editor. You need a routine to insert a character at the cursor position. In pseudo-assembly, this is:

1. Get the current line pointer
2. Shift all bytes from cursor to end of buffer right by one
3. Insert the new character
4. Update cursor position
5. Redraw

Each of these steps becomes a Rust method that emits a sequence of Z80 instructions:

fn emit_insert_char(&mut self) {
    self.label("insert_char");

    // Save the character to insert
    self.ld_addr_a(TEMP_A);

    // Get current line pointer
    self.ld_a_addr(CURSOR_ROW);
    self.call("get_line_ptr");  // HL = line start

    // Add cursor column offset
    self.ld_de_addr(CURSOR_COL);
    self.add_hl_de();           // HL = insert position

    // Calculate bytes to shift...
    // (many more instructions)

    // Use LDDR for the actual shift
    self.emit(&[0xED, 0xB8]);   // LDDR

    // Insert the character
    self.ld_a_addr(TEMP_A);
    self.ld_hl_ind_a();

    // Update counters and redraw
    self.call("increment_cursor");
    self.call("draw_current_line");
    self.ret();
}

This method generates about 80 bytes of Z80 machine code. By building up from primitives to routines to complete functions, complex programs become manageable.

Programs Built with the Workbench

The real test of any framework is what you can build with it. Here's what's running on the RetroShield today.

kz80_db: A dBASE II Clone

dBASE II was the database that launched a thousand businesses in the early 1980s. Before SQL became dominant, dBASE gave microcomputer users their first taste of structured data management. My clone implements the authentic 1981 file format: 8-byte headers, 16-byte field descriptors, fixed-length records with delete flags.

The file format is documented in the code itself:

DBF Header (8 bytes):
  Byte 0:    Version (0x02 for dBASE II)
  Bytes 1-2: Number of records (16-bit little-endian)
  Bytes 3-4: Month, Day of last update
  Bytes 5-6: Year of last update
  Byte 7:    Record length (including delete flag)

Field Descriptors (16 bytes each, terminated by 0x0D):
  Bytes 0-10:  Field name (11 bytes, null-padded)
  Byte 11:     Field type (C=Character, N=Numeric, L=Logical)
  Byte 12:     Field length
  Byte 13:     Decimal places (for N type)
  Bytes 14-15: Reserved

The implementation includes:

• CREATE to define new database structures with up to 16 fields
• USE to open existing .DBF files from the SD card
• APPEND to add records interactively
• LIST to display all records in columnar format
• EDIT to modify existing records with field-by-field prompts
• DELETE and PACK for soft-delete and physical removal
• GO TOP/BOTTOM and GO n for record navigation
• DISPLAY STRUCTURE to show field definitions

The generated ROM is about 4KB, fitting comfortably in the RetroShield's 8KB ROM space. It reads and writes real .DBF files that you can open in modern database tools like LibreOffice Calc or even current versions of dBASE.

Building this required implementing a command parser that handles the dot-prompt interface, string comparison routines for command matching, file I/O through the SD card interface with seek operations, and the full dBASE command set. Each command is a Rust method that emits the appropriate Z80 code:

fn emit_list_command(&mut self) {
    self.label("cmd_list");

    // Check if database is open
    self.ld_a_addr(DB_OPEN);
    self.or_a_a();
    self.jp_z("no_db_open");

    // Print column headers from field descriptors
    self.call("print_headers");

    // Loop through all records
    self.ld_hl(1);
    self.ld_addr_hl(CURRENT_REC);

    self.label("list_loop");
    self.call("read_record");
    self.call("print_record");

    // Increment and check against record count
    self.ld_hl_addr(CURRENT_REC);
    self.inc_hl();
    self.ld_addr_hl(CURRENT_REC);
    // ... 150+ more lines
}

The SD card interface deserves special mention. The RetroShield includes an SD card reader accessible through I/O ports. Commands like open, read, write, seek, and close are sent through a command register, with data transferred byte-by-byte through a data register. The workbench makes this tolerable by wrapping the low-level port operations in reusable routines.

kz80_ws: A WordStar Clone

WordStar defined text editing for a generation of writers. George R.R. Martin famously still uses it. The diamond cursor movement (^E ^S ^D ^X arranged like arrow keys on the keyboard), the block operations (^KB ^KK ^KC), the search functions, the word wrap, the careful attention to 80-column displays: all of this became muscle memory for millions of users.

The clone implements:

• Full cursor movement with ^E/^S/^D/^X and ^A/^F for word movement
• Insert and overwrite modes with ^V toggle
• Block operations: mark begin (^KB), mark end (^KK), copy (^KC), delete (^KY)
• File operations: save (^KS), save and exit (^KD), quit without saving (^KQ)
• Search (^QF), word wrap at configurable right margins
• Line operations: delete line (^Y), insert line break (^N)
• Quick movement: top of file (^QR), end of file (^QC), line start/end (^QS/^QD)
• VT100 terminal output with proper status line showing line/column/mode

The memory layout is carefully designed for the 8KB RAM constraint:

RAM (8KB):
  0x2000-0x201F  State variables (cursor, view, margins)
  0x2100-0x21FF  Input buffer
  0x2200-0x22FF  Filename buffer
  0x2800-0x3BFF  Text buffer (5KB)
  0x3C00-0x3DFF  Line index table
  0x3E00-0x3FFF  Stack

The word wrap implementation is particularly satisfying. When the cursor passes the right margin (default column 65), the editor scans backward to find the last space, then uses the Z80's LDDR instruction to shift the buffer and insert a CR/LF pair. The cursor repositions on the new line at exactly the right column to continue typing the wrapped word. All of this happens fast enough that the user just sees smooth text flow.

The screen update strategy matters on a 4MHz processor. Rather than redrawing the entire screen on each keystroke, the editor tracks what changed and only redraws the affected line. The VT100 "clear to end of line" escape sequence handles trailing garbage. This keeps the interface responsive despite the hardware limitations.

kz80_calc: A VisiCalc-Style Spreadsheet

VisiCalc was the "killer app" that made personal computers business tools. Dan Bricklin and Bob Frankston's 1979 creation turned the Apple II from a hobbyist toy into something accountants would buy. My version brings that experience to the Z80:

• 1024 cells (16 columns A-P by 64 rows) in 6KB of RAM
• 8-digit packed BCD arithmetic for accurate decimal math
• Formula support with cell references (A1+B2*C3)
• Operator precedence (* and / before + and -)
• Range functions: @SUM, @AVG, @MIN, @MAX, @COUNT
• Automatic recalculation when cells change
• Arrow key navigation and GOTO command for jumping to cells
• Cell types: numbers, labels, formulas, and repeating characters

The cell storage format uses 6 bytes per cell:

Cell format (6 bytes):
  byte 0:    type (0=empty, 1=number, 2=formula, 3=error, 4=repeat, 5=label)
  byte 1:    sign (0x00=positive, 0x80=negative)
  bytes 2-5: 8-digit packed BCD (d7d6 d5d4 d3d2 d1d0)

The BCD math was the hardest part. Binary floating-point would give wrong answers for financial calculations (the classic 0.1 + 0.2 != 0.3 problem). Packed BCD stores two decimal digits per byte, and the Z80's DAA (Decimal Adjust Accumulator) instruction handles single-byte addition correctly. But building 32-bit multiplication and division from 8-bit DAA takes hundreds of carefully sequenced instructions.

The formula parser handles expressions like =A1+B2*C3-@SUM(D1:D10). This required implementing recursive descent parsing in Z80 machine code, which the workbench made tractable by letting me focus on the algorithm rather than opcode encodings. The parser breaks formulas into tokens, builds a simple AST in memory, and evaluates it with proper operator precedence.

Beyond the Workbench

The workbench proved its value for these three substantial applications. But I've also built other Z80 projects that predate the workbench or use their own code generation approaches:

• kz80_c: A C compiler with its own emit infrastructure, developed before the workbench was extracted as a reusable crate
• kz80_lisp: A LISP interpreter with mark-and-sweep garbage collection
• kz80_prolog: Logic programming with unification and backtracking
• kz80_ml: An ML compiler with Hindley-Milner type inference
• kz80_fortran: FORTRAN77 subset for scientific computing nostalgia
• kz80_lua, kz80_smalltalk, kz80_chip8: Various interpreters and emulators

The experience building these earlier projects is what led to extracting the common patterns into the workbench. The emit/label/fixup pattern appeared independently in several codebases before I recognized it as a reusable abstraction.

Looking back at kz80_c, for instance, I can see the proto-workbench emerging. There's a CodeGen struct with an emit() method, a labels hashmap, and fixup resolution. The same pattern appears in kz80_lisp. Eventually it became clear that this infrastructure should be its own crate, tested once and reused everywhere.

The workbench also benefited from hindsight. Early projects had ad-hoc solutions for things like unique label generation (essential for compiling nested control structures) and relative jump calculation. The workbench handles these correctly from the start, saving debugging time on every subsequent project.

The Hardware: RetroShield Z80

For those unfamiliar with the RetroShield project, it's worth a brief explanation. The RetroShield is an Arduino shield designed by 8BitForce that lets you run real vintage CPUs. You plug an actual Z80 (or 6502, or 6809, or 8085) into a socket on the shield. The Arduino provides clock, reset, and memory by intercepting the CPU's bus signals.

The Z80 variant gives you:

• ROM at 0x0000 (size depends on your binary)
• 6KB RAM at 0x2000-0x37FF
• MC6850 ACIA for serial I/O at ports 0x80-0x81

The original RetroShield Z80 emulated the Intel 8251 USART for serial communication. In 2023, with help from RetroShield creator Erturk Kocalar, I added MC6850 ACIA emulation to run John Hardy's Forth interpreter. The MC6850 is what most CP/M software expects, making it the better choice for running vintage software. The Arduino sketch with MC6850 emulation is available in my RetroShield firmware collection on GitLab.

I added an SD card interface at ports 0x10-0x15, which isn't part of the standard RetroShield but integrates cleanly with the Arduino firmware. This gives the dBASE and WordStar clones persistent file storage.

This constrained environment is actually liberating. You can't reach for a 100MB framework or spawn threads. Every byte matters. The programs you write are complete, self-contained, and comprehensible. The entire WordStar clone is about 4KB of machine code. You can read a hex dump of the ROM and, with patience, trace exactly what every byte does.

The RetroShield connects to an Arduino Mega via two rows of 18 pins, or alternatively to a Teensy 4.1 using a special carrier board. Either way, you interact with your Z80 programs through a terminal emulator over USB serial. The VT100 and VT220 escape sequences that the workbench's terminal routines emit work perfectly in modern terminals like iTerm2 or the venerable screen command, connecting 1970s display protocols to 2020s software.

Why Rust?

Rust brings several advantages to this domain:

Type Safety: The compiler catches mistakes like passing a label where an address is expected, or using the wrong register size. This matters when generating machine code where a single wrong byte corrupts everything.

Zero Runtime: The generated ROMs contain only Z80 code, no runtime, no garbage collector. Rust's abstractions compile away completely.

Excellent Tooling: Cargo handles dependencies, testing, and publishing. The workbench is on crates.io; adding it to a project is one line in Cargo.toml.

Performance: Code generation is fast. Even the complex projects compile in under a second.

Expressiveness: Rust's type system lets me encode Z80 concepts cleanly. A label is a String, an address is a u16, and the compiler keeps them straight.

Lessons Learned

Building the workbench and using it for real projects taught me several things:

Start with the primitives right: The emit/label/fixup core hasn't changed since the first version. Getting the foundation solid paid dividends.

Standard library matters: Having I/O and terminal routines ready to call eliminated boilerplate from every project. I probably use call("print_string") a hundred times across all the projects.

Let the host do the work: Complex string manipulation, parsing, and data structure management happen in Rust on the host computer. The Z80 code just handles the runtime behavior. This split makes everything easier.

Readability over brevity: A Z80 program written in the workbench is longer than the equivalent hand-assembled hex, but it's readable and maintainable. When I need to fix a bug in the WordStar word wrap routine, I can read the Rust code and understand it.

Getting Started

The workbench is available on crates.io:

[dependencies]
retroshield-z80-workbench = "0.1"

A minimal program:

use retroshield_z80_workbench::prelude::*;

fn main() {
    let mut rom = CodeGen::new();

    rom.emit_startup(0x3FFF);
    rom.call("clear_screen");
    rom.ld_hl_label("msg");
    rom.call("print_string");
    rom.halt();

    rom.label("msg");
    rom.emit_string("Hello from Z80!\r\n");

    rom.include_stdlib();
    rom.resolve_fixups();
    rom.write_bin("hello.bin").unwrap();
}

Load hello.bin onto a RetroShield (or run it in a Z80 emulator), and you'll see the greeting on your terminal.

Conclusion

The Z80 is nearly 50 years old, but it's still fun to program. The retroshield-z80-workbench brings modern development practices to vintage hardware: type-safe code generation, proper dependency management, fast iteration, and readable source.

Whether you want to build a clone of classic software, implement your own programming language for 8-bit hardware, or just understand how computers work at the machine code level, having the right tools makes all the difference. And there's still nothing quite like watching your code run on a chip that predates most programmers alive today.

The code for the workbench and all the kz80_* projects is available on GitHub under BSD-3-Clause licenses. PRs welcome.

Over the past year, I have been building a collection of programming language compilers and interpreters targeting the venerable Zilog Z80 microprocessor. What started as an experiment in retrocomputing has grown into a comprehensive suite of tools spanning multiple programming paradigms: from the functional elegance of LISP to the object-oriented messaging of Smalltalk, from the structured programming of Pascal and Fortran to the low-level control of C. This anthology documents the common architectural patterns, the unique challenges of targeting an 8-bit processor, and the unexpected joys of bringing modern language implementations to 1970s hardware.

My fascination with the Z80 began in the mid-1990s when I got my first TI-85 graphing calculator. That unassuming device, marketed for algebra and calculus homework, contained a Z80 running at 6 MHz with 28KB of RAM. Discovering that I could write programs in Z80 assembly and run them on this pocket computer was revelatory. I accumulated a small library of Z80 assembly books and spent countless hours learning the instruction set, writing simple games, and understanding how software meets hardware at the most fundamental level. Three decades later, this project represents a return to that formative obsession, now armed with modern tools and a deeper understanding of language implementation.

The RetroShield Platform

The RetroShield is a family of hardware adapters that bridge vintage microprocessors to modern Arduino development boards. The product line covers a remarkable range of classic CPUs: the MOS 6502 (powering the Apple II and Commodore 64), the Motorola 6809 (used in the TRS-80 Color Computer), the Intel 8085, the SC/MP, and the Zilog Z80. Each variant allows the original processor to execute real machine code while the Arduino emulates memory, peripherals, and I/O.

For this project, I focused exclusively on the RetroShield Z80. The Z80's rich instruction set, hardware BCD support via the DAA instruction, and historical significance as the CPU behind CP/M made it an ideal target for language implementation experiments. The RetroShield Z80 connects the actual Z80 chip to an Arduino Mega (or Teensy adapter for projects requiring more RAM), which emulates the memory and peripheral chips. This arrangement provides the authenticity of running on actual Z80 silicon while offering the convenience of modern development workflows.

The standard memory map provides 8KB of ROM at addresses 0x0000-0x1FFF and 6KB of RAM at 0x2000-0x37FF, though the Teensy adapter expands this significantly to 256KB. Serial I/O is handled through an emulated MC6850 ACIA chip at ports 0x80 and 0x81, providing the familiar RS-232 interface that connects these vintage programs to modern terminals.

It needs to be mentioned that if you do have a Z80 RetroShield and you want to run the binaries produced by the compilers collections on actual hardware, you will need a couple things: 1) bin2c, this is a program that will take a Z80 binary and turn it into a PROGMEM statement that you can put into an Arduino sketch. 2) Look at this sketch - there is code in there for emulating the MC6850 ACIA.

Common Compiler Architecture: Lexer, Parser, AST, Codegen

Every compiler in this collection follows a similar multi-stage architecture, a pattern that has proven itself across decades of compiler construction. Understanding this common structure reveals how the same fundamental approach can target vastly different source languages while producing efficient Z80 machine code.

The Lexer: Breaking Text into Tokens

The lexer (or tokenizer) is the first stage of compilation, responsible for transforming raw source code into a stream of tokens. Each language has its own lexical grammar: LISP recognizes parentheses and symbols, C identifies keywords and operators, Smalltalk distinguishes between message selectors and literals. Despite these differences, every lexer performs the same fundamental task of categorizing input characters into meaningful units.

In our Rust implementations, the lexer typically maintains a position in the source string and provides a next_token() method that advances through the input. This produces tokens like Token::Integer(42), Token::Plus, or Token::Identifier("factorial"). The lexer handles the tedious work of skipping whitespace, recognizing multi-character operators, and converting digit sequences into numbers.

The Parser: Building the Abstract Syntax Tree

The parser consumes the token stream and constructs an Abstract Syntax Tree (AST) that represents the hierarchical structure of the program. Most of our compilers use recursive descent parsing, a technique where each grammar rule becomes a function that may call other rule functions. This approach is intuitive, produces readable code, and handles the grammars of most programming languages effectively.

For example, parsing an arithmetic expression like 3 + 4 * 5 requires understanding operator precedence. The parser might have functions like parse_expression(), parse_term(), and parse_factor(), each handling operators at different precedence levels. The result is an AST where the multiplication is grouped as a subtree, correctly representing that it should be evaluated before the addition.

Code Generation: Emitting Z80 Machine Code

The code generator walks the AST and emits Z80 machine code. This is where the rubber meets the road: abstract operations like "add two numbers" become concrete sequences of Z80 instructions like LD A,(HL), ADD A,E, and LD (DE),A.

Most of our compilers generate code directly into a byte buffer, manually encoding each instruction's opcode and operands. This approach, while requiring intimate knowledge of the Z80 instruction set, gives us precise control over the generated code and avoids the complexity of an intermediate representation or separate assembler pass.

The DAA Instruction and BCD Arithmetic

One of the most fascinating aspects of Z80 programming is the DAA (Decimal Adjust Accumulator) instruction, opcode 0x27. This single instruction makes the Z80 surprisingly capable at decimal arithmetic, which proves essential for implementing numeric types on an 8-bit processor.

What is BCD?

Binary Coded Decimal (BCD) is a numeric representation where each decimal digit is stored in 4 bits (a nibble). Rather than storing the number 42 as binary 00101010 (its true binary representation), BCD stores it as 0100 0010, with the first nibble representing 4 and the second representing 2. This "packed BCD" format stores two decimal digits per byte.

While BCD is less space-efficient than pure binary (you can only represent 0-99 in a byte rather than 0-255), it has a crucial advantage: decimal arithmetic produces exact decimal results without rounding errors. This is why BCD was the standard for financial calculations on mainframes and why pocket calculators (including the famous TI series) used BCD internally.

How DAA Works

When you perform binary addition on two BCD digits, the result may not be valid BCD. Adding 0x09 and 0x01 gives 0x0A, but 0x0A is not a valid BCD digit. The DAA instruction corrects this: it examines the result and the half-carry flag (which indicates a carry from bit 3 to bit 4, i.e., from the low nibble to the high nibble) and adds 0x06 to any nibble that exceeds 9. After DAA, that 0x0A becomes 0x10, correctly representing decimal 10 in BCD.

This process works for both addition (after ADD or ADC instructions) and subtraction (after SUB or SBC instructions, where DAA subtracts 0x06 instead of adding it). The Z80 remembers whether the previous operation was addition or subtraction through its N flag.

BCD in Our Compilers

Several of our compilers use 4-byte packed BCD integers, supporting numbers up to 99,999,999 (8 decimal digits). The addition routine loads bytes from both operands starting from the least significant byte, adds them with ADC (add with carry) to propagate carries between bytes, applies DAA to correct each byte, and stores the result. The entire operation takes perhaps 20 bytes of code but provides exact decimal arithmetic on an 8-bit processor.

Here is a simplified version of our BCD addition loop:

This pattern appears in kz80_c, kz80_fortran, kz80_smalltalk, and kz80_lisp, demonstrating how a hardware feature designed in 1976 still provides practical benefits for language implementation.

The Evolution: From Assembly to C to Rust

The journey of implementing these compilers taught us valuable lessons about choosing the right tool for the job, and our approach evolved significantly over time.

First Attempt: Pascal in Z80 Assembly

Our first language implementation was kz80_pascal, a Pascal interpreter written entirely in Z80 assembly language. This approach seemed natural: if you are targeting the Z80, why not write directly in its native language?

The reality proved challenging. Z80 assembly, while powerful, is unforgiving. Building a recursive descent parser in assembly requires manually managing the call stack, carefully preserving registers across function calls, and debugging through hex dumps of memory. The resulting interpreter works and provides an interactive REPL for Pascal expressions, but extending it requires significant effort. Every new feature means more assembly, more potential for subtle bugs, and more time spent on implementation details rather than language design.

Second Attempt: Fortran 77 in C with SDCC

For kz80_fortran, we tried a different approach: writing the interpreter in C and cross-compiling with SDCC (Small Device C Compiler). This was dramatically more productive. C provided structured control flow, automatic stack management, and the ability to organize code into manageable modules.

The result is a comprehensive Fortran 77 subset with floating-point arithmetic (via BCD), subroutines and functions, arrays, and block IF statements. The C source compiles to approximately 19KB of Z80 code, fitting comfortably in ROM with room for program storage in RAM.

However, this approach has limitations. SDCC produces functional but not always optimal code, and debugging requires understanding both the C source and the generated assembly. The interpreter also requires the Teensy adapter with 256KB RAM, as the Arduino Mega's 4KB is insufficient for the runtime data structures.

The Rust Workbench: Our Final Form

Our breakthrough came with the realization that we did not need the compiler itself to run on the Z80, only the generated code. This insight led to what we call the "Rust workbench" approach: write the compiler in Rust, running on a modern development machine, and have it emit Z80 binary images.

This architecture provides enormous advantages:

Modern tooling: Cargo manages dependencies and builds, rustc catches bugs at compile time, and we have access to the entire Rust ecosystem for testing and development.

Fast iteration: Compiling a Rust program takes seconds; testing the generated Z80 code in our emulator takes milliseconds. Compare this to the multi-minute flash cycles required when the compiler runs on the target.

Comprehensive testing: Each compiler includes both Rust unit tests (testing the lexer, parser, and code generator individually) and integration tests that compile source programs and verify their output in the emulator.

Zero-dependency output: Despite being written in Rust, the generated Z80 binaries have no runtime dependencies. They are pure machine code that runs directly on the hardware.

This approach now powers kz80_lisp, kz80_c, kz80_lua, kz80_smalltalk, kz80_chip8, and retrolang. Each is a standalone Rust binary that reads source code and produces a 32KB ROM image.

The Z80 Emulator

None of this would be practical without a way to test generated code quickly. Our RetroShield Z80 Emulator provides exactly this: a cycle-accurate Z80 emulation with the same memory map and I/O ports as the real hardware.

The emulator comes in two versions: a simple passthrough mode (retroshield) that connects stdin/stdout directly to the emulated serial port, and a full TUI debugger (retroshield_nc) with register displays, disassembly views, memory inspection, and single-step execution. The passthrough mode enables scripted testing, piping test inputs through the emulator and comparing outputs against expected results. The TUI debugger proves invaluable when tracking down code generation bugs.

The emulator uses the superzazu/z80 library for CPU emulation, which provides accurate flag behavior and correct cycle counts. Combined with our MC6850 ACIA emulation, it provides a faithful recreation of the RetroShield environment without requiring physical hardware.

Self-Hosting Compilers: LISP and C

Two of our compilers achieve something remarkable: they can compile themselves and run on the target hardware. This property, called "self-hosting," is a significant milestone in compiler development.

What Does Self-Hosting Mean?

A self-hosting compiler is one written in the language it compiles. The classic example is the C compiler: most C compilers are themselves written in C. But this creates a chicken-and-egg problem: how do you compile a C compiler if you need a C compiler to compile it?

The solution is bootstrapping. You start with a minimal compiler written in some other language (or in machine code), use it to compile a slightly better compiler written in the target language, and iterate until you have a full-featured compiler that can compile its own source code. Once bootstrapped, the compiler becomes self-sustaining: future versions compile themselves.

kz80_lisp: A Self-Hosted LISP Compiler

kz80_lisp (crates.io) includes a LISP-to-Z80 compiler written in LISP itself. The compiler.lisp file defines functions that traverse LISP expressions and emit Z80 machine code bytes directly into memory. When you call (COMPILE '(+ 1 2)), it generates the actual Z80 instructions to load 1 and 2 and add them.

The self-hosted compiler supports arithmetic expressions, nested function calls, and can generate code that interfaces with the runtime's I/O primitives. While not a full replacement for the Rust-based code generator, it demonstrates that LISP is expressive enough to describe its own compilation to machine code.

kz80_c: A Self-Hosted C Compiler

kz80_c (crates.io) goes further: its self/cc.c file is a complete C compiler written in the C subset it compiles. This compiler reads C source from stdin and outputs Z80 binary to stdout, making it usable in shell pipelines:

The self-hosted C compiler supports all arithmetic operators, pointers, arrays, global variables, control flow statements, and recursive functions. Its main limitation is memory: the compiler source is approximately 66KB, exceeding the 8KB input buffer available on the Z80. This is a fundamental hardware constraint, not a compiler bug. In theory, a "stage 0" minimal compiler could bootstrap larger compilers.

Why Self-Hosting Matters

Self-hosting is more than a technical achievement; it validates the language implementation. If the compiler can compile itself correctly, it demonstrates that the language is expressive enough for real programs and that the code generator produces working code under complex conditions. For our Z80 compilers, self-hosting also connects us to the history of computing: the original Small-C compiler by Ron Cain in 1980 was similarly self-hosted on Z80/CP-M systems.

The Language Implementations

kz80_lisp

A minimal LISP interpreter and compiler featuring the full suite of list operations (CAR, CDR, CONS), special forms (QUOTE, IF, COND, LAMBDA, DEFINE), and recursive function support. The implementation includes a pure-LISP floating-point library and the self-hosted compiler mentioned above.

• GitHub: https://github.com/ajokela/kz80_lisp
• Crates.io: https://crates.io/crates/kz80_lisp

kz80_c

A C compiler supporting char (8-bit), int (16-bit), float (BCD), pointers, arrays, structs, and a preprocessor with #define and #include. The runtime library provides serial I/O and comprehensive BCD arithmetic functions. The self-hosted variant can compile and run C programs entirely on the Z80.

• GitHub: https://github.com/ajokela/kz80_c
• Crates.io: https://crates.io/crates/kz80_c

kz80_smalltalk

A Smalltalk subset compiler implementing the language's distinctive message-passing syntax with left-to-right operator evaluation. Expressions like 1 + 2 * 3 evaluate to 9 (not 7), matching Smalltalk's uniform treatment of binary messages. All arithmetic uses BCD with the DAA instruction.

• GitHub: https://github.com/ajokela/kz80_smalltalk
• Crates.io: https://crates.io/crates/kz80_smalltalk

kz80_lua

A Lua compiler producing standalone ROM images with an embedded virtual machine. Supports tables (Lua's associative arrays), first-class functions, closures, and familiar control structures. The generated VM interprets Lua bytecode, with frequently-used operations implemented in native Z80 code for performance.

• GitHub: https://github.com/ajokela/kz80_lua
• Crates.io: https://crates.io/crates/kz80_lua

kz80_fortran

A Fortran 77 interpreter with free-format input, REAL numbers via BCD floating point, block IF/THEN/ELSE/ENDIF, DO loops, subroutines, and functions. Requires the Teensy adapter for sufficient RAM. Written in C and cross-compiled with SDCC.

• GitHub: https://github.com/ajokela/kz80_fortran

kz80_pascal

A Pascal interpreter implemented in pure Z80 assembly. Provides an interactive REPL for expression evaluation with integer arithmetic, boolean operations, and comparison operators. A testament to the challenges of assembly language programming.

• GitHub: https://github.com/ajokela/kz80_pascal

retrolang

A custom systems programming language with Pascal/C-like syntax, featuring 16-bit integers, 8-bit bytes, pointers, arrays, inline assembly, and full function support with recursion. Compiles to readable Z80 assembly before assembling to binary.

• GitHub: https://github.com/ajokela/retrolang
• Crates.io: https://crates.io/crates/retrolang

kz80_chip8

A static recompiler that transforms CHIP-8 programs into native Z80 code. Rather than interpreting CHIP-8 bytecode at runtime, the compiler analyzes each instruction and generates equivalent Z80 sequences. Classic games like Space Invaders and Tetris run directly on the hardware.

• GitHub: https://github.com/ajokela/kz80_chip8
• Crates.io: https://crates.io/crates/kz80_chip8

Why Rust for Compiler Development?

The choice of Rust for our compiler workbench was not accidental. Several features make it exceptionally well-suited for this work.

Strong typing catches bugs early. When you're generating machine code, off-by-one errors or type mismatches can produce binaries that crash or compute wrong results. Rust's type system prevents many such errors at compile time.

Pattern matching excels at AST manipulation. Walking a syntax tree involves matching on node types and recursively processing children. Rust's match expressions with destructuring make this natural and exhaustive (the compiler warns if you forget a case).

Zero-cost abstractions. We can use high-level constructs like iterators, enums with data, and trait objects without runtime overhead. The generated compiler code is as efficient as hand-written C.

Excellent tooling. Cargo's test framework made it easy to build comprehensive test suites. Each compiler has dozens to hundreds of tests that run in seconds, providing confidence when making changes.

Memory safety without garbage collection. This matters less for the compilers themselves (which are desktop tools) but more for our mental model: thinking about ownership and lifetimes transfers naturally to thinking about Z80 register allocation and stack management.

Conclusion

Building these compilers has been a journey through computing history, from the Z80's 1976 architecture to modern Rust tooling, from the fundamentals of lexing and parsing to the intricacies of self-hosting. The BCD arithmetic that seemed like a curiosity became a practical necessity; the emulator that started as a debugging aid became essential infrastructure; the Rust workbench that felt like an optimization became the key to productivity.

The Z80 remains a remarkable teaching platform. Its simple instruction set is comprehensible in an afternoon, yet implementing real languages for it requires genuine compiler engineering. Every language in this collection forced us to think carefully about representation, evaluation, and code generation in ways that higher-level targets often obscure.

All of these projects are open source under BSD-3-Clause licenses. The compilers are available on both GitHub and crates.io, ready to install with cargo install. Whether you are interested in retrocomputing, compiler construction, or just curious how programming languages work at the metal level, I hope these tools and their source code prove useful.

The Z80 may be nearly 50 years old, but it still has lessons to teach.

There's something deeply satisfying about running code on vintage hardware. The blinking cursor, the deliberate pace of execution, the direct connection between your keystrokes and the machine's response. The RetroShield by Erturk Kocalar brings this experience to modern makers by allowing real vintage CPUs like the Zilog Z80 to run on Arduino boards. But what if you could experience that same feeling directly in your web browser?

That's exactly what I set out to build: a complete Z80 emulator that runs RetroShield firmware in WebAssembly, complete with authentic CRT visual effects and support for multiple programming language interpreters.

Try It Now

Select a ROM below and click "Load ROM" to start. Click on the terminal to focus it, then type to interact with the interpreter.

The RetroShield Platform

Before diving into the emulator, it's worth understanding what makes the RetroShield special. Unlike software emulators that simulate a CPU in code, the RetroShield uses a real vintage microprocessor. The Z80 variant features an actual Zilog Z80 chip running at its native speed, connected to an Arduino Mega or Teensy that provides:

• Memory emulation: The Arduino's SRAM serves as the Z80's RAM, while program code is stored in the Arduino's flash memory
• I/O peripherals: Serial communication, typically through an emulated MC6850 ACIA or Intel 8251 USART
• Clock generation: The Arduino provides the clock signal to the Z80

This hybrid approach means you get authentic Z80 behavior - every timing quirk, every undocumented opcode - while still having the convenience of USB connectivity and easy program loading.

The RetroShield is open source hardware and available on Tindie. For larger programs, the Teensy adapter expands available RAM from about 4KB to 256KB.

The Hardware Up Close

Here's my RetroShield Z80 setup with the Teensy adapter:

The Zilog Z80 CPU sits in the 40-pin DIP socket, with the Teensy 4.1 providing memory emulation and I/O handling beneath.

The physical hardware runs identically to the browser emulator above - the same ROMs, the same interpreters, the same authentic Z80 execution.

Why Build a Browser Emulator?

Having built several interpreters and tools for the RetroShield, I found myself constantly cycling through the development loop: edit code, compile, flash to Arduino, test, repeat. A software emulator would speed this up significantly, but I also wanted something I could share with others who might not have the hardware.

WebAssembly seemed like the perfect solution. It runs at near-native speed in any modern browser, requires no installation, and can be embedded directly in a web page. Someone curious about retro computing could try out a Fortran 77 interpreter or Forth environment without buying any hardware.

Building the Emulator in Rust

I chose Rust for the emulator implementation for several reasons:

1. Excellent WASM support: Rust's wasm-bindgen and wasm-pack tools make compiling to WebAssembly straightforward
2. Performance: Rust compiles to efficient code, important for cycle-accurate emulation
3. The rz80 crate: Andre Weissflog's rz80 provides a battle-tested Z80 core

The emulator architecture is straightforward:

┌─────────────────────────────────────────────────┐
│                 Web Browser                      │
│  ┌───────────────────────────────────────────┐  │
│  │            JavaScript/HTML                 │  │
│  │  - Terminal display with CRT effects      │  │
│  │  - Keyboard input handling                │  │
│  │  - ROM loading and selection              │  │
│  └─────────────────────┬─────────────────────┘  │
│                        │ wasm-bindgen            │
│  ┌─────────────────────▼─────────────────────┐  │
│  │         Rust/WebAssembly Core             │  │
│  │  ┌─────────────┐  ┌─────────────────────┐ │  │
│  │  │  rz80 CPU   │  │  Memory (64KB)      │ │  │
│  │  │  Emulation  │  │  ROM + RAM          │ │  │
│  │  └─────────────┘  └─────────────────────┘ │  │
│  │  ┌─────────────────────────────────────┐  │  │
│  │  │  I/O Emulation                       │  │  │
│  │  │  - MC6850 ACIA (ports $80/$81)      │  │  │
│  │  │  - Intel 8251 USART (ports $00/$01) │  │  │
│  │  └─────────────────────────────────────┘  │  │
│  └───────────────────────────────────────────┘  │
└─────────────────────────────────────────────────┘

Dual Serial Chip Support

One challenge was supporting ROMs that use different serial chips. The RetroShield ecosystem has two common configurations:

MC6850 ACIA (ports $80/$81): Used by many homebrew projects including MINT, Firth Forth, and my own Fortran and Pascal interpreters. The ACIA has four registers (control, status, transmit data, receive data) mapped to two ports, with separate read/write functions per port.

Intel 8251 USART (ports $00/$01): Used by Grant Searle's popular BASIC port and the EFEX monitor. The 8251 is simpler with just two ports - one for data and one for control/status.

The emulator detects which chip to use based on ROM metadata and configures the I/O handlers accordingly.

Memory Layout

The standard RetroShield memory map looks like this:

Most of my interpreters use a layout where code occupies the lower addresses and data/stack occupy higher memory. The Fortran interpreter, for example, places its program text storage at $6700 and variable storage at $7200, with the stack growing down from $8000.

The CRT Effect

No retro computing experience would be complete without the warm glow of a CRT monitor. I implemented several visual effects using pure CSS:

Scanlines: A repeating gradient overlay creates the horizontal line pattern characteristic of CRT displays:

.crt-container::before {
    background: linear-gradient(
        rgba(18, 16, 16, 0) 50%,
        rgba(0, 0, 0, 0.25) 50%
    );
    background-size: 100% 4px;
}

Chromatic aberration: CRT displays have slight color fringing due to the electron beam hitting phosphors at angles. I simulate this with animated text shadows that shift red and blue components:

@keyframes textShadow {
    0% {
        text-shadow: 0.4px 0 1px rgba(0,30,255,0.5),
                    -0.4px 0 1px rgba(255,0,80,0.3);
    }
    /* ... animation continues */
}

Flicker: Real CRTs had subtle brightness variations. A randomized opacity animation creates this effect without being distracting.

Vignette: The edges of CRT screens were typically darker than the center, simulated with a radial gradient.

The font: I'm using the Glass TTY VT220 font, a faithful recreation of the DEC VT220 terminal font from the 1980s. It's public domain and adds significant authenticity to the experience.

The Language Interpreters

The emulator comes pre-loaded with several language interpreters, each running as native Z80 code:

Fortran 77 Interpreter

This is my most ambitious RetroShield project: a subset of Fortran 77 running interpretively on an 8-bit CPU. It supports:

• REAL numbers via BCD (Binary Coded Decimal) floating point with 8 significant digits
• INTEGER and REAL variables with implicit typing (I-N are integers)
• Arrays up to 3 dimensions
• DO loops with optional STEP
• Block IF/THEN/ELSE/ENDIF statements
• SUBROUTINE and FUNCTION subprograms
• Intrinsic functions: ABS, MOD, INT, REAL, SQRT

Here's a sample session:

FORTRAN-77 Interpreter v0.3
RetroShield Z80
Ready.
> PROGRAM FACTORIAL
  INTEGER I, N, F
  N = 7
  F = 1
  DO 10 I = 1, N
  F = F * I
  10 CONTINUE
  WRITE(*,*) N, '! =', F
  END
Program entered. Type RUN to execute.
> RUN
7 ! = 5040

The interpreter is written in C and cross-compiled with SDCC. At roughly 21KB of code, it pushes the limits of what's practical on the base RetroShield, which is why it requires the Teensy adapter.

MINT (Minimal Interpreter)

MINT is a wonderfully compact stack-based language. Each command is a single character, making it incredibly memory-efficient:

> 1 2 + .
3
> : SQ D * ;
> 5 SQ .
25

Firth Forth

A full Forth implementation by John Hardy. Forth's stack-based paradigm and extensibility made it popular on memory-constrained systems:

> : FACTORIAL ( n -- n! ) 1 SWAP 1+ 1 DO I * LOOP ;
> 7 FACTORIAL .
5040

Grant Searle's BASIC

A port of Microsoft BASIC that provides the classic BASIC experience:

Z80 BASIC Ver 4.7b
Ok
> 10 FOR I = 1 TO 10
> 20 PRINT I * I
> 30 NEXT I
> RUN
1
4
9
...

Technical Challenges

Building this project involved solving several interesting problems:

Memory Layout Debugging

The Fortran interpreter crashed mysteriously when entering lines with statement labels. After much investigation, I discovered the CODE section had grown to overlap with the DATA section. The linker was told to place data at $5000, but code had grown past that point. The fix was updating the memory layout to give code more room:

# Before: code overlapped data
LDFLAGS = --data-loc 0x5000

# After: proper separation
LDFLAGS = --data-loc 0x5500

This kind of bug is particularly insidious because it works fine until the code grows past a certain threshold.

BCD Floating Point

Implementing floating-point math on a Z80 without hardware support is challenging. I chose BCD (Binary Coded Decimal) representation because:

1. Exact decimal representation: No binary floating-point surprises like 0.1 + 0.2 != 0.3
2. Simpler conversion: Reading and printing decimal numbers is straightforward
3. Reasonable precision: 8 BCD digits give adequate precision for an educational interpreter

Each BCD number uses 6 bytes: 1 for sign, 1 for exponent, and 4 bytes holding 8 packed decimal digits.

Cross-Compilation with SDCC

The Small Device C Compiler (SDCC) targets Z80 and other 8-bit processors. While it's an impressive project, there are quirks:

• No standard library functions that assume an OS
• Limited optimization compared to modern compilers
• Memory model constraints require careful attention to data placement

I wrote a custom crt0.s startup file that initializes the stack, sets up the serial port, and calls main().

Running the Emulator

The emulator runs at roughly 3-4 MHz equivalent speed, depending on your browser and hardware. This is actually faster than the original Z80's typical 4 MHz, but the difference isn't noticeable for interactive use.

To try it:

1. Visit the Z80 Emulator page
2. Select a ROM from the dropdown (try Fortran 77)
3. Click "Load ROM"
4. Click on the terminal to focus it
5. Start typing!

For Fortran, try entering a simple program:

PROGRAM HELLO
WRITE(*,*) 'HELLO, WORLD!'
END

Then type RUN to execute it.

What's Next

There's always more to do:

• More ROM support: Expanding to additional retro languages like LISP, Logo, or Pilot
• Debugger integration: Showing registers, memory, and allowing single-stepping
• Save/restore state: Persisting the emulator state to browser storage
• Mobile support: Touch-friendly keyboard for tablets and phones

Source Code and Links

Everything is open source:

• Emulator source (Rust/WASM)
• Fortran interpreter source
• RetroShield hardware designs
• RetroLang compiler - my custom language for Z80

The RetroShield hardware is available from 8bitforce on Tindie.

Acknowledgments

• Erturk Kocalar - Creator of the RetroShield platform
• Andre Weissflog - Author of the rz80 Z80 emulator core
• Grant Searle - Z80 BASIC and reference designs
• John Hardy - Author of Firth Forth, MINT, and Monty

There's something magical about running 49-year-old CPU architectures in a modern web browser. The Z80 powered countless home computers, embedded systems, and arcade games. With this emulator, that legacy is just a click away.

Gilbreths, Vorticism and the Echoes of Artificial Intelligence in the Twenty-First-Century Knowledge Economy

Introduction

The first decades of the twentieth century were a crucible of technological, scientific and cultural transformation. The steam-driven factory floor, the internal-combustion automobile, the telegraph-to-telephone network, and the nascent film industry all collapsed distance and accelerated the rhythm of everyday life. In that moment of accelerated modernity two seemingly unrelated phenomena emerged on opposite sides of the Atlantic: the Gilbreths' scientific-management laboratory in the United States, and the Vorticist avant-garde in Britain.

Both were responses to a shared "milieu"—a world in which the machine was no longer a peripheral tool but the central fact of existence. The Gilbreths turned the machine into a system of human motion, dissecting work into its smallest elements (the "therbligs") and re-engineering tasks for efficiency, ergonomics and profit. Vorticists, led by Wyndham Lewis and allied with figures such as Ezra Pound and Henri Gaudier-Brzeska, seized upon the same mechanical dynamism in a visual language of sharp angles, fractured planes and kinetic abstraction.

A century later, the rise of artificial intelligence is reshaping the same terrain, but this time the target is not manual labor on the factory floor; it is knowledge work, the very act of thinking, deciding and creating. Yet the cultural logic that animated the Gilbreths and the Vorticists resurfaces in the AI era: a faith in rationalization, an obsession with breaking complex processes into analyzable units, a belief that design—whether of a workflow, a painting, or an algorithm—can impose order on the chaos of modern life.

This essay weaves together three strands. First, it sketches the broader historical and intellectual atmosphere that nurtured both the Gilbreths and Vorticism. Second, it juxtaposes their concrete practices and aesthetic strategies, drawing out the convergences in their conceptualization of motion, fragmentation, control and progress. Third, it maps these early-twentieth-century dynamics onto the present AI-driven re-organization of knowledge labor, arguing that the same cultural grammar underlies both epochs, even as the material substrates have shifted from bricklaying to neural networks.

1. The Early-Twentieth-Century Milieu

Technological Acceleration

Between 1900 and 1920 the world witnessed a multiplication of speed. The internal-combustion engine made automobiles and aircraft possible; the electric motor powered factories and household appliances; the telephone and radio collapsed geographic distance; the cinema rendered motion visible and repeatable. Historian David Edgerton has shown that these "new machines" were not simply tools but actors that reshaped social relations (Edgerton, The Shock of the Old, 2006). The very perception of time became quantifiable: a stopwatch could now register the fraction of a second it took a worker to raise a hammer, a clerk to type a word, or a runner to cross a track.

Scientific Management and the Quest for Rational Order

Frederick Winslow Taylor published The Principles of Scientific Management (1911), arguing that work could be transformed into a science through measurement, standardization and hierarchical control. Taylor's ideas traveled swiftly across the Atlantic, finding eager audiences in American industry and, later, in British engineering firms. The core premise was that human labor could be rendered as predictable, repeatable data, amenable to optimization.

The Gilbreths—Frank B. Gilbreth Sr. (a mechanical engineer) and Lillian M. Gilbreth (a psychologist)—expanded Taylor's blueprint. They introduced motion-study photography, a method of capturing workers' movements on film, then dissecting each frame to isolate "therbligs," the elementary units of motion (the word itself a reversal of "Gilbreth"). Their work was both scientific and humane: they claimed that eliminating unnecessary motions would reduce fatigue, increase safety and, paradoxically, improve the worker's quality of life. Their 1915 book Motion Study blended engineering diagrams with psychological insight, making the Gilbreths the archetype of industrial ergonomics.

The Cultural Avant-Garde

Concurrently, a wave of artistic experimentation was erupting across Europe. Cubism (Picasso, Braque) deconstructed visual reality into geometric facets; Futurism (Marinetti, Balla) glorified speed, noise and the machine; Constructivism (Tatlin, Rodchenko) championed functional design as a social weapon. In London, a small cadre of writers and painters, disillusioned with the lingering Victorian aesthetic, coalesced around the journal BLAST (1914-1915).

The manifesto of Vorticism, authored chiefly by Wyndham Lewis, declared a desire to capture the "vortex"—the point where energy, motion and form converge. Vorticist works are characterized by hard-edged angularity, stark color contrasts and a sense of centrifugal force. They rejected the lyrical softness of the Pre-Raphaelite tradition and the pastoral nostalgia of the Edwardian era, instead embracing the "hard, machine-like precision" of the new industrial world.

Overlapping Intellectual Currents

Both the Gilbreths and the Vorticists were embedded in a broader intellectual climate that prized measurement, abstraction and the re-creation of reality. The rise of psychophysics, behaviorism, and physiological psychology introduced the notion that human perception and action could be quantified. In parallel, philosophers such as Henri Bergson were wrestling with the concept of duration and the mechanization of time, while sociologists like Georg Simmel explored the "blasé" effect of urban modernity. The shared vocabulary of "efficiency," "speed," "fragmentation" and "design" became the lingua franca of both engineers and artists.

2. Parallel Strategies: From Motion Study to Vortex

The Machine as Central Fact

Both movements privileged the machine not as a peripheral tool but as a defining lens through which to understand humanity. The Gilbreths approached human labor as a component of a larger production system, treating the body like a mechanical part. Their methods of representation—motion-study film frames, thermographic charts, time-and-motion diagrams—reduced the worker to analyzable data. Their ontological stance held that reality could be reduced to measurable motions, with the machine serving as the baseline condition of life.

The Vorticists operated from a parallel framework but expressed it through aesthetic means. They rendered the human figure and urban landscape as networks of intersecting mechanical forms, employing sharp angular compositions, overlapping planes, and stylized gears and dynamized lines. For them, reality was a flux of forces, and the "vortex" captured the dynamic, mechanized energy of modern existence.

In both cases, the human body was subordinated to, or fused with, a system of motion. For the Gilbreths, a worker's hand was a lever; for the Vorticists, a dancer's limb could be a blade of light cutting through the air.

Fragmentation and Reassembly

The Gilbreths' therbligs (e.g., "reach," "grasp," "move") represent a conceptual atomization of work. By isolating each atomic action, they could re-assemble a sequence that minimized waste and maximized output. This analytical practice mirrors the visual fragmentation employed by Vorticist painters, who broke down objects into geometric primitives before re-constituting them on canvas.

Consider a typical Gilbreth motion-study photograph of a bricklayer: the image is a series of still frames, each showing the worker's arm at a distinct angle. The analyst's task is to trace the trajectory, identify redundant motions, and propose a smoother path. In a Vorticist painting such as Wyndham Lewis's The Crowd (1914-15), the same crowd is depicted as a constellation of overlapping triangles and intersecting lines, each fragment suggesting a movement, a direction, a force. The similarity lies not in content but in methodology: a belief that complex reality becomes intelligible when decomposed into simpler parts.

Control, Order and Design

Both camps produced manifestos that served as design blueprints for their respective domains.

The Gilbreths published practical handbooks—Motion Study (1915), Applied Motion Study (1922)—that provided step-by-step protocols for reorganizing factories, hospitals and even homes. Their famous household experiment, depicted in Cheaper by the Dozen (1948), turned family life into a laboratory of efficiency.

The Vorticists issued the BLAST manifesto (1914), a terse proclamation that called for "a new art that will cut away the old, the sentimental, the decorative". It demanded clarity, precision, and a rejection of "softness"—values that echo the Gilbreths' insistence on eliminating "soft" motions that do not contribute to productive output.

Both therefore exerted cultural authority by prescribing how the world should be organized—whether through a Gantt chart or a bold, angular composition.

Ambivalent Faith in Progress

The Gilbreths believed that scientific optimization would lead to a more humane workplace. Yet their work also laid the groundwork for later Taylorist dehumanization, where workers became interchangeable cogs. Their optimism was tempered by the reality that efficiency could be weaponized for profit, not for worker welfare.

Vorticists, especially Lewis, celebrated the "machine aesthetic" but also expressed an undercurrent of skepticism. Lewis's later writings (e.g., The Apes of God, 1930) reveal a cynical view of mass culture and the mechanization of society. The vortex, while a source of energy, can also become a whirlpool of alienation.

Thus, both movements embody a dual vision of modernity: a promise of liberation through order, paired with a fear of loss of individuality.

3. The AI Turn: Re-Engineering Knowledge Work

From Bricklaying to Algorithms

If the Gilbreths turned the physical act of building into a set of measurable motions, today's AI researchers turn the cognitive act of reasoning into data. Machine-learning pipelines ingest millions of text fragments, label them, and train neural networks that can generate, summarize, and evaluate human language. The "therblig" of a knowledge worker—reading, analyzing, drafting—can now be instrumented by click-stream data, eye-tracking, and keystroke dynamics.

Just as a motion-study camera captured the kinematics of a worker, modern digital platforms capture the logistics of a mind at work. The "process mining" tools used in enterprise software map the sequence of digital actions much as Gilbreth charts mapped the sequence of physical actions.

Fragmentation of Cognitive Tasks

AI development follows the same atomization logic that underpinned both the Gilbreths and the Vorticists. Large language models (LLMs) are trained on tokenized text, where each token—often a sub-word fragment—is a basic unit of meaning. The model learns statistical relationships between tokens, then re-assembles them into sentences, paragraphs, or code.

Similarly, the micro-task platforms (e.g., Amazon Mechanical Turk) break down complex knowledge work (data labeling, content moderation) into tiny, repeatable units that can be distributed across a crowd. The "crowd" becomes a modern analog of the bricklayer's workshop, and the platform's algorithmic workflow is the contemporary "assembly line".

Design, Control and the Algorithmic Order

Just as the Gilbreths produced process charts and Vorticists drafted manifestos, AI researchers issue model cards, datasheets for datasets, and ethical guidelines. These documents codify how the system should behave, what data it may use, and how it ought to be evaluated—mirroring the design-by-specification ethos of early scientific management.

The rise of "prompt engineering"—the craft of phrasing inputs to LLMs to obtain desired outputs—can be read as a new form of motion study. Prompt engineers dissect the model's internal "motion" (attention patterns, token probabilities) and rearrange the prompt to optimize the "efficiency" of the model's response.

Ambivalence and Ethical Dilemmas

The Gilbreths' optimism about worker welfare was later undercut by automation-induced job loss and the rise of "scientific" surveillance of labor. Vorticism's celebration of the machine later seemed naïve in the face of the World Wars and the totalitarian use of technology.

AI today reproduces this ambivalence. Proponents hail it as a tool that will free humanity from routine cognition, allowing us to focus on creativity and empathy. Critics warn of algorithmic bias, disinformation, and the erosion of skilled labor. The "vortex" of AI can either be a centrifugal force that propels society forward or a black-hole that absorbs human agency.

4. Comparative Synthesis: Themes Across the Century

The Machine as Ontological Baseline

Across all three movements, the machine serves not merely as a tool but as a fundamental framework for understanding human existence. The Gilbreths treated the human body as a component of a larger mechanical system. The Vorticists rendered human figures as geometric, machine-like forms on canvas. Today's AI researchers model human cognition as data pipelines and neural "circuits." Each epoch finds its own way to subordinate organic complexity to mechanical logic.

Fragmentation and Reassembly

The pattern of breaking down complex wholes into analyzable parts, then reconstituting them in optimized form, appears consistently across all three contexts. The Gilbreths isolated "therbligs" from continuous motion. Vorticist artists broke visual reality into planes and reassembled them into the vortex. Modern AI systems tokenize text, distribute cognitive tasks across micro-work platforms, and build modular model components. The underlying faith remains the same: that decomposition reveals the essence of things and enables their improvement.

Design as Control

Each movement produced its own form of prescriptive documentation. The Gilbreths created process charts, standardized tools, and ergonomic workstation designs. The Vorticists issued manifestos prescribing aesthetic order and "hard edges." AI practitioners develop model cards, governance frameworks, and prompt engineering guides. All represent attempts to codify and control complex systems through explicit design principles.

Faith in Progress Tempered by Anxiety

The Gilbreths promised that efficiency would bring both productivity and worker welfare, yet their methods also enabled dehumanization. The Vorticists celebrated speed and mechanical energy while hinting at alienation in their fractured compositions. AI promises cognitive augmentation while raising concerns about surveillance and the erosion of human expertise. Each technological moment carries this dual character: the hope of liberation alongside the fear of submission.

The Shifting Cultural Milieu

The Gilbreths operated within a milieu shaped by Taylorism, psychophysics, mass media, and rapid urbanization. The Vorticists emerged amid Futurism, Cubism, Constructivism, and the upheaval of the First World War. Today's AI revolution unfolds against the backdrop of big data, ubiquitous connectivity, platform capitalism, and post-pandemic remote work. Though the specific historical conditions differ, the structural logic linking these moments remains remarkably stable. What changes is the material substrate—bricks, paint, or bits—and the scale of impact—factory floors, galleries, or global digital ecosystems.

5. The "New Vortex": AI as Contemporary Avant-Garde

Just as Vorticism attempted to visualize the invisible forces of industrial modernity, AI functions as a conceptual vortex that reshapes how we see knowledge. The latent space of a language model can be visualized as a high-dimensional field of probabilities, a kind of abstract energy landscape. Artists and designers now employ AI to generate images (e.g., DALL-E, Midjourney) that echo Vorticist aesthetics: sharp, kinetic, synthetic. The algorithmic brushstroke replaces the painter's line, yet the visual language still speaks of speed, fragmentation, and mechanized beauty.

Moreover, the cultural discourse around AI mirrors the manifestos of early avant-garde movements. Papers such as "The Ethics of Artificial Intelligence" (Bostrom & Yudkowsky, 2014) and corporate statements like Google's AI Principles (2018) function as modern manifestos, setting out a vision of a rational, humane future while warning against the dark vortex of misuse.

6. Implications for the Future of Work and Culture

Re-thinking Efficiency

The Gilbreths taught that efficiency is not merely speed, but the minimization of wasteful motion. In the AI era, efficiency must be re-conceptualized as cognitive economy: reducing unnecessary mental load, automating routine reasoning, and presenting information in ways that align with human attention patterns. However, a purely quantitative approach—optimizing click-through rates or model loss functions—runs the risk of reducing the richness of human judgment, just as early Taylorism reduced workers to data points.

Agency and the "Human-Machine" Hybrid

Both Vorticism and the Gilbreths celebrated the integration of human and machine, yet they also highlighted a tension: the loss of the organic in favor of the mechanical. Today, human-AI collaboration (often called "centaur" models) seeks a synthesis where humans guide, correct, and imbue AI with values, while AI handles scale and pattern detection. The artistic "vortex" becomes a collaborative vortex—a shared space where the algorithm's output is a raw material that the human refines.

Ethical Governance as Modern Manifesto

Just as Vorticist manifestos set out a normative framework for artistic production, AI governance documents aim to define norms for algorithmic behavior. The challenge is to avoid the pitfalls of technocratic paternalism—the belief that a small elite can dictate the shape of society through scientific design, a stance implicit in early scientific management. Democratic participation, interdisciplinary oversight, and transparent "process charts" (e.g., model interpretability dashboards) can help ensure that the AI vortex does not become a black-hole of control.

Conclusion

The Gilbreths and the Vorticists were, in their own ways, architects of the modern machine age. The former turned the human body into a calibrated component of industrial systems, while the latter rendered human experience as a kinetic, geometric abstraction. Both operated within a cultural environment that prized measurement, fragmentation, and the belief that design could impose order on a rapidly changing world.

A century later, artificial intelligence stands at a comparable crossroads. The same grammar of fragmentation, reassembly, and control underlies the transformation of knowledge work. Motion-study films have been supplanted by digital telemetry; therbligs have given way to token embeddings; Vorticist canvases now coexist with AI-generated visualizations of latent spaces.

Yet, as history shows, each wave of technological rationalization brings both liberation and alienation. The Gilbreths' optimism about a more humane workplace was later tempered by concerns over mechanistic dehumanization; Vorticism's celebration of the machine was later haunted by the specter of war and totalitarian control. In the AI epoch, we must likewise balance the promise of cognitive augmentation with vigilance against algorithmic opacity, bias, and the erosion of skilled judgment.

The lesson from the early twentieth century is not that the machine should be rejected, but that human agency must remain the central design parameter. If we can learn to treat AI not as a new "vortex" that swallows us, but as a collaborative partner that can be shaped through transparent, ethically grounded processes, we may fulfill the Gilbreths' original hope—more efficient work without sacrificing humanity—and realize a Vorticist vision of a world where form, function, and freedom converge in the bright, kinetic heart of the modern age.

Selected Bibliography

• Edgerton, David. The Shock of the Old: Technology and Global History Since 1900. Oxford University Press, 2006.
• Gilbreth, Frank B., and Lillian M. Gilbreth. Motion Study: A Method for Improving Industrial Efficiency. Sturgis & Walton, 1915.
• Lewis, Wyndham. A Blast Manifesto. 1914. In Vorticism: An Anthology, ed. Paul Edwards, Oxford University Press, 1977.
• Marinetti, Filippo Tommaso. The Futurist Manifesto. 1909.
• Bostrom, Nick, and Eliezer Yudkowsky. "The Ethics of Artificial Intelligence." Cambridge Handbook of Artificial Intelligence, 2014.
• Simmel, Georg. The Metropolis and Mental Life. 1903.
• Taylor, Frederick Winslow. The Principles of Scientific Management. Harper & Brothers, 1911.
• Zuboff, Shoshana. The Age of Surveillance Capitalism. PublicAffairs, 2019.

Abstract

Modern AI assistants like ChatGPT have fundamentally changed user expectations around conversational interfaces. Users now expect to have coherent, multi-turn conversations where the AI remembers what was said earlier in the discussion. However, when building AI-powered bots on top of messaging platforms like Signal, Telegram, or SMS, developers face a fundamental architectural challenge: these platforms are inherently stateless. Each message arrives as an independent event with no built-in mechanism for maintaining conversational context.

This paper examines a production implementation that bridges this gap, enabling persistent multi-turn AI conversations over Signal's stateless messaging protocol. We explore the database schema design, the command parsing architecture, and a novel inline image reference system that allows users to incorporate visual context into ongoing conversations.

1. Introduction

1.1 The Statefulness Problem

Large Language Models (LLMs) like GPT-4 and GPT-5 are stateless by design. Each API call is independent—the model has no memory of previous interactions unless the developer explicitly includes conversation history in each request. Services like ChatGPT create the illusion of memory by maintaining conversation state server-side and replaying the full message history with each new user input.

When building a bot on a messaging platform, developers must solve this same problem, but with additional constraints:

1. Message Independence: Each incoming message from Signal (or similar platforms) arrives as a discrete event with no connection to previous messages.

2. Multi-User Environments: In group chats, multiple users may be conducting separate conversations with the bot simultaneously.

3. Asynchronous Delivery: Messages may arrive out of order or with significant delays.

4. Platform Limitations: Most messaging APIs provide no native support for threading or conversation tracking.

5. Resource Constraints: Storing complete conversation histories for every interaction can become expensive, both in terms of storage and API costs (since longer histories mean more tokens per request).

1.2 Design Goals

Our implementation targets the following objectives:

• Conversation Continuity: Users should be able to continue previous conversations by referencing a conversation ID.
• New Conversation Simplicity: Starting a fresh conversation should require no special syntax—just send a message.
• Multi-Modal Support: Users should be able to reference images stored in the system within their conversational context.
• Cost Transparency: Each response should report the API cost and attribute it correctly for multi-user billing scenarios.
• Thread Safety: The system must handle concurrent conversations from multiple users without data corruption.

2. Database Schema Design

2.1 Conversation Tables

The persistence layer uses SQLite with a straightforward two-table design:

CREATE TABLE gpt_conversations (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    created_at TEXT NOT NULL
);

CREATE TABLE gpt_conversation_messages (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    conversation_id INTEGER NOT NULL,
    created_at TEXT NOT NULL,
    role TEXT NOT NULL,
    content TEXT NOT NULL,
    FOREIGN KEY (conversation_id) REFERENCES gpt_conversations(id)
);

The gpt_conversations table serves as a lightweight header, storing only the conversation ID and creation timestamp. The actual message content lives in gpt_conversation_messages, which maintains the full history of each conversation.

2.2 Schema Rationale

Several design decisions merit explanation:

Minimal Conversation Metadata: The gpt_conversations table intentionally stores minimal information. We considered adding fields like user_id, title, or summary, but found these complicated the implementation without providing sufficient value. The conversation ID alone is enough to retrieve and continue any conversation.

Text Storage for Timestamps: Rather than using SQLite's native datetime types, we store ISO 8601 formatted strings. This provides timezone awareness (critical for a system serving users across time zones) and human readability when debugging.

Content as Plain Text: The content field stores the raw message text, not a structured format. This keeps the schema simple and avoids premature optimization. When multi-modal content (like inline images) is needed, we resolve references at query time rather than storing binary data in the conversation history.

Foreign Key Constraints: The foreign key relationship between messages and conversations ensures referential integrity and enables cascading deletes if conversation cleanup is needed.

3. Conversation Management API

3.1 Core Operations

The database abstraction layer exposes three primary operations:

def create_gpt_conversation(first_message: GPTMessage) -> int:
    """Create a new conversation and return its ID."""
    with get_db_connection() as conn:
        cur = conn.cursor()
        cur.execute(
            "INSERT INTO gpt_conversations (created_at) VALUES (?)",
            (pendulum.now("America/Chicago").isoformat(),),
        )
        new_id = cur.lastrowid
        conn.commit()
        add_message_to_conversation(new_id, first_message)
        return new_id

The create_gpt_conversation function atomically creates both the conversation record and its first message. This ensures that no conversation exists without at least one message, maintaining data consistency.

def add_message_to_conversation(conversation_id: int, message: GPTMessage):
    """Append a message to an existing conversation."""
    with get_db_connection() as conn:
        cur = conn.cursor()
        cur.execute(
            """INSERT INTO gpt_conversation_messages
               (conversation_id, created_at, role, content)
               VALUES (?, ?, ?, ?)""",
            (conversation_id, pendulum.now().isoformat(),
             message.role, message.content),
        )
        conn.commit()

def get_messages_for_conversation(conversation_id: int) -> List[GPTMessage]:
    """Retrieve all messages in chronological order."""
    with get_db_connection() as conn:
        cur = conn.cursor()
        cur.execute(
            """SELECT created_at, role, content
               FROM gpt_conversation_messages
               WHERE conversation_id = ?
               ORDER BY created_at ASC""",
            (conversation_id,),
        )
        rows = cur.fetchall()
        return [GPTMessage(role=row[1], content=row[2]) for row in rows]

3.2 The GPTMessage Data Class

Messages are represented using a simple data class that mirrors the OpenAI API's message format:

@dataclass
class GPTMessage:
    role: str      # "user", "assistant", or "system"
    content: str   # The message text (or structured content for multi-modal)

This alignment with the OpenAI API structure means messages can be retrieved from the database and passed directly to the API without transformation, reducing complexity and potential for bugs.

4. Command Parsing and Conversation Flow

4.1 Command Syntax

The bot supports an optional conversation ID in its command syntax:

gpt <prompt>                    # Start new conversation
gpt <conversation_id> <prompt>  # Continue existing conversation

This is implemented via a regex pattern that makes the conversation ID optional:

def _process_gpt_command(text: str, command: str, model: GPTModel) -> bool:
    pat = rf"^{command} (\d+ )?\s?(.*)"
    m = re.search(pat, text, flags=re.IGNORECASE | re.DOTALL)
    if not m:
        return False

    conversation_id = m.groups()[0]  # None if not provided
    prompt = m.groups()[1]

4.2 Conversation Branching Logic

The command handler implements distinct paths for new versus continued conversations:

if conversation_id:
    # Continue existing conversation
    signal_archive_db.add_message_to_conversation(
        conversation_id, GPTMessage(role="user", content=prompt)
    )
    messages = signal_archive_db.get_messages_for_conversation(conversation_id)
    conv_id = conversation_id
else:
    # Start new conversation
    first_message = GPTMessage(role="user", content=prompt)
    conv_id = signal_archive_db.create_gpt_conversation(first_message)
    messages = signal_archive_db.get_messages_for_conversation(conv_id)

For continued conversations, we first persist the new user message, then retrieve the complete history. For new conversations, we create the conversation record (which automatically adds the first message), then retrieve it back. This ensures consistency—what we send to the API exactly matches what's stored in the database.

4.3 Response Handling and Storage

After receiving the AI's response, we store it as an assistant message:

gpt_response = gpt_api.gpt_completion(api_messages, model=model)
response_text = gpt_response.get("text", "Error: No text in response")

bot_message = GPTMessage(role="assistant", content=response_text)
signal_archive_db.add_message_to_conversation(conv_id, bot_message)

send_message(
    f"[conversation {conv_id}] {response_text}\n"
    f"cost: \${cost:.4f}, payer: {payer}"
)

The response always includes the conversation ID, making it easy for users to continue the conversation later. Including cost and payer information provides transparency in multi-user environments where API expenses are shared or attributed.

5. Multi-Modal Conversations: Inline Image References

5.1 The Challenge

Signal allows sending images as attachments, but these are ephemeral—they arrive with the message and aren't easily referenced later. For AI conversations, users often want to ask follow-up questions about an image discussed earlier, or reference images from the bot's archive in new conversations.

5.2 The imageid= Syntax

We implemented a lightweight markup syntax that lets users embed image references in their prompts:

gpt imageid=123 What's happening in this image?
gpt 42 imageid=123 imageid=456 Compare these two images

The syntax is intentionally simple—imageid= followed by a numeric ID. Multiple images can be included in a single prompt.

5.3 Implementation

Image references are resolved at request time through a two-stage process:

IMAGE_ID_REGEX = re.compile(r"imageid=(\d+)", re.IGNORECASE)

def _build_inline_image_content(prompt: str) -> tuple[list | str, list[int]]:
    """Convert imageid= references to OpenAI API image payloads."""

    image_ids = IMAGE_ID_REGEX.findall(prompt)
    if not image_ids:
        return prompt, []

    contents: list[dict] = []
    cleaned_prompt = IMAGE_ID_REGEX.sub("", prompt).strip()
    contents.append({"type": "text", "text": cleaned_prompt})

    embedded_ids: list[int] = []
    for raw_id in image_ids:
        image_id = int(raw_id)
        image_result = image_manager.get_image_by_id(image_id)
        if not image_result:
            raise ValueError(f"Image ID {image_id} not found")

        _, image_path = image_result
        image_bytes = image_manager.read_image_bytes(image_path)
        image_b64 = base64.b64encode(image_bytes).decode("utf-8")

        contents.append({
            "type": "image_url",
            "image_url": {"url": f"data:image/jpeg;base64,{image_b64}"}
        })
        embedded_ids.append(image_id)

    return contents, embedded_ids

The function extracts image IDs from the prompt, removes the imageid= markers from the text, loads each referenced image from disk, base64-encodes it, and constructs the multi-modal content structure expected by the OpenAI API.

5.4 Applying to Full Conversations

Since conversations may span multiple messages with image references, we apply this transformation to the entire message history:

def _prepare_messages_with_inline_images(
    messages: list[GPTMessage],
) -> tuple[list[GPTMessage], list[int]]:
    """Transform all messages, resolving image references."""

    prepared: list[GPTMessage] = []
    referenced_image_ids: list[int] = []

    for message in messages:
        content = message.content
        if message.role == "user" and isinstance(content, str):
            content, ids = _build_inline_image_content(content)
            referenced_image_ids.extend(ids)

        prepared.append(GPTMessage(role=message.role, content=content))

    return prepared, referenced_image_ids

This approach means the database stores the original imageid= references as plain text, while the actual image data is resolved fresh for each API call. This has several advantages:

1. Storage Efficiency: We don't duplicate image data in conversation history.
2. Image Updates: If an image is re-processed or corrected, subsequent conversation continuations automatically use the updated version.
3. Auditability: The stored conversation clearly shows which images were referenced.

6. Concurrency and Thread Safety

6.1 Threading Model

Each command runs in its own daemon thread to avoid blocking the main message processing loop:

def _process_gpt_command(text: str, command: str, model: GPTModel) -> bool:
    # ... validation ...

    current_user_context = gpt_api.get_user_context()

    def my_func():
        try:
            gpt_api.set_user_context(current_user_context)
            # ... conversation processing ...
        finally:
            gpt_api.clear_user_context()

    thread = threading.Thread(target=my_func)
    thread.daemon = True
    thread.start()
    return True

6.2 User Context Propagation

The system tracks which user initiated each request for cost attribution. Since this context is stored in thread-local storage, we must capture it before spawning the worker thread and restore it inside the thread:

current_user_context = gpt_api.get_user_context()

def my_func():
    try:
        gpt_api.set_user_context(current_user_context)
        # ... API calls use this context for billing ...
    finally:
        gpt_api.clear_user_context()

6.3 Database Connection Safety

SQLite connections are managed via context managers, ensuring proper cleanup even if exceptions occur:

with get_db_connection() as conn:
    cur = conn.cursor()
    # ... operations ...
    conn.commit()

Each database operation acquires its own connection, avoiding issues with SQLite's threading limitations while maintaining data consistency.

7. Practical Considerations

7.1 Conversation Length and Token Limits

As conversations grow, they consume more tokens per API call. The current implementation sends the complete history with each request, which can become expensive for long conversations. Production deployments might consider:

• Summarization: Periodically summarizing older messages to reduce token count.
• Windowing: Only sending the N most recent messages.
• Smart Truncation: Using the model to identify and retain the most relevant context.

7.2 Error Handling

The implementation includes robust error handling for common failure modes:

try:
    api_messages, embedded_images = _prepare_messages_with_inline_images(messages)
except ValueError as e:
    logger.error(f"Failed to attach images for GPT request: {e}")
    send_message(str(e))
    return

Invalid image references fail fast with clear error messages rather than sending malformed requests to the API.

7.3 User Experience

The response format provides all information users need to continue conversations:

[conversation 42] Here's my analysis of the image...
cost: \$0.0234, payer: jon

Users can immediately reference conversation 42 in their next message to continue the discussion.

8. Conclusion

Building persistent conversational AI over stateless messaging platforms requires careful consideration of data modeling, state management, and user experience. Our implementation demonstrates that a relatively simple database schema combined with thoughtful command parsing can provide a seamless multi-turn conversation experience.

The inline image reference system shows how platform limitations can be overcome through creative syntax design, allowing users to build rich multi-modal conversations without the messaging platform's native support.

This architecture has proven robust in production, handling concurrent users, long-running conversations, and multi-modal content while maintaining data consistency and providing transparency into API costs. The patterns described here are applicable beyond Signal to any stateless messaging platform where persistent AI conversations are desired.