TinyComputers.io (Posts about Machine Learning)

Teaching an LLM a Language It Has Never Seen

A.C. Jokela — Thu, 02 Apr 2026 13:00:00 GMT

Lattice is a programming language I designed. Its central feature is the phase system: every runtime value carries a mutability tag that transitions between states the way matter moves between liquid and solid. You declare a variable with flux (mutable) or fix (immutable). You freeze a value to make it immutable, thaw it to get a mutable copy, and sublimate it to make it permanently frozen. forge blocks let you build something mutably and have the result exit as immutable. None of this exists in any other language.

Lattice does not appear in Claude's training data. I designed the language after the knowledge cutoff. There is no Lattice source code on GitHub (other than my own repository). There are no Stack Overflow answers. There is no tutorial ecosystem, no community blog posts, no textbook chapters. The only documentation that exists is the code itself, a 38-chapter handbook I wrote, and three blog posts on this site.

Claude writes Lattice fluently. It writes correct programs using the phase system, the concurrency primitives, the module system, and the trait/impl pattern. It writes struct definitions with per-field phase annotations. It uses forge blocks and anneal expressions correctly. And it wrote a 4,955-line self-hosted compiler in Lattice, for Lattice: a complete tokenizer, parser, and bytecode code generator that reads .lat source files and emits .latc bytecode binaries.

The question is how any of this is possible when the model has never seen the language before.

The Rust Smell

The answer starts with syntax. Here is a Lattice function:

fn greet(name: String) -> String {
    return "Hello, ${name}!"
}

And here is the Rust equivalent:

fn greet(name: &str) -> String {
    format!("Hello, {name}!")
}

The fn keyword, the colon-separated type annotations, the -> return type, the curly braces: Claude has seen these patterns millions of times in Rust code. When it encounters them in Lattice, it doesn't need to learn a new syntax. It needs to recognize a familiar one.

This extends deep into the language. Lattice structs look like Rust structs:

struct Point {
    x: Float,
    y: Float
}

Lattice enums look like Rust enums:

enum Shape {
    Circle(Float),
    Rectangle(Float, Float)
}

Lattice match expressions look like Rust match expressions:

match shape {
    Shape::Circle(r) => pi() * r * r,
    Shape::Rectangle(w, h) => w * h,
    _ => 0.0
}

Lattice traits and impl blocks look like Rust traits and impl blocks:

trait Printable {
    fn display(self: any) -> String
}

impl Printable for Point {
    fn display(self: any) -> String {
        return "(${self.x}, ${self.y})"
    }
}

Closures use the same |params| body syntax. The .. range operator works the same way. The ? postfix operator propagates errors. for item in collection iterates. let binds variables. The structural similarity is pervasive enough that a model trained on Rust can parse and generate Lattice code without any Lattice-specific training.

I did not design Lattice to be AI-friendly. I designed it because Rust's syntax is good and I wanted to use it for a language with different semantics. But the side effect is that Claude can write Lattice from day one because the syntax activates the same neural pathways that Rust does. The model doesn't know it's writing a different language. It knows it's writing code that looks like Rust, and the structural patterns transfer.

The Phase System: Where Familiarity Ends

The Rust resemblance carries Claude through basic Lattice programs without difficulty. Where it gets interesting is the phase system, because this is where Lattice has no analog in any language Claude has seen.

In Rust, mutability is a static property: let mut x = 5; or let x = 5;. You decide at declaration time and the compiler enforces it. In Lattice, mutability is a runtime state that values transition through:

flux counter = 0          // mutable
counter = counter + 1     // allowed: counter is fluid

freeze(counter)           // transition: fluid → crystal
counter = counter + 1     // runtime error: counter is crystal

flux copy = thaw(counter) // get a mutable copy
copy = copy + 1           // allowed: copy is fluid

Claude handles this correctly. When I describe the phase system and provide examples, Claude generates code that uses flux and fix declarations appropriately, calls freeze() at the right points, and avoids mutating crystal values. The model maps flux to "mutable variable" and fix to "immutable variable" in its internal representation, and the transition functions (freeze, thaw) become explicit state changes that it tracks through the program.

The harder constructs are the ones with no familiar analog.

forge blocks are mutable construction zones whose output exits as immutable:

fix config = forge {
    flux c = {}
    c.host = "localhost"
    c.port = 8080
    c.debug = false
    c   // exits the forge block as crystal
}
// config is now crystal; cannot be modified

Claude gets this right because the pattern (build something mutably, freeze the result) maps to the builder pattern in Rust and other languages. The syntax is novel but the concept isn't.

anneal is harder. It temporarily thaws a crystal value into a mutable binding for the duration of a block, then re-freezes it:

fix settings = forge { flux s = {}; s.theme = "dark"; s }

anneal(settings) |s| {
    s.theme = "light"   // temporarily mutable
}
// settings is crystal again, with theme = "light"

Claude produces correct anneal code when given the semantics, but it occasionally generates patterns that would work in Rust (taking a &mut reference) but don't apply in Lattice (where anneal is the only way to modify a crystal value in place). The model's Rust intuitions are strong enough to produce syntactically valid Lattice but sometimes semantically incorrect programs, because it defaults to Rust's mutation model when the Lattice-specific construct is unfamiliar.

The reactive phase system is where Claude needs the most guidance. react, bond, and seed have no precedent in any mainstream language:

flux temperature = 72.0

react("temperature", fn(name, old_phase, new_phase) {
    print("${name} changed from ${old_phase} to ${new_phase}")
})

freeze(temperature)  // triggers the reaction callback

flux primary = "active"
flux mirror = "active"

bond("mirror", "primary", "sync")  // when primary changes phase, mirror follows

freeze(primary)  // mirror also freezes

Claude can produce these patterns when given the API, but it doesn't intuit them. It never suggests react or bond unprompted, because there's nothing in its training data that would trigger the association. These constructs must be taught explicitly. The Rust smell gets Claude through 80% of Lattice. The last 20% requires actual specification.

The Spectrum of Difficulty

Working with Claude on Lattice code over several months has revealed a clear gradient of difficulty:

Trivial (Rust transfer): Functions, structs, enums, match expressions, closures, for loops, string interpolation, module imports, error propagation with ?. Claude writes these correctly on the first attempt because they're syntactically identical to Rust.

Easy (new vocabulary, familiar concept): flux/fix declarations, freeze()/thaw() calls, basic phase checking. Claude maps these to mutable/immutable patterns it already knows. The vocabulary is new; the concept isn't.

Moderate (new pattern, teachable): forge blocks, anneal expressions, crystallize blocks, struct field-level phase annotations (alloy structs). These require explanation, but once Claude sees one or two examples, it generalizes correctly. The builder pattern and block-scoped mutation are close enough to existing patterns that the model bridges the gap.

Hard (no analog, requires specification): Reactive phase operations (react, bond, seed), phase pattern matching (fluid val =>, crystal val =>), the concurrency constraint that only crystal values can be sent on channels, strict mode's consumption semantics for freeze. Claude can use these but never invents them. They must be explicitly described.

The concurrency constraint is a good example of the "hard" category. In Lattice, data sent on a channel must be crystal:

let ch = Channel::new()
flux data = "mutable"

// ch.send(data)  // runtime error: cannot send fluid value

freeze(data)
ch.send(data)     // works: data is now crystal

This rule exists because crystal values are deeply immutable: they can't be modified by the sender after transmission, which eliminates data races structurally. Claude understands the concept (Rust has Send and Sync traits that serve a similar purpose), but it doesn't automatically apply Lattice's specific rule without being told. Left to its own devices, Claude will try to send fluid values on channels, because that's what you'd do in Go or Python. The constraint must be stated.

Strict mode (#mode strict at the top of a file) is another case where Claude needs explicit guidance. In strict mode, let is banned (you must use flux or fix), freeze() consumes the original binding (Rust-like move semantics), and crystal bindings cannot be assigned to at all, not even as a runtime error. Claude can write strict-mode Lattice, but it defaults to casual-mode patterns unless reminded. The model's prior is "permissive runtime" because that's what most dynamic languages are.

The gradient correlates exactly with how much the construct resembles something in Rust or another mainstream language. When the syntax is familiar, Claude's transfer learning handles it. When the concept is familiar but the syntax is new, one or two examples are enough. When both the syntax and the concept are novel, Claude needs the specification.

The Self-Hosted Compiler

The strongest evidence that Claude can deeply understand a language it was never trained on is latc.lat: a 4,955-line self-hosted compiler written in Lattice, for Lattice.

The compiler reads .lat source files and emits .latc bytecode binaries. It has twelve sections:

Opcode constant definitions (mapping all 100+ VM opcodes to integers)
Token stream and cursor helpers (peek, advance, expect, match_tok)
Compiler state management (save/restore for nested compilation)
Error reporting
Bytecode emit helpers (emit_byte, emit_jump, patch_jump, emit_loop)
Constant pool management (integers, floats, strings, closures)
Scope and variable resolution (begin_scope, end_scope, resolve_local, upvalue tracking)
Expression parsing (precedence climbing, binary/unary ops, calls, field access)
Statement compilation (let/flux/fix, if/while/for, return, match, try/catch)
Declaration compilation (functions, structs, enums, traits, impl blocks)
Binary serialization (writing the LATC file format with magic bytes, version header, chunk data)
Main entry point

Claude wrote this. Not "Claude assisted with this" or "Claude generated boilerplate for this." Claude wrote a recursive descent parser for Lattice's grammar, a bytecode compiler that emits correct opcodes for the phase system, and a binary serializer that produces files the C runtime can load and execute. The compiler bootstraps: you run it with the C-based clat interpreter, and it produces bytecode that the same interpreter executes.

The compiler itself uses Lattice's phase system for its own internal state. The compiler's mutable working data (the bytecode buffer, the constant pool, the local variable tracking arrays) is declared with flux:

flux code = []
flux c_lines = []
flux constants = []
flux local_name_arr = []
flux local_depth_arr = []
flux local_captured_arr = []
flux local_count = 0

This is the compiler eating its own dogfood. The mutable state that the compiler needs to build bytecode is declared using the same phase system that the compiler is compiling. The phase keywords aren't decorative here; they're structurally necessary because the compiler modifies these arrays on every opcode emission and scope transition.

The compiler has 118 functions across 12 sections, with 554 opcode references. It handles every construct in the language: flux/fix declarations, forge blocks, freeze/thaw/sublimate calls, anneal and crystallize expressions, struct and enum definitions with phase annotations, trait/impl blocks, match expressions with phase-aware pattern matching, structured concurrency with scope/spawn, channel operations, try/catch, defer, and the complete expression grammar with correct operator precedence.

Writing a self-hosted compiler requires understanding the language at every level simultaneously. The tokenizer must know every keyword, operator, and delimiter. The parser must handle every grammatical production, including the phase-specific constructs (forge, anneal, crystallize) that exist nowhere in Claude's training data. The code generator must emit the correct opcodes for phase transitions, reactive bindings, and structured concurrency. And the whole thing must be written in the language being compiled, which means Claude is writing Lattice to compile Lattice, using constructs it learned from examples rather than training data.

The compiler's serialization section writes the LATC binary format byte by byte:

fn serialize_latc(ch: any) {
    ser_buf = Buffer::new(0)

    // Header: "LATC" + version(1) + reserved(0)
    write_u8(76)    // 'L'
    write_u8(65)    // 'A'
    write_u8(84)    // 'T'
    write_u8(67)    // 'C'
    write_u16_le(1) // format version
    write_u16_le(0) // reserved

    serialize_chunk(ch)
    return ser_buf
}

This is not pattern matching against compiler source code from the training data. No Lattice compiler exists in the training data. Claude wrote a compiler for a language that has no prior art, in a language that has no prior art, producing a binary format that has no prior art. Every decision (the magic bytes, the chunk serialization order, the upvalue encoding) came from understanding the specification I provided and the runtime behavior of the C-based interpreter.

What I Actually Gave Claude

The teaching process was less structured than you might expect. There was no formal curriculum, no staged introduction of concepts, no carefully sequenced lesson plan. And I should be honest about the recursive nature of what happened: Claude Code was the primary tool for building Lattice itself. The language, the C implementation, the grammar, the runtime, the test suite, the handbook: all of it was built with Claude Code. I designed the language and directed the implementation, but Claude wrote the C, the LaTeX, and the example programs.

So the situation is: Claude wrote Lattice (the implementation), and then Claude wrote in Lattice (the programs and the self-hosted compiler). The model built the language and then learned the language it built. The "teaching material" that Claude uses to write Lattice code is documentation and examples that Claude itself produced in earlier sessions.

The artifacts:

The C implementation: ~80 source files, the parser, the VM, the phase system runtime. Built with Claude Code from my architectural direction.
A handbook: 38 chapters covering every feature, with worked examples. Written in LaTeX with Claude Code. This lives in a repository that Claude can read in subsequent sessions.
Example programs (examples/phase_demo.lat, examples/sorting.lat, examples/state_machine.lat) that demonstrate idiomatic Lattice. Written by Claude Code.
815 test files under AddressSanitizer that exercise every construct. Written by Claude Code.
An EBNF grammar reference as an appendix to the handbook.

When I work with Claude on Lattice code, I don't paste the entire handbook into the context window. Claude has access to the project directory. It reads files as needed. If I ask it to write a function that uses forge, it reads examples/phase_demo.lat or chapters/ch11-phases-explained.tex to see how forge works. If I ask it to add an opcode to the compiler, it reads include/stackopcode.h and src/stackvm.c to understand the existing instruction set.

The key insight: Claude doesn't need to be trained on a language to write it. It needs access to the specification and examples at inference time. And in this case, those specifications and examples were produced by Claude itself in prior sessions. The model's understanding is constructed on the fly from documentation in its context, not retrieved from weights. This is why the Rust resemblance matters so much: the syntax gives Claude a structural scaffold, and the specification (which Claude wrote) fills in the semantics.

This is also why the self-hosted compiler was possible. By the time Claude wrote latc.lat, it had already written the entire language implementation, the handbook, the test suite, and hundreds of example programs. The language had moved from "novel" to "familiar" through accumulated context, not through training. Each session built on the last. Each example reinforced the phase system's rules. By the time the compiler was attempted, Claude's working understanding of Lattice (constructed from its own prior output) was deep enough to write a 5,000-line program that correctly compiles the language. The model taught itself a language by building the language first.

Why Syntax Matters More Than Semantics

The Lattice experience suggests something counterintuitive about how LLMs interact with programming languages: syntax transfer is more powerful than semantic understanding.

Claude can write correct Lattice because Lattice looks like Rust. The semantic differences (phase system vs. ownership, runtime type checking vs. compile-time guarantees, garbage collection vs. RAII) are significant, but they don't prevent Claude from producing working code. The model generates syntactically valid Lattice from Rust patterns and then adjusts the semantics when corrected.

This has implications for language design. If you want AI tooling to support your language from day one, without waiting for it to appear in training data, design your syntax to rhyme with something popular. Lattice's resemblance to Rust wasn't designed for AI, but it is the reason AI can write it. A language with a radically different syntax (APL, Forth, J) would be much harder for Claude to learn from examples alone, even if the semantics were simpler.

The reverse is also true: a language with familiar syntax but deeply unfamiliar semantics (like Lattice's reactive phase system) will produce code that looks correct but occasionally behaves wrong. Claude's Rust intuitions are strong enough to generate valid-looking phase code, but the model sometimes falls back to Rust's mutation model when the Lattice-specific behavior is more constrained. The syntax transfers perfectly. The semantics require teaching.

Implications for Language Designers

If you're designing a new programming language in 2026, the AI tooling question is unavoidable. Your language won't have IDE plugins, autocompleters, or AI coding assistants on day one. The community doesn't exist yet. The training data doesn't include your language. Every other language your users work with has Copilot or Claude support. Yours doesn't.

Lattice suggests a strategy: make your syntax rhyme with something an LLM already knows.

This isn't about copying Rust. Lattice has genuinely novel semantics. The phase system, the reactive bindings, the alloy structs with per-field phase annotations: none of these exist in Rust. But they're expressed through syntax (keywords, braces, type annotations, block expressions) that maps directly to Rust's structural patterns. Claude can parse the syntax without help and learn the semantics from examples.

The alternative is designing a syntax so novel that LLMs can't bootstrap from existing knowledge. This is a legitimate design choice; some ideas genuinely need new notation. But the cost is high: your users won't get AI assistance until your language appears in training data, which requires the language to become popular first, which is harder without AI assistance. It's a chicken-and-egg problem that familiar syntax sidesteps.

The practical recommendation: novel semantics, familiar syntax. Invent the ideas. Borrow the notation. Let the LLM cross the bridge on syntax and learn the semantics on the other side.

What This Means for the "AI Writes Code" Conversation

The Lattice case study complicates the popular narrative about AI code generation in both directions.

For the optimists who say AI can learn anything: Claude cannot invent the reactive phase system. It cannot propose bond or seed or anneal without being told they exist. The novel constructs, the ones that make Lattice a genuinely different language rather than a Rust reskin, are invisible to the model until explicitly specified. AI transfer learning has limits, and those limits are at the boundaries of what the training data contains.

For the pessimists who say AI can only regurgitate training data: Claude wrote a 5,000-line self-hosted compiler for a language it has never seen. That is not regurgitation. The compiler produces correct bytecode for constructs (phase transitions, reactive bonds, per-field phase annotations) that exist in no other language. The model assembled knowledge from its understanding of compilers generally, Rust syntax specifically, and the Lattice specification I provided, and produced something genuinely new. Antirez called this "assembling knowledge" when he observed the same phenomenon with his Z80 emulator project. I think that's the right term.

The truth is somewhere that neither camp wants to occupy. LLMs can go far beyond their training data when the new territory is structurally adjacent to something they know. They cannot go beyond their training data when the new territory is structurally novel. The boundary between "adjacent" and "novel" is syntax. Familiar syntax is a bridge. Novel syntax is a wall. Novel semantics behind familiar syntax is a trap: the model crosses the bridge confidently and then occasionally falls.

Lattice exists in all three zones simultaneously. Its Rust-like surface lets Claude cross the bridge. Its phase system is the novel semantics behind familiar syntax. And the self-hosted compiler is proof that the bridge, once crossed, supports weight that no one expected.

I didn't set out to test the limits of LLM language understanding when I designed Lattice. I set out to build a programming language with a novel approach to mutability. The AI dimension was a side effect: I used Claude Code as my development tool because I use Claude Code for everything, and the language happened to be learnable because it happened to look like Rust. But the result is one of the more complete demonstrations of LLM transfer learning applied to a genuinely novel domain: not just writing programs in an unfamiliar language, but writing a compiler for that language, in that language, from a specification that exists nowhere in the training data.

The 4,955 lines of latc.lat are the proof that LLMs can go further than their training data when the conditions are right. The conditions are: familiar syntax, clear specification, accessible examples, and a human who knows when the model is wrong. Remove any one of those and the compiler doesn't get written. But with all four in place, the model produces something that works, that compiles, and that no human typed by hand.

Distilled Reasoning on Strix Halo: Running a Claude-Trained Thinking Model Locally

A.C. Jokela — Sun, 29 Mar 2026 14:00:00 GMT

There is a specific moment in the open-source LLM ecosystem that keeps recurring: someone takes a frontier model's outputs, uses them as training data for a smaller model, and publishes the result. The technique is called distillation, and it has been applied to coding ability, instruction following, and general knowledge. What is newer is distilling reasoning—the step-by-step chain-of-thought process that models like Claude use internally when working through complex problems.

Jackrong's Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled is one of the more interesting examples. It takes the Qwen3.5-27B base model and fine-tunes it on thousands of reasoning trajectories extracted from Claude 4.6 Opus. The result is a model that exposes its thinking process through <think> tags before delivering a final answer, mimicking the extended thinking behavior that Anthropic built into Claude natively. In 4-bit quantization, the entire model fits in about sixteen gigabytes.

I wanted to know two things. First, whether this kind of distilled reasoning actually works—whether a 27B model can meaningfully replicate the structured thinking of a model orders of magnitude larger. Second, whether the AMD Strix Halo APU, with its unified memory architecture and integrated RDNA 3.5 GPU, could run it at useful speeds. The answer to both turned out to be more nuanced than a simple yes or no.

The Hardware

The machine is the same AMD Ryzen AI MAX+ 395 that has appeared in several previous posts. It is an APU: CPU and GPU on the same die, sharing the same pool of LPDDR5X memory. There is no PCIe bus between the processor and the graphics engine. There is no dedicated VRAM to fill up. The GPU sees roughly 65GB of addressable memory out of the system's 122GB total, which means a 16GB quantized model loads without any of the memory pressure games you play on discrete GPU setups.

This matters for local LLM inference because the bottleneck for most language models is memory bandwidth, not compute. Tokens are generated one at a time, each requiring a full pass through the model's weights. The faster you can stream those weights from memory to the processing units, the faster you generate tokens. The Strix Halo's LPDDR5X provides roughly 120 GB/s of bandwidth to the unified memory pool. A discrete GPU like the RTX 4090 has 1 TB/s of bandwidth to its dedicated VRAM, but the Strix Halo never has to copy weights across a PCIe bus. For models that fit entirely in the GPU's addressable space, the unified architecture eliminates an entire class of overhead.

The system runs Ollama 0.17.6, which wraps llama.cpp and provides model management and an HTTP inference API. ROCm 7.2 handles the GPU compute layer, though Ollama's GGUF inference path is primarily CPU-based with GPU offloading for specific operations. The gfx1151 GPU target is not yet in the mainline PyTorch or llama.cpp kernel prebuilds, so HSA_OVERRIDE_GFX_VERSION=11.0.0 remains necessary to map it to the closest supported target (gfx1100, Navi 31).

The Model

The model's architecture is straightforward: Qwen3.5-27B, a 27 billion parameter transformer, fine-tuned via supervised learning on structured reasoning data. What makes it interesting is the training data. The creator assembled three datasets:

Opus-4.6-Reasoning-3000x-filtered: Three thousand reasoning trajectories extracted from Claude 4.6 Opus, filtered for quality.
claude-4.5-opus-high-reasoning-250x: Two hundred and fifty examples of high-intensity structured reasoning from an earlier Claude version.
Qwen3.5-reasoning-700x: Seven hundred step-by-step problem-solving examples.

The combined training signal teaches the model to produce output in a specific format: a <think> block containing the reasoning process, followed by a clean final answer. This is architecturally similar to what Anthropic does with Claude's extended thinking, except that Claude's thinking is a native capability of the model's training and architecture, while this is a behavior pattern learned through supervised fine-tuning on examples of that behavior.

The distinction matters, and I will come back to it.

The model is distributed in GGUF format, which is the standard for llama.cpp and Ollama. I used the Q4_K_M quantization, which compresses the model's weights from 16-bit floats to 4-bit integers with a mixed precision scheme that preserves more information in attention layers. The file is 15.4GB on disk. The model card reports 29-35 tokens per second on an RTX 3090; I was curious what the Strix Halo would deliver.

Setting It Up

Getting the model running took less than ten minutes. Download the GGUF file from HuggingFace:

mkdir -p ~/models/qwen35-reasoning
curl -L -o ~/models/qwen35-reasoning/model.gguf \
  'https://huggingface.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-GGUF/resolve/main/Qwen3.5-27B.Q4_K_M.gguf'

Note the filename. The HuggingFace repo is named Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled-GGUF, but the actual GGUF files inside follow a simpler naming scheme: Qwen3.5-27B.Q4_K_M.gguf. I wasted time trying to guess the full distilled name before checking the API.

Create an Ollama Modelfile that imports the local GGUF and sets inference parameters:

FROM /home/alex/models/qwen35-reasoning/model.gguf

PARAMETER temperature 0.6
PARAMETER top_p 0.95
PARAMETER num_ctx 8192
PARAMETER repeat_penalty 1.2
PARAMETER stop "<|endoftext|>"
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|eot_id|>"

SYSTEM "You are a deep-thinking AI assistant. For complex questions,
use <think>...</think> tags to show your reasoning process before
providing the final answer."

Then:

ollama create qwen35-reasoning -f Modelfile

Ollama copies the GGUF into its own blob store, parses the architecture metadata, and registers it as a runnable model. The whole process takes about a minute on local storage.

The Stop Token Problem

The first run produced correct output followed by infinite repetition. The model answered a calculus question perfectly, then appended "This gives us the final answer:" and repeated the entire solution, over and over, until it hit the context window limit. The previous MarkTechPost article that inspired this experiment did not mention this issue, likely because their test prompts were short enough that the repetition was not obvious.

The fix is explicit stop tokens in the Modelfile. Without them, the model does not know when to stop generating. This is a common issue with GGUF models imported into Ollama without a proper chat template: the model's native end-of-sequence tokens are not being interpreted by the inference engine. Adding <|endoftext|>, <|im_end|>, and <|eot_id|> as stop parameters catches the three most common EOS tokens used by Qwen and Llama-family models.

The repeat_penalty of 1.2 provides a second layer of defense by penalizing the model for reusing recent tokens. This helps but is not sufficient on its own. Without the stop tokens, the model can produce novel-but-meaningless text that avoids exact repetition while still degenerating into nonsense. More on this shortly.

Where It Works: Structured Problems

With the stop tokens in place, the model performs well on structured mathematical and analytical problems. I gave it a calculus question: find the derivative of x³sin(x) using the product rule.

The response was genuinely good. The model opened a <think> block, identified the two component functions, recalled the product rule formula, computed each derivative, and applied the rule. Then it closed the think block and produced a clean, well-formatted answer with LaTeX notation, step-by-step derivation, and a factored final form. The thinking trace was coherent and tracked the actual reasoning process. It was not filler; each line in the trace corresponded to a meaningful step.

Generation speed on the Strix Halo: 10.3 tokens per second. Not fast by cloud standards, but responsive enough for interactive use. You see the thinking appear in real time, which is surprisingly useful: you can watch the model work through the problem and catch errors before it commits to a final answer.

For structured problems—mathematics, code analysis, formal logic—the distilled reasoning is genuinely functional. The model identifies subproblems, works through them sequentially, and arrives at correct answers. The think tags provide transparency into the process that you do not get from a standard instruction-tuned model.

Where It Falls Apart: The River Crossing

I ran the classic wolf-goat-cabbage river crossing puzzle as a comparison test, the same prompt on both the distilled Qwen model and Claude Haiku 4.5 via the Anthropic API.

Claude Haiku returned a perfect, concise seven-step solution in 2.9 seconds. Two hundred and twenty-three tokens. The answer identified the critical insight (bring the goat back on one return trip), laid out the sequence clearly, and stopped.

The Qwen model started well. It correctly identified that the goat must go first, recognized the wolf-goat conflict at the destination, and identified the need to bring the goat back. Then, around step three of the solution, the model began editorializing. "Oh joy what fun times ahead us humans truly enjoy sometimes huh?!" it wrote, mid-solution. Within a few more sentences, the output had degenerated into an unbroken stream-of-consciousness rant that cascaded into a wall of increasingly disconnected words. Not repeated words—the repeat penalty prevented that—but a firehose of unique, semantically null text that continued until it filled the entire 8,192-token context window.

The output was, to use a technical term, unhinged. The model went from a correct partial solution to word salad in about two hundred tokens, and there was no recovery. The stop tokens could not save it because the model was not producing any end-of-sequence markers. It had entered a mode where it was generating fluent English syntax with zero semantic content, which is exactly the kind of failure that stop tokens and repeat penalties cannot catch.

What the Comparison Reveals

The numbers tell the story concisely:

	Claude Haiku 4.5	Qwen3.5-27B (Strix Halo)
Time	2.9 seconds	Hit 8K context limit
Speed	75.9 tok/s	~10 tok/s
Output	223 tokens, correct	Thousands of tokens, degenerated
Cost	$0.0009	Free

But the comparison is not really about speed or cost. It is about the difference between native reasoning and distilled reasoning.

Claude's extended thinking is a capability that emerges from the model's architecture and training at scale. The model has internalized what it means to reason through a problem, including knowing when to stop, when a line of reasoning is unproductive, and when to switch strategies. These are meta-cognitive skills that are extremely difficult to distill.

The Qwen model learned the format of reasoning—the think tags, the step-by-step structure, the pattern of stating subproblems and working through them—from three thousand examples. What it did not learn, and arguably cannot learn from supervised fine-tuning alone, is the judgment about when reasoning is going off the rails. A model that has truly internalized reasoning has implicit quality checks: it recognizes incoherence in its own output and corrects course. A model that has learned to mimic reasoning produces the surface pattern without the underlying self-monitoring.

This is visible in the failure mode. The model did not produce wrong reasoning. It produced no reasoning. It exited the reasoning pattern entirely and entered a generation mode that had nothing to do with the problem. A model with genuine reasoning capability would have recognized the incoherence and either corrected or terminated. The distilled model had no such circuit breaker.

The Economics

The cost comparison deserves its own section because it is often cited as the primary motivation for running local models.

The Claude Haiku API call cost nine-tenths of a cent. If you ran a thousand similar queries per day, you would spend about nine dollars. That is less than the electricity cost of running the Strix Halo for a day under load. The Strix Halo draws roughly 65 watts at idle and 150 watts under GPU inference load. At Minnesota's residential electricity rate of around twelve cents per kilowatt-hour, running inference eight hours a day costs about fourteen cents. But the hardware itself cost north of two thousand dollars. You would need to amortize that over thousands of hours of inference to reach cost parity with the API, and only if you value your debugging time at zero.

The economic case for local inference is not about per-query cost. It is about use cases where you need unlimited queries without metering, where data cannot leave your network, or where you want to experiment with model behavior without worrying about a bill. If you are evaluating a model's failure modes by running hundreds of adversarial prompts—which is exactly what I was doing—the local model is the right tool because you are not optimizing for answer quality. You are optimizing for the freedom to explore.

The Strix Halo as an Inference Platform

Ten tokens per second for a 27B Q4 model is respectable for an APU. It is not competitive with a discrete GPU: an RTX 3090 delivers 29-35 tokens per second on the same model, roughly three times faster. But the Strix Halo was not designed to compete with discrete GPUs on raw throughput.

What it offers instead is capacity. The unified memory pool means you can load models that would not fit on most consumer GPUs. A Q8_0 quantization of this same model would be 28.6GB, which exceeds the VRAM of an RTX 4090 (24GB) but fits comfortably in the Strix Halo's addressable space. You could load a 70B Q4 model (roughly 40GB) without any of the layer-splitting gymnastics required on multi-GPU setups. I have run Llama 3.1 70B Q4 on this machine, and while the generation speed drops to about 4-5 tokens per second, it runs without errors or memory pressure.

For a machine that also serves as a daily desktop, development workstation, and video generation server (it runs LTX-2.3 on the same hardware), the ability to casually load and test a 27B reasoning model without dedicated GPU infrastructure is the actual value proposition. You do not plan a session. You do not allocate resources. You type ollama run qwen35-reasoning and it works.

Lessons for the Blog Post Reader

If you want to replicate this setup, here is what I would emphasize:

The stop tokens are non-negotiable. Without explicit <|endoftext|>, <|im_end|>, and <|eot_id|> stop parameters in your Modelfile, the model will produce infinite output on many prompts. This is not documented in the model card and is not mentioned in the MarkTechPost article that covers this implementation. It is the single most important configuration detail.

The model is good at structured problems and bad at open-ended ones. Mathematics, code analysis, formal logic—anything where the reasoning has a clear structure and a definitive endpoint—works well. Open-ended problems, creative tasks, or anything requiring sustained coherent narrative are risky. The model can degenerate catastrophically and without warning.

A repeat penalty helps but does not solve the fundamental issue. Setting repeat_penalty to 1.2 prevents exact repetition loops but does not prevent the semantic degeneration I observed on the river crossing problem. The model simply produces unique garbage instead of repeated garbage.

Distillation captures form, not judgment. The think tags are real and useful. The step-by-step reasoning format works. What is missing is the implicit self-monitoring that frontier models have: the ability to recognize when their own output has become incoherent and to course-correct. This is probably the hardest thing to distill, because it is not present in the training examples. The examples show successful reasoning. They do not show the model catching and recovering from failed reasoning, because Claude's failed reasoning attempts are filtered out before the training data is assembled.

Where This Goes

The distilled reasoning model is, despite its failure modes, genuinely interesting. The <think> tags provide a form of transparency that standard instruction-tuned models lack. When the model is working correctly—which is most of the time on appropriate tasks—you get a window into the reasoning process that helps you evaluate the answer's quality before you act on it.

The failure mode is also instructive. It demonstrates, concretely, the gap between learning a behavior pattern and internalizing the capability that produces that pattern. Supervised fine-tuning on reasoning trajectories can teach a model to produce reasoning-shaped output, but it cannot, from three thousand examples, teach the model to actually reason in the way the source model does. That requires either far more training data, a different training methodology (reinforcement learning from reasoning feedback, perhaps), or simply a larger model with more capacity to internalize the underlying patterns.

For now, the practical advice is: use these models for what they are good at, know their failure modes, and do not trust the output on open-ended problems without reading the thinking trace. The trace is the feature. If the trace is coherent, the answer is probably good. If the trace starts to wander, stop reading and retry.

The model runs on my desk, generates ten tokens per second, costs nothing per query, and shows its work. For a sixteen-gigabyte download and ten minutes of setup time, that is a reasonable deal—as long as you know what you are buying.

Running a 22B Video Model on Four Tesla P40s

A.C. Jokela — Fri, 20 Mar 2026 13:00:00 GMT

LTX-Video 2.3 is a 22 billion parameter model that generates video from text prompts. It was designed for modern hardware: GPUs with bfloat16 support, high-bandwidth memory, and enough VRAM to hold the full model on one or two cards. The Tesla P40 has none of these things. It is a Pascal-generation GPU from 2016, with 24GB of GDDR5X per card, no native bfloat16, no Tensor Cores, and a PCIe 3.0 bus. It was built for data center inference workloads that no longer exist.

I have four of them in a rack-mount server in an unheated shop building in Minnesota. Together they provide 96GB of VRAM. The question was whether that 96GB, spread across four old cards, could run a model that was never meant to run on any of them.

The answer is yes, with significant caveats and a substantial amount of code to work around hardware limitations that the model's authors never anticipated.

The Problem

LTX-Video 2.3's transformer has 48 blocks. At fp16 precision, the model weights alone consume roughly 44GB. With the Gemma text encoder, the video VAE encoder/decoder, the spatial upsampler, and the audio components, the full pipeline needs more memory than any single P40 can provide. The model doesn't fit on one card. It doesn't fit on two. It barely fits on three, with no room for activations during inference.

Four cards at 24GB each gives 96GB total, which is enough for the weights with room for intermediate activations. But CUDA doesn't automatically spread a model across multiple GPUs. You have to tell it how.

The standard approach for multi-GPU inference is accelerate's dispatch_model, which automatically distributes model layers across available GPUs based on memory constraints. This works for the Gemma text encoder, which is a straightforward transformer. For the LTX transformer, it doesn't work, because the model has a custom forward pass with audio-video cross-attention that accelerate's automatic dispatch can't handle correctly. The model needs to move data between GPUs at specific points in the forward pass, and accelerate doesn't know where those points are.

The solution was manual pipeline parallelism: split the 48 transformer blocks evenly across four GPUs (12 blocks per card), keep the shared components (patchify projections, normalization, output projections) on GPU 0, and write a custom forward pass that moves tensors between devices at block boundaries.

The Precision Problem

Even with the model split across four cards, nothing worked on the first attempt. Or the fifth. Getting LTX-Video running on Pascal hardware was an iterative process, with Claude Code generating solutions and me testing them against the actual hardware. Each failure revealed another assumption the model made about the GPU it would run on. The feedback loop was brutal: load a 22B model across four GPUs, wait eight minutes for a test generation, get a black frame or a NaN error, diagnose which precision boundary caused it, generate a fix, and try again.

The first problem was bfloat16. The model weights are stored in bf16 format. Pascal GPUs cannot compute in bf16. PyTorch handles this silently for some operations by promoting to fp32, but other operations fail or produce garbage. The initial approach was the obvious one: monkey-patch torch.bfloat16 to redirect to torch.float16. This seemed to work at load time. The model loaded, the weights populated, no errors. Then the first forward pass produced NaN everywhere. The monkey-patch had corrupted the safetensors weight loading. The weights loaded as fp16 bit patterns interpreted as bf16 values, which is not the same thing. A bf16 value of 1.0 has a different bit pattern than an fp16 value of 1.0. Reinterpret one as the other and you get a number that's either wildly wrong or NaN.

The second attempt tried running everything in fp16 natively, converting weights properly during load. This got further: the model produced output that wasn't NaN. But the output was a solid green frame. The intermediate activations in the transformer blocks were overflowing fp16 range. Values above 65,504 become infinity in fp16, and the model's internal representations regularly exceed that during the attention and feedforward passes. The green frame was the model's attempt to decode latents that had been clipped to infinity at some point in the pipeline.

The working solution was to let the model builder properly convert weights from bf16 to fp16 on load, then run the entire computation pipeline in float32. The weights sit in memory as fp16 (saving space), but every computation promotes to fp32 before executing. This required patching F.linear to handle mixed dtype inputs:

_orig_linear = F.linear
def _mixed_linear(input, weight, bias=None):
    if input.dtype != weight.dtype:
        weight = weight.to(input.dtype)
        if bias is not None:
            bias = bias.to(input.dtype)
    return _orig_linear(input, weight, bias)
F.linear = _mixed_linear

The same pattern extends to every normalization function and every convolution operation. Layer norm, group norm, RMS norm, conv1d through conv_transpose3d: all patched to handle mixed dtypes and accumulate in float32. Without these patches, intermediate values overflow fp16 range (values above 65,504 become infinity) and the output is a black frame.

The Gemma Problem

The text encoder is Google's Gemma 3, a separate model that converts text prompts into embeddings the video transformer can condition on. Gemma's attention mechanism overflows when run in fp16 on Pascal hardware. The attention scores grow large enough to exceed fp16 range, producing NaN values that propagate through the rest of the pipeline.

The fix was running the entire Gemma encoder in float32. This uses more memory, but the text encoder only runs once per generation (to encode the prompt), and its weights can be freed from GPU memory before the transformer starts. The sequence is: load Gemma across all four GPUs using accelerate, encode the prompt in float32, delete the encoder, free the memory, then load the video transformer.

def encode_prompt_float32(prompt, model_ledger, device):
    model_ledger.dtype = torch.float32
    te = model_ledger.text_encoder()
    # Dispatch across all 4 GPUs for memory
    max_memory = get_balanced_memory(
        te, max_memory={i: "22GiB" for i in range(4)},
        no_split_module_classes=["Gemma3DecoderLayer"],
    )
    te = dispatch_model(te, device_map=device_map)
    hidden_states, attention_mask = te.encode(prompt)
    # Free GPU memory before transformer loads
    del te
    gc.collect()
    torch.cuda.empty_cache()

This load-encode-delete cycle is ugly but necessary. There isn't enough total memory to hold both Gemma and the video transformer simultaneously, even across four cards. The sequential approach works because each component only needs to exist during its phase of the pipeline.

The Pipeline

The generation runs in two stages, matching LTX-Video's distilled inference schedule.

Stage 1 generates a half-resolution latent video (e.g., 256x384) through 8 denoising steps. Each step runs the full 48-block transformer, with data moving across all four GPUs:

def patched_process(video, audio, perturbations):
    for i, block in enumerate(ltx.transformer_blocks):
        dev = block_devices[i]
        video = move_args_to_device(video, dev)
        audio = move_args_to_device(audio, dev)
        video, audio = block(video=video, audio=audio,
                             perturbations=perturbations)
    video = move_args_to_device(video, device0)
    audio = move_args_to_device(audio, device0)
    return video, audio

Every GPU boundary involves a tensor transfer across PCIe 3.0. With 12 blocks per GPU, there are 3 boundary crossings per denoising step (GPU 0 to 1, 1 to 2, 2 to 3), plus a final transfer back to GPU 0. With 8 denoising steps, that's 32 cross-device transfers per stage, each moving both video and audio state tensors. PCIe 3.0 x16 has a theoretical bandwidth of ~16 GB/s. The tensors being transferred are small relative to the bandwidth (attention states and activations, not full weight matrices), so the overhead is manageable. But it adds up.

Stage 1 takes roughly 4 minutes for 241 frames at 24 fps (a 10-second clip). The spatial upsampler then doubles the resolution. Stage 2 runs 3 more denoising steps at full resolution (512x768), taking roughly 6.5 minutes. The VAE decoder converts latents to pixels and generates the audio track in another 40 seconds.

Total generation time for a 10-second, 512x768 video with audio: approximately 18.5 minutes. For a 1-second clip (25 frames): about 8 minutes. For a 4-second clip (97 frames): about 10.5 minutes.

The Memory Layout

During inference, the four GPUs aren't loaded equally. GPU 0 carries extra weight because it hosts all the shared components (patchify projections, normalization layers, output projections) plus its 12 transformer blocks. The actual memory distribution:

GPU	VRAM Used	Role
0	10.8 GB	Shared components + blocks 0-11
1	9.3 GB	Blocks 12-23
2	9.3 GB	Blocks 24-35
3	9.3 GB	Blocks 36-47

That's 38.7 GB of the available 96 GB. The remaining 57 GB provides headroom for activations, KV cache growth, and the VAE decoder. There's enough margin that generation never OOMs, even at 241 frames.

The API

Running inference from the command line is fine for testing, but generating videos for blog content requires something more practical. I wrapped the generation script in a FastAPI server with an async job queue:

# Submit a text-to-video job
curl -X POST http://10.1.1.24:8585/jobs \
  -F "prompt=A cinematic flyover of a Zilog Z80 processor on a PCB" \
  -F "duration=10" -F "seed=42"

# Submit an image-to-video job
curl -X POST http://10.1.1.24:8585/jobs \
  -F "prompt=A fluffy orange cat dancing" \
  -F "duration=4" -F "image=@cat.jpg"

# Check status
curl http://10.1.1.24:8585/jobs/07420abb6d82

# Download result
curl http://10.1.1.24:8585/jobs/07420abb6d82/video -o output.mp4

Jobs queue and execute sequentially. The GPU can only handle one generation at a time, and the load-encode-delete cycle for Gemma means there's significant setup overhead per job. The API spawns each job as a subprocess, which gives clean GPU memory cleanup between runs. If a generation crashes (which happened frequently during development), the next job starts fresh.

The server supports both text-to-video and image-to-video. Image conditioning locks the first frame to a provided image and generates subsequent frames from it, which produces more controllable results for specific visual subjects. In practice, image-to-video is the more useful mode. Text-to-video gives the model complete creative freedom, which means the output is unpredictable. You might ask for a Z80 processor and get something that looks like a generic IC, or something that looks like a Z80, depending on the seed. Image-to-video lets you provide the exact first frame you want and the model animates from there. For blog content where visual accuracy matters, starting from a real photograph or a specific reference image gives consistently better results.

What the Output Looks Like

The video quality is genuinely good. LTX-Video 2.3 produces coherent motion, reasonable physics, and detailed textures. Here are three examples, generated entirely on the P40 server:

Text-to-video: "A cinematic flyover of a Zilog Z80 processor on a printed circuit board" (10 seconds, 18.5 minutes to generate)

Image-to-video: "A fluffy orange cat with a hat dancing" (4 seconds, 10.5 minutes to generate)

Text-to-video: "A cat sitting on a windowsill, sunlight streaming in" (1 second, 8 minutes to generate)

The model understands object permanence, lighting consistency, and basic spatial relationships. The Z80 flyover produces a recognizable IC package with surrounding components, proper lighting, and smooth camera movement.

The audio is a different story. LTX-Video 2.3 generates an audio track alongside the video, but the results are inconsistent. Prompts describing characters speaking produce odd ambient music instead of voices. Prompts describing environments produce vaguely appropriate soundscapes. The audio pipeline works mechanically (it generates real audio waveforms via a separate VAE decoder and vocoder), but the semantic connection between prompt and audio output is weak. For blog content, I'd likely strip the generated audio and add narration or music separately.

The 512x768 resolution at 24fps is usable for web content. It's not 4K. It's not going to replace stock footage for production video. But for blog hero images in motion, visual demonstrations, or supplementary content alongside text, it works.

What This Cost

The hardware cost is zero incremental. The four P40s and the server already existed for LLM inference. LTX-Video is an additional workload on the same hardware.

The electricity cost is modest. The server draws roughly 500W under full GPU load. An 18.5-minute generation (10-second video at full resolution) consumes about 0.15 kWh, roughly $0.024 at Minnesota residential rates. You could generate forty 10-second clips for a dollar.

The real cost was development time. Getting from "model downloaded" to "working generation pipeline" took many iterations across multiple sessions with Claude Code. Each precision-related failure mode (bf16 corruption, fp16 overflow, mixed-dtype kernel errors, NaN propagation through attention) required diagnosis, a hypothesis, a code change, and a test cycle that involved loading a 22B model across four GPUs. The feedback loop was slow. A single test takes 8 to 18 minutes to confirm whether a change worked. Many didn't.

The Broader Point

A 22 billion parameter video generation model was not designed to run on 2016 hardware. The authors assumed bf16, assumed modern attention kernels, assumed enough memory on one or two cards. None of those assumptions hold on the P40.

But the model runs anyway, because the underlying math doesn't actually require any of those features. Bfloat16 is a convenience, not a requirement; float32 computes the same function. Flash attention is an optimization, not a necessity; standard attention produces identical results. And 96GB across four cards is 96GB, regardless of whether it's cutting-edge HBM3 or decade-old GDDR5X.

The generation is slow. Eighteen minutes for ten seconds of video is not competitive with a single A100, which would finish the same job in under two minutes. The float32 computation pipeline roughly doubles the FLOPS required compared to the bf16 path the model was designed for, and the PCIe 3.0 transfers between four separate memory pools add latency that a single modern GPU with unified HBM would never incur. But competitive wasn't the point. The point was that four GPUs I bought on eBay for a thousand dollars total, sitting in a server in a shop building, can run a model that was released this month. The gap between "latest model" and "latest hardware" is not as wide as the spec sheets suggest, as long as you're willing to write the code that bridges it.

The P40 server was already paying for itself on LLM inference and TTS generation. Video generation is one more workload on a machine that I own, running models that I choose, on a schedule that I control. The 18-minute wait is the price of not asking anyone's permission.

Processing 51,000 Photos with AI on AMD Strix Halo

A.C. Jokela — Sat, 14 Mar 2026 17:00:00 GMT

I have roughly 20 years of photos sitting on a home fileserver. They span 2001 to 2020, shot on everything from a Minolta DiMAGE F100 to a Nikon D5100 to various iPhones over the years. A mix of 21,554 JPEGs and 29,860 Nikon RAW files (51,414 images total) organized in a Lightroom backup directory by year, month, and date. Most were shot handheld, many in a hurry. The kind of archive that accumulates when you take photos for two decades without ever going back to curate them.

The Lightroom catalog that once made sense of all this was long gone, lost to a drive migration somewhere around 2018. What remained was a directory tree of raw files with no organization beyond the date folders. No star ratings, no keywords, no collections. Just files. Thousands of them, some sideways, some crooked, all unlabeled.

I wanted to fix that. Not manually (I don't have a month to spend in Lightroom) but programmatically. The goals were straightforward: correct orientation issues, straighten crooked horizons, generate AI descriptions of every photo's content, and catalog the whole archive in a queryable database. The kind of batch processing job that would have been impractical five years ago but is now entirely doable with the right hardware and a weekend of scripting.

The Hardware

Two machines on the local network, each with a distinct role:

Machine	Role	Key Specs
Fileserver	NAS / photo storage	28TB RAID (`/md0`), 125GB RAM, NFS exports
GPU workstation	ML inference	AMD Ryzen AI Max+ 395, Radeon 8060S, 121GB RAM

The fileserver is a straightforward storage box. The interesting machine is the GPU workstation running an AMD Strix Halo APU, specifically the AI Max+ 395 with its integrated Radeon 8060S. I've written about this chip before, and it continues to impress for inference workloads. The RDNA 3.5 integrated GPU shares system memory, giving it access to 65.2 GB of VRAM without the typical constraints of a discrete card. For a model like BLIP that needs maybe 2 GB, that's absurdly generous, but it means you never have to think about VRAM budgets, which is a luxury when you're iterating on a processing pipeline.

The fileserver already had NFS configured, exporting /md0 to the local subnet. One mount command on the GPU workstation and both machines could see the same filesystem:

sudo mount -t nfs fileserver.localnet:/md0 /md0

No file copying, no rsync scripts, no staging directories. The photos live on the NAS and get processed in-place over the network. Gigabit Ethernet introduces some I/O overhead (each 25 MB NEF file takes 200–300ms to read across the wire), but for an overnight batch job, the simplicity of a single shared filesystem is worth the throughput trade-off. If this were a recurring workflow, I'd invest in 10GbE, but for a one-time archive processing run, gigabit got it done.

The Software Stack

Everything runs in a Python virtual environment on the GPU workstation:

PyTorch 2.9.1+rocm6.3: ML framework with AMD ROCm backend
BLIP (Salesforce/blip-image-captioning-large): vision-language model for image captioning
OpenCV 4.13: horizon detection via Canny edge detection and Hough transforms
rawpy 0.26.1: Nikon NEF/NRW decoding (wraps LibRaw)
piexif: EXIF metadata extraction for JPEGs
exiftool: EXIF extraction for RAW files (called as a subprocess)
SQLite: metadata and results database

The gfx1151 Situation

If you've followed my previous posts on Strix Halo, you know the drill. The Radeon 8060S reports as gfx1151 in ROCm, which is newer than what PyTorch's ROCm wheels officially target. The fix is the same environment variable override:

export HSA_OVERRIDE_GFX_VERSION=11.0.0

This maps the GPU to a generic gfx11 target. In practice, it works without issues, with no compute errors and no performance penalties. ROCm 6.16 on this machine also reports amdgcn-amd-amdhsa--gfx11-generic as a supported ISA, which is likely why the override works cleanly. I've been running production workloads with this flag for months now without incident.

The Processing Pipeline

Each photo passes through five stages: EXIF extraction, orientation correction, horizon detection and straightening, AI captioning, and finally saving the corrected image and cataloging everything in SQLite.

EXIF Metadata Extraction

For JPEGs, piexif reads the embedded EXIF data directly; it's a pure Python library that parses the binary EXIF structure without needing any external dependencies. For NEF/NRW files, piexif can't handle Nikon's proprietary container format, so I shell out to exiftool with JSON output (exiftool -json -n <file>). The -n flag is important; it returns numeric values instead of human-readable strings, which makes downstream processing much cleaner.

The extracted fields cover the full gamut: camera make and model, lens, dates, exposure settings (shutter speed, aperture, ISO, focal length), flash, white balance, metering mode, GPS coordinates, and the original orientation tag.

EXIF data is notoriously inconsistent across two decades of cameras. I'll come back to this; it became a debugging story of its own.

Orientation Correction

The EXIF orientation tag (values 1 through 8) encodes how the camera was held when the photo was taken. A value of 1 means the image is right-side up. A value of 6 means the camera was rotated 90 degrees clockwise. Value 3 means 180 degrees. Some values encode horizontal or vertical flips. The full matrix looks like this:

ops = {
    2: [Image.FLIP_LEFT_RIGHT],
    3: [Image.ROTATE_180],
    4: [Image.FLIP_TOP_BOTTOM],
    5: [Image.FLIP_LEFT_RIGHT, Image.ROTATE_270],
    6: [Image.ROTATE_270],
    7: [Image.FLIP_LEFT_RIGHT, Image.ROTATE_90],
    8: [Image.ROTATE_90],
}

Out of the 51,411 successfully processed photos, 8,797 (17.1%) needed orientation correction. The majority came from the Nikon D5100 and iPhone 4, both of which set the orientation tag but don't bake the rotation into the pixel data itself. Without this correction, nearly one in five photos would display sideways or upside-down in any viewer that doesn't respect EXIF orientation.

Here's what that looks like in practice. The raw pixel data from this iPhone photo is stored sideways; the camera recorded an EXIF orientation tag of 6, meaning "rotate 90 degrees clockwise to display correctly." Any viewer that ignores that tag renders the image on its side:

Before: raw pixel data (EXIF orientation 6, displayed sideways)

After: orientation corrected based on EXIF tag

Horizon Detection and Straightening

This stage uses classical computer vision, no neural network needed. The approach:

Downscale the image to 1200px on the long side for speed
Convert to grayscale, apply Gaussian blur
Run Canny edge detection
Crop to the vertical middle 50%, since the horizon is rarely at the extreme top or bottom of a frame
Apply the Hough Line Transform to find line segments, requiring a minimum length of one-quarter the image width
Filter to near-horizontal lines (within 20 degrees of level)
Compute a weighted average of the detected angles, weighted by line length

The key is the threshold window. If the detected angle is less than 0.5 degrees, it's not worth correcting, since you'd introduce interpolation artifacts for no visible benefit. If it's greater than 15 degrees, it's probably not a tilted horizon at all; it's either intentional composition or the algorithm latching onto a staircase railing. The correction itself uses cv2.warpAffine with Lanczos interpolation and a reflective border mode, followed by an inward crop to eliminate any border artifacts:

crop_factor = 1.0 / (cos(angle) + sin(angle) * (min(h, w) / max(h, w)))

The initial implementation used Canny edge detection and Hough line transforms, classical CV techniques from the 1980s. Fast, deterministic, 100ms per image. But it had a fatal flaw: it couldn't distinguish between a tilted horizon and a roofline receding toward a vanishing point. Architecture, roads, staircases, any strong line in the middle band of the image would register as a "tilted horizon," and the algorithm would dutifully rotate the image to "correct" it. In practice, this meant a significant number of photos were being made worse, not better.

The fix was to replace Hough line detection with semantic segmentation. SegFormer (nvidia/segformer-b2-finetuned-ade-512-512), trained on the ADE20K dataset, segments each image into 150 classes, including sky. The approach is simple: find the sky pixels, trace the bottom edge of the sky region, fit a line to that boundary, and measure its angle. If there's no sky (less than 5% of the image), or the sky boundary is too fragmented (fewer than 20 points), skip the correction entirely.

This eliminates false positives on indoor shots, close-ups, architecture, and anything without a visible sky. SegFormer runs on CPU at about 0.4 seconds per image; the model is only 25M parameters, so it doesn't need the GPU. The GPU stays dedicated to BLIP captioning.

Two examples from the corrected archive. This bridge over a river had a 2.68-degree clockwise tilt, and the bridge deck and far shore are visibly leveled:

Before: 2.68° clockwise tilt

After: horizon straightened via sky boundary detection

This rocky Lake Superior shore had a 3.85-degree clockwise tilt, and the far horizon is leveled:

Before: 3.85° clockwise tilt

After: horizon straightened

AI Captioning with BLIP

The Salesforce/blip-image-captioning-large model generates natural language descriptions of each photo. It runs in float16 on the Radeon 8060S. Each image is resized to a maximum of 1024px before inference. Beam search with 5 beams and a 75-token limit generates the caption:

output_ids = model.generate(
    **inputs,
    max_new_tokens=75,
    num_beams=5,
    early_stopping=True,
)

Caption inference takes about 0.5–0.7 seconds per image, consistent regardless of whether the input was a JPEG or a decoded NEF. The model handles a wide variety of subjects surprisingly well. Some examples from the archive:

"a brown and white dog standing next to a blue chair"
"two silos sitting in the middle of a field"
"a bird sitting on a branch of a tree"
"a wooden sign that says hoban road in front of some trees"
"a blurry photo of a car driving down a snowy road"
"a dog being groomed by a woman in a salon"

The captions tend toward a "there is a..." pattern, and they occasionally get details wrong (BLIP once described a photo of my living room as "a hotel lobby," which is generous). But for searchability and cataloging purposes, they're remarkably useful. Being able to query WHERE caption LIKE '%dog%' across 51,000 photos and get meaningful results is something that would have required manual tagging before models like BLIP existed. For an archive this size, "good enough" captions on every photo are vastly more useful than perfect captions on none of them.

Save and Catalog

Corrected images are saved as high-quality JPEGs (quality 92) to /md0/photos_processed/images/, mirroring the original directory structure. NEF and NRW files are converted to JPEG in the process; the corrected archive is a uniform format. All metadata flows into a SQLite database with WAL journaling, tracking 40+ fields per photo: every piece of EXIF data, processing flags (was orientation corrected? was the horizon straightened? by how many degrees?), the AI caption, file hashes, dimensions, and processing timestamps.

The database makes the archive queryable in ways that were never possible before:

-- What cameras did I use, and when?
SELECT camera_model, MIN(date_taken), MAX(date_taken), COUNT(*)
FROM photos GROUP BY camera_model ORDER BY COUNT(*) DESC;

-- Photos with GPS data
SELECT filename, caption, gps_latitude, gps_longitude
FROM photos WHERE gps_latitude IS NOT NULL;

-- How crooked were my photos, by camera?
SELECT camera_model, ROUND(AVG(ABS(horizon_angle_degrees)), 2) as avg_tilt, COUNT(*)
FROM photos WHERE horizon_corrected = 1
GROUP BY camera_model ORDER BY avg_tilt DESC;

The EXIF Tuple Bug

The first processing pass completed 51,414 photos, but with 2,146 errors. All of them were TypeError: type tuple doesn't define __round__ method. For a pipeline that had been running cleanly on thousands of Nikon D5100 and D60 photos, this was unexpected.

The root cause turned out to be a two-part problem with how certain budget cameras from the 2008–2012 era write EXIF rational numbers.

Part 1: Malformed Tuples

The EXIF standard stores rational numbers as (numerator, denominator) pairs. Most cameras follow this. But some, particularly a batch of older point-and-shoots, wrote the ExposureBiasValue field as a 4-element tuple like (36, 0, 18, 0) instead of the expected 2-element (36, 0).

My _rational_to_float helper only handled 2-tuples:

def _rational_to_float(val):
    if isinstance(val, tuple) and len(val) == 2:
        if val[1] == 0:
            return None
        return val[0] / val[1]
    return val  # passes through 4-tuples as raw tuples

When a 4-tuple fell through, the downstream round() call choked on it. The fix was simple: return None for any tuple that isn't a standard rational pair.

Part 2: None Propagation

Even after fixing Part 1, many of these same cameras had written (36, 0), a rational with a zero denominator. The function correctly returned None for division by zero, but the calling code then did round(None, 2), triggering the same TypeError with a slightly different message.

The fix was a _safe_round wrapper:

def _safe_round(val, digits=1):
    if val is None:
        return None
    try:
        return round(val, digits)
    except TypeError:
        return None

After both fixes, the second pass recovered all 2,143 photos. The remaining 3 errors were genuine file corruption: a truncated JPEG, a NEF that LibRaw couldn't parse, and a NEF with filesystem-level I/O errors. Probably bad sectors on the source drive. Those can't be fixed in code.

This is one of those bugs that only surfaces at scale. Run the pipeline on a hundred Nikon photos and everything works perfectly. Run it on 51,000 photos spanning 15 different camera models over 20 years, and every edge case in the EXIF spec comes out to play. The lesson, which I should have internalized long ago: never trust external data formats at scale without defensive parsing on every field. The EXIF spec is a suggestion, not a contract, and camera manufacturers have been interpreting it creatively since the early 2000s.

Resumability

A 15-hour batch job will inevitably need to be restarted: bugs, system updates, a random hound disconnects the magsafe power cord from my MacBook Pro. The script tracks progress in SQLite and skips completed files on restart:

def is_already_processed(conn, source_path):
    row = conn.execute(
        "SELECT id FROM photos WHERE source_path = ? AND error IS NULL",
        (source_path,),
    ).fetchone()
    return row is not None

Photos that failed with errors are intentionally not skipped, so fixing a bug and re-running automatically retries them. This made the EXIF debugging cycle painless: fix the parser, clear the failed rows from the database, relaunch, and only the 2,143 affected photos get reprocessed.

Performance

The pipeline sustained 1.0–1.8 photos per second, depending on file format:

Stage	Time per Photo
JPEG load	~10ms
NEF decode (rawpy)	~400ms
MD5 hash	~5ms (JPEG), ~100ms (NEF)
Horizon detection	~100ms
BLIP inference	~500–700ms
JPEG save	~50ms

BLIP inference dominates the runtime. NEF decoding is the second bottleneck; each RAW file is 20–30 MB and requires full demosaicing through LibRaw. The NFS overhead for reading large NEFs over gigabit Ethernet is noticeable but not the primary constraint.

Total wall time: 15.5 hours across two passes for 51,414 photos. The BLIP model uses roughly 2 GB of the 65.2 GB available VRAM on the Strix Halo. Memory was never a concern.

Final Results

Metric	Count	Percentage
Total photos	51,414	100%
Successfully processed	51,411	99.99%
Orientation corrected	8,797	17.1%
Horizon straightened	15,251	29.7%
AI captioned	51,411	99.99%
Unrecoverable errors	3	0.006%

The top cameras in the archive tell the story of 20 years of gear:

Camera	Photos
Nikon D5100	24,073
Nikon D60	8,734
iPhone 4	2,664
Nikon D3100	1,698
Panasonic DMC-FX07	975
Minolta DiMAGE F100	870
iPad	803
iPhone 5s	698
Samsung SCH-I500	645

The output lives on the NAS:

Corrected images: /md0/photos_processed/images/, 51,411 JPEGs preserving the original year/month/date folder structure, all NEFs converted, all orientation and horizon corrections applied.
SQLite database: /md0/photos_processed/photos.db, 40+ fields per photo with full EXIF metadata, processing results, and AI-generated captions.
Processing log: /md0/photos_processed/processing.log, timestamped record of the entire run.

Takeaways

AMD's Strix Halo continues to earn its keep for ML inference. The HSA_OVERRIDE_GFX_VERSION=11.0.0 workaround remains necessary, but once set, PyTorch and ROCm run without complaints. The 65 GB shared VRAM pool means you can load models without thinking about memory budgets, a workflow advantage that's easy to underestimate until you've experienced it.

Classical computer vision still has its place. The horizon detection pipeline uses Canny edge detection and Hough transforms, techniques from the 1980s. No training data, no GPU needed, deterministic results, and the whole thing runs in 100ms per image. For geometric corrections on photographic images, you don't need a neural network. You need line detection.

EXIF is a minefield. Twenty years of cameras from different manufacturers means every edge case in the spec gets exercised. Tuple lengths vary, denominators are zero, fields are missing or repurposed. If you're parsing EXIF at scale, assume nothing about the data's shape and validate everything.

Resumability is non-negotiable for long-running jobs. Tracking progress in the database and skipping completed work made it trivial to iterate on bugs. Without this, every fix would mean reprocessing 51,000 photos from scratch.

NFS over gigabit is fine for batch processing. Not optimal, but for an overnight job, the network overhead from NAS-attached storage is acceptable. The real bottleneck was ML inference at 0.6 seconds per photo. If I were doing this regularly, 10GbE would be worth the upgrade, but for a one-time archive processing run, gigabit got the job done.

The whole project, from first SSH to final database entry, took about a day of wall time, most of which was unattended processing. The scripting itself was maybe three hours of work. Twenty years of photos, cataloged and corrected overnight. Not bad for a Strix Halo and some Python. The full source is available on GitHub.

What I didn't expect was how useful the database would be after the fact. Being able to ask "show me every photo I took with the D5100 at ISO 3200 or higher" or "find photos with GPS data from 2015" turns a pile of files into something that actually tells a story. The AI captions add another dimension; I can now search my own photo archive by content, not just metadata. It's the kind of capability that makes you wonder why photo management software hasn't done this for years. The models have been available. The hardware has been affordable. Someone just needed to wire it together.

The Real Cost of Running Qwen TTS Locally: Three Machines Compared

A.C. Jokela — Thu, 12 Mar 2026 14:00:00 GMT

Every post on this site has an audio version. A small player at the top, a few minutes of narration, generated entirely on local hardware. No cloud API, no per-character fees, no data leaving the network. I wrote about setting up the pipeline on AMD Strix Halo earlier this year, and the system has been running in production since, generating narrations for new posts, regenerating old ones when I revise them, and occasionally processing long-form content that would cost real money through Google Cloud TTS or ElevenLabs.

But I now have three machines capable of running Qwen3-TTS, and they could not be more different from each other. An Apple M3 Max laptop. An AMD Ryzen AI MAX+ 395 mini desktop with integrated Radeon graphics. And a four-GPU Tesla P40 server built from decade-old enterprise hardware bought on eBay. Three different silicon vendors, three different compute backends (MPS, ROCm, and CUDA) running the same model on the same text.

The question I wanted to answer is simple: how do they actually compare? Not on paper. Not in theoretical FLOPS. In wall-clock time, generating real audio from a real blog post.

The answer turned out to be more interesting than I expected, because the numbers tell a story about hardware architecture that raw specifications completely miss.

The Setup

The model is Qwen3-TTS-12Hz-1.7B-CustomVoice, a 1.7 billion parameter autoregressive text-to-speech model from Alibaba's Qwen team. It generates natural-sounding speech with multiple speaker voices. I use the Eric voice for all blog narrations: clear, professional, well-paced for technical content.

The three machines:

Apple M3 Max, a MacBook Pro with Apple's M3 Max chip. 14 CPU cores, 30 GPU cores, 64GB unified memory. The GPU runs through PyTorch's MPS (Metal Performance Shaders) backend. This is my daily driver laptop, and it generates TTS when I am writing and editing posts.

AMD Radeon 8060S, a Bosgame M5 mini desktop running AMD's Ryzen AI MAX+ 395. This is a Strix Halo APU with integrated RDNA 3.5 graphics, not a discrete GPU. It shares 128GB of DDR5 system memory with the CPU, with roughly 96GB addressable as VRAM. The GPU runs through ROCm 7.2 with PyTorch 2.9.1. The gfx1151 architecture requires specific PyTorch wheels from AMD's pre-release index and several environment variable overrides to function. I wrote a full setup guide for this machine.

NVIDIA Tesla P40, a 2U rack-mount server with four Tesla P40 GPUs, each with 24GB of GDDR5X. Pascal architecture from 2016. Compute capability 6.1. No Tensor Cores, no native bfloat16 support. The benchmark uses a single P40, since Qwen TTS runs on one GPU. This machine lives in an unheated shop building in Minnesota and screams through the winter when the BMC misinterprets sub-zero ambient temperatures as a hardware malfunction.

All three machines run the same model checkpoint, the same text input, and the same speaker voice. The only differences are the silicon and the compute backend.

The Benchmark

I used a standardized 2,411-character passage, five paragraphs on the Jevons Paradox, dense enough to exercise the model's prosody and pacing on real written content. Each machine ran three consecutive generations from the same loaded model, producing roughly three minutes of audio per run. The first run includes kernel compilation and cache warmup; subsequent runs reflect steady-state performance.

The metric that matters is Real-Time Factor (RTF): how many seconds of wall-clock time it takes to generate one second of audio. An RTF of 1.0 means the model generates audio at exactly real-time speed. Below 1.0 is faster than real-time. Above 1.0 means you are waiting.

Individual Runs

Apple M3 Max (MPS)

Run	Generation Time	Audio Length	RTF
1	698.5s	197.7s	3.53
2	533.1s	184.2s	2.89
3	447.8s	179.2s	2.50
Average	559.8s	187.0s	2.97

AMD Radeon 8060S (ROCm)

Run	Generation Time	Audio Length	RTF
1	729.2s	173.6s	4.20
2	460.0s	204.8s	2.25
3	548.2s	214.2s	2.56
Average	579.1s	197.5s	3.00

NVIDIA Tesla P40 (CUDA)

Run	Generation Time	Audio Length	RTF
1	1511.4s	204.1s	7.41
2	1225.7s	171.6s	7.14
3	1537.2s	206.7s	7.44
Average	1424.8s	194.1s	7.33

Summary

Machine	GPU	Avg RTF	Best RTF	Avg Gen Time
MacBook Pro	M3 Max (MPS)	2.97	2.50	559.8s
Bosgame M5	Radeon 8060S (ROCm)	3.00	2.25	579.1s
Penguin 2U	Tesla P40 (CUDA)	7.33	7.14	1424.8s

What the Numbers Mean

The headline result is that the M3 Max and Radeon 8060S are essentially tied, and the Tesla P40 is roughly 2.4 times slower than both. But that summary hides the interesting details.

The Warmup Effect Is Massive

On both the M3 Max and the Radeon 8060S, the first run is dramatically slower than subsequent runs. The M3 Max goes from RTF 3.53 on run 1 to RTF 2.50 on run 3, a 29% improvement. The AMD shows an even larger swing: RTF 4.20 on run 1 dropping to RTF 2.25 on run 2, a 46% improvement.

This is kernel compilation. Both MPS and ROCm compile GPU kernels on first use and cache them for subsequent calls. The Qwen TTS model hits a wide variety of kernel shapes during autoregressive generation (different sequence lengths, different attention patterns) and each new shape triggers a compilation on the first encounter. By run 2, most of the common shapes are cached, and performance stabilizes.

The P40 shows almost no warmup effect. RTF 7.41 on run 1, 7.14 on run 2, 7.44 on run 3. CUDA's kernel compilation is faster and more mature, so the overhead is absorbed within the first few seconds rather than spread across the entire run. But this maturity does not translate into faster inference; CUDA compiles faster, but the P40's hardware is fundamentally slower at the operations this model requires.

This has a practical implication that matters: short benchmarks on MPS and ROCm are misleading. I initially ran a quick 276-character test on all three machines before doing the full benchmark. The short test showed the AMD at RTF 9.20, almost identical to the P40's RTF 10.01, and far behind the M3 Max's RTF 2.84. That result nearly led me to conclude the AMD was performing as poorly as decade-old hardware. The longer benchmark, with its warmup effect amortized across more generation, revealed the truth: the AMD is just as fast as the M3 Max once the kernels are cached. If I had stopped at the short test, I would have drawn exactly the wrong conclusion.

Why the P40 Is So Slow

The Tesla P40 is a Pascal-generation GPU from 2016. It has 3,840 CUDA cores and 24GB of GDDR5X memory. On paper, it should be competitive; 12 TFLOPS of FP32 compute is not trivial. And for LLM inference through Ollama, the P40 performs remarkably well, outperforming quad T4 instances on models up to 8B parameters.

TTS is a different workload. Qwen3-TTS is an autoregressive transformer that generates audio tokens one at a time, each conditioned on all previous tokens. This means the inference is heavily memory-bandwidth bound during the decoding phase, and compute-bound during the attention and feedforward passes. The model is distributed in bfloat16 precision, which the P40 cannot compute natively; Pascal predates bfloat16 support entirely. PyTorch silently promotes bf16 operations to fp32 on the P40, roughly doubling the computation per operation and halving the effective throughput.

The P40 also lacks the SDPA (Scaled Dot-Product Attention) hardware acceleration that newer architectures provide. On the M3 Max, MPS routes attention through Metal's optimized primitives. On the AMD, ROCm's AOTriton provides experimental flash attention support. On the P40, attention runs through standard CUDA kernels without any of these accelerations. For a model that generates thousands of autoregressive steps per audio clip, each involving a full attention pass over the growing sequence, this compounds dramatically.

The P40 is not bad hardware. It is excellent hardware for the workloads it was designed for: batch inference on quantized LLMs where its 24GB of VRAM per card creates a memory advantage. But autoregressive TTS in bfloat16 hits every one of its architectural weaknesses simultaneously.

Unified Memory Wins This Workload

Both the M3 Max and the Radeon 8060S use unified memory architectures, where the CPU and GPU share the same physical memory pool. The M3 Max has 64GB of unified LPDDR5. The Radeon 8060S shares 128GB of DDR5 with the CPU, with roughly 96GB addressable as VRAM.

For a 1.7B parameter model in bf16, the weights occupy roughly 3.4GB. The model fits comfortably on all three machines. But the autoregressive generation pattern creates a stream of intermediate activations (KV cache entries, attention scores, feedforward intermediates) that grow with the sequence length. On a unified memory architecture, these intermediates exist in the same memory space as the model weights, avoiding any PCIe transfer overhead. On the P40, every interaction between CPU and GPU crosses a PCIe 3.0 bus.

For LLM inference, where the bottleneck is token generation throughput and the KV cache fits in VRAM, the P40's discrete memory is fine. For TTS, where the model generates hundreds of audio tokens per second of speech and the attention window grows continuously, the memory access pattern favors unified architectures.

This is not a universal statement about unified versus discrete memory. A modern discrete GPU with HBM2e or GDDR6X and PCIe 4.0 or 5.0 would likely outperform both the M3 Max and the Radeon 8060S on this workload. The P40's problem is not that its memory is discrete; it is that its memory is slow and its bus is narrow by 2026 standards.

The Model Architecture Question

While benchmarking Qwen TTS, I also ran a quick comparison with F5-TTS on the AMD machine to sanity-check the results. F5-TTS is a flow-matching model, fundamentally different from Qwen's autoregressive approach. Where Qwen generates audio tokens sequentially, each conditioned on all previous tokens, F5 generates audio in parallel through an iterative refinement process.

The difference is stark. On the same Radeon 8060S, the same text, the same hardware:

Model	Generation Time	Audio Length	RTF
Qwen3-TTS	579.1s (avg)	197.5s	3.00
F5-TTS	17.4s	27.2s	0.64

F5-TTS is faster than real-time. Qwen3-TTS takes three times longer than the audio it produces. On normalized terms, F5 is roughly five times faster than Qwen at steady-state, and the gap widens on shorter content where Qwen's warmup overhead is proportionally larger.

This is not an apples-to-apples quality comparison. Qwen3-TTS generally produces more natural prosody, better handling of complex sentence structures, and more consistent speaker identity across long passages. F5-TTS is excellent but can occasionally drift in voice character or pacing on very long content. For blog narration, both are well above the threshold of "good enough," and the quality difference is smaller than you might expect given the architectural gap.

The point is that hardware is only half the story. The choice of model architecture can matter more than the choice of GPU. A flow-matching model on integrated AMD graphics outperforms an autoregressive model on Apple's best laptop silicon by a wide margin. If generation speed is the constraint, switching models gains more than switching hardware.

What This Costs in Practice

The abstract benchmark numbers translate into concrete time and electricity costs when you are generating audio for a library of blog posts.

A typical TinyComputers post runs 3,000 to 5,000 words, producing 15 to 25 minutes of narrated audio. At steady-state RTF:

Machine	15 min audio	25 min audio	System Power
M3 Max	~38 min	~63 min	~50W
Radeon 8060S	~38 min	~63 min	~100W
Tesla P40	~110 min	~183 min	~400W

The M3 Max and Radeon 8060S are tied on generation time, but the M3 Max draws roughly half the system power. For a single post, the electricity cost difference is negligible, a fraction of a cent. For batch processing a backlog of thirty posts, the M3 Max costs about \$0.18 in electricity versus \$0.36 for the AMD and \$3.50 for the P40.

None of these numbers are alarming. Even the P40, at nearly two and a half hours per post and 400 watts from the wall, costs under fifteen cents in electricity per narration at Minnesota residential rates. The equivalent Google Cloud TTS job would cost \$4 to \$16 per post depending on the voice quality tier.

To put cloud costs in perspective: I recently ran a fiction novel through Google's Chirp3-HD voice: 82,000 words, roughly 500,000 characters of text plus SSML markup. The bill came to \$17.25 at Google's rate of \$30 per million characters. That is not unreasonable for a one-off project, but it adds up quickly if you are generating audio regularly. The entire library of TinyComputers narrations (dozens of posts, hours of audio) has cost me nothing beyond the electricity to run the machines I already own. The economics of local TTS are favorable on every machine in the comparison.

The real cost is time. If I am generating audio for a single new post, I start it on whichever machine is idle and check back in an hour. If I am regenerating audio for twenty posts after changing the speaker voice or updating the pipeline, the M3 Max or AMD will finish overnight. The P40 would take most of a weekend.

The Right Machine for the Job

After running these benchmarks, my workflow has shifted. The M3 Max is the default for new post narration; it is fast, quiet, and I am usually sitting in front of it when I finish writing. The AMD handles batch jobs and overnight processing, where its slightly higher power draw does not matter and its equivalent speed makes it interchangeable with the Mac. The P40 server is reserved for what it does best: running large language models through Ollama, where its 96GB of aggregate VRAM gives it an advantage that neither the Mac nor the AMD can match.

The P40 can still generate TTS in a pinch, and it does; when both other machines are occupied, I will queue a job on the P40 and accept the longer wait. But for a workload that is inherently autoregressive, memory-bandwidth sensitive, and dependent on bf16 precision, a ten-year-old Pascal GPU is the wrong tool.

What surprised me most is how well the AMD performs. The Radeon 8060S is an integrated GPU sharing system memory with the CPU. It has no HBM, no dedicated VRAM, no NVLink. Its ROCm software stack requires environment variable hacks, pre-release PyTorch wheels, and a GFX version override to function at all. And yet, once the kernels warm up, it matches Apple's best laptop silicon stride for stride. The raw hardware is there: 40 RDNA 3.5 compute units with access to a deep pool of DDR5 memory. The software just needs to get out of the way, and on run 2 and beyond, it does.

Lessons

Three takeaways from this exercise that generalize beyond TTS:

Short benchmarks lie. Kernel compilation overhead on MPS and ROCm is large enough to dominate a short test. If you are evaluating a new model on non-CUDA hardware, run it at least twice before drawing conclusions. The first run is measuring the software stack, not the hardware.

Architecture matters more than clock speed. The P40 has more raw FLOPS than the Radeon 8060S. It does not matter. The P40 lacks native bf16, lacks efficient attention primitives, and sits behind a PCIe 3.0 bus. The Radeon has all three, and ties a chip designed by Apple's custom silicon team. For autoregressive models, the architectural fit between model and hardware dominates everything else.

Model choice can outweigh hardware choice. F5-TTS running on the weakest GPU in this comparison is five times faster than Qwen3-TTS running on the strongest. If your constraint is generation speed and you can accept a modest quality trade-off, switching to a flow-matching architecture gains more than any hardware upgrade short of a data center GPU.

The audio player at the top of each post on this site represents a few minutes of machine time on one of these three machines. Which machine generated it depends on the day, the workload, and what else is running. The listener cannot tell the difference. The audio sounds the same regardless of whether it was generated on a laptop, a mini desktop, or a rack-mount server in a cold Minnesota shop. That is the real benchmark: not which machine is fastest, but that all three are fast enough.

Discretizing Continuous ML Models: Offline Ballistic Coefficient Corrections via Lookup Table Approximation

A.C. Jokela — Wed, 28 Jan 2026 14:30:00 GMT

Abstract

Machine learning models for ballistic coefficient (BC) correction have demonstrated significant improvements in trajectory prediction accuracy by capturing velocity-dependent drag variations that traditional constant-BC assumptions cannot model. However, deploying such models in field conditions presents challenges: network connectivity requirements, latency constraints, and computational overhead on resource-limited devices. This paper presents a methodology for discretizing continuous ML models into offline lookup tables, specifically addressing the problem of ballistic coefficient corrections across the flight envelope. We construct caliber-specific 5-dimensional lookup tables (BC5D) indexed by bullet weight, base BC, muzzle velocity, instantaneous velocity, and drag model type. Our approach samples the continuous ML function at fixed intervals and relies on piecewise-linear interpolation for queries between sample points. Empirical evaluation demonstrates that this discretization achieves velocity predictions within 5% of the continuous ML model through supersonic and early transonic regimes, with predictable divergence of 10-15% in deep transonic regions (Mach 0.8-1.2) where the underlying physics exhibit pronounced non-linearities. We argue that this accuracy-connectivity trade-off represents a practical compromise for field deployment, analogous to the relationship between analog signals and digital sampling in audio engineering.

1. Introduction and Thesis

The ballistic coefficient (BC) serves as the primary aerodynamic descriptor for projectile flight, encoding the bullet's ability to overcome air resistance into a single dimensionless quantity. Traditionally, manufacturers publish BC values measured under specific conditions (typically referenced to standard atmospheric density at sea level) and these values are treated as constants throughout the trajectory calculation. This simplification, while computationally convenient, ignores a well-documented physical reality: drag characteristics vary substantially with velocity, particularly as projectiles decelerate through transonic regimes where the relationship between Mach number and drag coefficient undergoes rapid, non-linear transitions [1, 2].

Machine learning approaches have emerged as a promising solution to this limitation. By training models on empirical drag data (obtained through Doppler radar tracking, spark range measurements, or computational fluid dynamics simulations) researchers can capture the complex, velocity-dependent nature of aerodynamic drag with greater fidelity than constant-BC assumptions permit [3, 4]. These ML models accept multiple input parameters (bullet geometry, muzzle velocity, current velocity, atmospheric conditions) and output a correction factor that adjusts the published BC to reflect instantaneous flight conditions.

However, ML model deployment introduces practical constraints that conflict with many real-world use cases. Precision shooting applications often occur in environments lacking reliable network connectivity. Mobile devices and embedded systems may lack the computational resources for real-time model inference. Latency requirements for interactive ballistics calculators may preclude round-trip API calls to remote servers. These constraints motivate investigation into methods for deploying ML-derived insights without the ML infrastructure.

Thesis: Continuous machine learning models for ballistic coefficient correction can be effectively discretized into offline lookup tables that preserve the essential predictive improvements while eliminating connectivity and computational dependencies. The discretization introduces a piecewise-linear approximation that follows the general trend of the continuous model but exhibits stair-step behavior at sample boundaries, a trade-off analogous to digital audio sampling, where sufficiently fine discretization renders the steps imperceptible for practical applications.

This paper makes three primary contributions:

A methodology for constructing caliber-specific 5-dimensional BC correction tables from continuous ML models
Empirical analysis of approximation fidelity across the velocity envelope, with particular attention to transonic degradation
A practical deployment architecture enabling offline operation while maintaining compatibility with online systems

2. Background and Related Work

2.1 Ballistic Coefficient Fundamentals

The ballistic coefficient, as formalized by Ingalls and later refined by the Sporting Arms and Ammunition Manufacturers' Institute (SAAMI), relates a projectile's drag characteristics to a standard reference projectile [5]. The G1 and G7 drag models, representing flat-base and boat-tail projectile shapes respectively, define these reference functions. A projectile's BC expresses the ratio of its sectional density to its form factor relative to the standard:

$$BC = \frac{SD}{i} = \frac{m/d^2}{C_D/C_{D_{ref}}}$$

where $m$ is mass, $d$ is diameter, $C_D$ is the projectile's drag coefficient, and $C_{D_{ref}}$ is the reference projectile's drag coefficient at the same Mach number [6].

The critical insight motivating this work is that the form factor $i$ is not constant; it varies with Mach number, particularly in the transonic regime (Mach 0.8-1.2) where shock wave formation and boundary layer interactions produce complex aerodynamic effects [7]. Modern Doppler radar measurements have quantified these variations, revealing that effective BC can change by 20-40% between supersonic cruise and transonic deceleration [8].

2.2 Model Compression and Quantization

The challenge of deploying complex models in resource-constrained environments has driven extensive research in model compression techniques. Neural network quantization reduces model precision from 32-bit floating point to lower bit widths (16-bit, 8-bit, or even binary), achieving 4-32x compression with modest accuracy degradation [9, 10]. Knowledge distillation trains smaller "student" models to mimic larger "teacher" models, transferring predictive capability without the full parameter count [11].

Lookup table (LUT) approximation represents an extreme form of model compression: rather than deploying a parameterized model, we pre-compute outputs for a grid of input values and interpolate between them. This approach has deep roots in computer graphics (texture mapping, color correction) [12], signal processing (trigonometric function evaluation) [13], and embedded systems (sensor linearization) [14].

The key insight from this literature is that LUT approximation quality depends on three factors: (1) the smoothness of the underlying function, (2) the density of the sampling grid, and (3) the interpolation scheme employed. For sufficiently smooth functions, linear interpolation over a fine grid achieves arbitrarily low approximation error. Non-linearities and discontinuities require finer sampling in affected regions or higher-order interpolation schemes.

2.3 Lookup Tables in Physics Simulation

Lookup table approaches have a long history in physics simulation, particularly for computationally expensive functions that must be evaluated repeatedly. Atmospheric models commonly employ tabulated thermodynamic properties, interpolating between pre-computed values for temperature, pressure, and density [15]. Real-time graphics engines use LUTs for physically-based rendering calculations, trading memory for computation [16].

In ballistics specifically, tabulated drag functions have been standard since the 19th century. The original Ingalls tables provided drag coefficient values at discrete Mach numbers, with interpolation for intermediate velocities [17]. Modern implementations like JBM Ballistics and Applied Ballistics continue this tradition, albeit with finer discretization and more sophisticated interpolation [18].

Our contribution extends this paradigm by tabulating not the drag function itself but the correction to a drag function: the multiplicative factor that transforms a published BC into an effective BC accounting for velocity-dependent variations captured by ML models.

3. Methodology

3.1 BC5D Table Architecture

We construct lookup tables spanning five dimensions, hence the designation "BC5D":

Bullet weight (grains): Captures mass-dependent momentum retention characteristics
Base BC (dimensionless): The manufacturer-published ballistic coefficient
Muzzle velocity (fps): Initial conditions affecting Reynolds number and flight regime
Current velocity (fps): Instantaneous velocity determining Mach-dependent drag
Drag model type (categorical): G1, G7, or custom drag functions

This 5-dimensional parameterization follows from the input signature of our continuous ML correction model, which accepts these parameters and returns a multiplicative correction factor in the range [0.5, 1.5]. A correction of 1.0 indicates no adjustment; values below 1.0 indicate reduced effective drag (higher effective BC), while values above 1.0 indicate increased drag.

3.2 Caliber-Specific Tables

Rather than constructing a single monolithic table covering all calibers, we generate separate tables for each bullet diameter: .224 (5.56mm), .243 (6mm), .264 (6.5mm), .277 (6.8mm), .284 (7mm), .308 (7.62mm), and .338 (8.6mm). This caliber-specific approach offers several advantages:

Reduced file size: Each table covers only the weight and BC ranges relevant to that caliber. A .224 table need not include entries for 300-grain bullets, nor does a .338 table require entries for 55-grain bullets. Typical table sizes range from 1.0-1.5 MB per caliber.

Targeted accuracy: Bin boundaries can be optimized for each caliber's typical parameter ranges. The .224 table uses weight bins from 50-90 grains, while the .308 table spans 125-220 grains.

Independent updates: Refinements to one caliber's model can be deployed without forcing users to re-download tables for calibers they don't use.

3.3 Sampling and Bin Definition

For each dimension, we define discrete bins that balance granularity against storage requirements:

Weight: 12 bins spanning caliber-appropriate range (e.g., 125-220 gr for .308)
Base BC: 16 bins from 0.200 to 0.800
Muzzle velocity: 10 bins from 1800 to 3500 fps
Current velocity: 20 bins from 600 to 3200 fps
Drag model: 3 values (G1, G7, G8)

The current velocity dimension receives the finest discretization because BC corrections vary most rapidly with instantaneous velocity, particularly in transonic regimes. The resulting 5D grid contains approximately 115,000 cells per drag model type, yielding total table sizes of 1.0-1.5 MB depending on caliber-specific range spans.

3.4 Table Generation Process

Table generation proceeds by exhaustively querying the continuous ML model at each grid point:

for each drag_model in [G1, G7, G8]:
    for each weight_bin in weight_bins:
        for each bc_bin in bc_bins:
            for each mv_bin in muzzle_velocity_bins:
                for each cv_bin in current_velocity_bins:
                    correction = ml_model.predict(
                        weight=weight_bin,
                        bc=bc_bin,
                        muzzle_velocity=mv_bin,
                        current_velocity=cv_bin,
                        drag_model=drag_model
                    )
                    store(correction)

The resulting values are stored in a binary format with an 80-byte header containing metadata (version, caliber, dimensions, timestamp, CRC32 checksum) followed by float32 correction values in row-major order.

3.5 Runtime Interpolation

At query time, the lookup procedure locates the surrounding grid points in each dimension and performs multi-linear interpolation. For a 5D query point, this involves identifying 32 surrounding vertices (2^5) and computing the weighted average based on the query point's position within the hypercube.

For efficiency, the implementation uses vectorized operations where possible, pre-computes dimension strides for direct array indexing, and caches recently accessed tables to avoid repeated disk I/O.

Figure 1: The continuous ML model (red) produces smooth BC corrections across the velocity range. The discretized lookup table (blue) samples at fixed intervals, creating a stair-step approximation. Note the increased correction factors in the transonic region (900-1300 fps).

4. Results and Analysis

4.1 Approximation Fidelity

We evaluated the BC5D lookup tables against the continuous ML model across a comprehensive test suite: 168-grain .308 projectiles with G1 BC of 0.475, fired at 2700 fps muzzle velocity. Table 1 presents velocity predictions at distances from 200 to 1000 yards.

Table 1: Remaining Velocity Comparison (fps)

Range	Physics Only	BC5D Lookup	Online ML	Δ (Lookup vs ML)
200 yd	2334	2330	2298	+32 fps (+1.4%)
400 yd	2002	1994	1951	+43 fps (+2.2%)
600 yd	1703	1688	1642	+46 fps (+2.8%)
800 yd	1444	1416	1364	+52 fps (+3.8%)
1000 yd	1198	1154	1008	+146 fps (+14.5%)

Several patterns emerge from this comparison. First, both BC5D lookup and online ML show substantially more velocity decay than physics-only calculations using constant BC, validating that both approaches capture drag enhancement effects invisible to traditional methods. Second, the lookup tables track the ML model within 3-4% through 800 yards, representing the supersonic and early transonic portions of the flight. Third, significant divergence appears at 1000 yards (+14.5%), where the projectile has decelerated deep into the transonic regime.

4.2 Energy Predictions

Table 2 presents the same comparison for remaining kinetic energy, which exhibits squared sensitivity to velocity errors.

Table 2: Remaining Energy Comparison (ft-lb)

Range	Physics Only	BC5D Lookup	Online ML	Δ (Lookup vs ML)
200 yd	2033	2024	1970	+54 ft-lb (+2.7%)
400 yd	1495	1483	1420	+63 ft-lb (+4.4%)
600 yd	1081	1062	1005	+57 ft-lb (+5.7%)
800 yd	778	748	694	+54 ft-lb (+7.8%)
1000 yd	535	497	379	+118 ft-lb (+31.1%)

Energy predictions show proportionally larger deviations due to the v² relationship, reaching 31% at 1000 yards. However, for practical shooting applications, the 800-yard accuracy of 7.8% remains within acceptable bounds for most use cases.

Figure 2: Deviation of BC5D lookup table predictions from the continuous ML model. Note that velocity deviations remain under 5% through 800 yards, with pronounced divergence at 1000 yards where transonic effects dominate.

4.3 Transonic Degradation Analysis

The pronounced divergence at 1000 yards reflects a fundamental characteristic of our discretization approach: piecewise-linear interpolation cannot faithfully reproduce the rapid, non-linear BC variations occurring in transonic flow. Between Mach 1.2 and Mach 0.8 (approximately 1300-900 fps at sea level), shock wave formation and detachment produce drag coefficient changes that defy smooth approximation.

The continuous ML model, trained on Doppler-derived measurements through this regime, captures these non-linearities through its learned function representation. The lookup table, sampling at fixed velocity intervals, necessarily smooths over rapid transitions between samples. This smoothing introduces systematic bias: the lookup table predicts more gradual drag increases than actually occur, resulting in optimistic velocity and energy predictions.

Three potential mitigations exist for this transonic fidelity gap:

Finer sampling: Reducing velocity bin spacing in the transonic region (e.g., 25 fps instead of 100 fps) would capture more of the non-linear structure, at the cost of increased table size.
Non-linear interpolation: Cubic or spline interpolation could better approximate curved function behavior between samples, with increased computational cost.
Hybrid approaches: Using lookup tables for supersonic flight and falling back to simplified analytical transonic models could bound worst-case errors without requiring connectivity.

4.4 Stair-Step vs. Smooth Curve Analogy

The relationship between continuous ML and discretized lookup tables parallels the distinction between analog and digital signals in audio engineering. The ML model evaluates its learned function continuously: every input maps to a precisely computed output through the model's parameter space, drawing a smooth curve through the correction landscape. The lookup table samples this smooth curve at fixed intervals, storing discrete values that are linearly interpolated at query time.

Consider a CD's 44.1 kHz sampling rate: by capturing 44,100 amplitude values per second, digital audio achieves perceptual equivalence to the analog source because the samples are dense enough that interpolation artifacts fall below human hearing thresholds. The same principle applies here; our velocity bins are fine enough (typically 100 fps spacing) that for most of the flight envelope, the stair-step approximation is imperceptible in practical shooting applications.

The transonic regime represents our "high-frequency content": rapid changes that require proportionally finer sampling to capture faithfully. Just as audio systems may exhibit aliasing when sampling signals containing frequencies above the Nyquist limit, our lookup tables exhibit approximation error when the underlying function changes faster than our sampling density can track.

5. Discussion

5.1 Practical Deployment Considerations

The BC5D tables have been deployed via a content delivery network with caliber-specific downloads. Users retrieve only the tables for calibers they actually shoot, with typical total downloads of 3-5 MB for a two-caliber configuration. Tables are cached locally with CRC32 validation ensuring data integrity after download.

The command-line interface supports three operational modes:

Online ML: Direct API queries for maximum accuracy (requires connectivity)
Offline BC5D: Lookup table interpolation (no connectivity required)
Physics only: Traditional constant-BC calculation (baseline fallback)

This tiered approach allows users to select the accuracy-connectivity trade-off appropriate to their situation: competitive shooters may prefer online ML for load development, while field use may necessitate offline tables.

5.2 Comparison to Related Approaches

Our work relates to several established techniques in the model compression literature. Unlike neural network quantization, which reduces precision of model parameters, we compute exact outputs at sample points and interpolate between them; the stored values are full-precision, only the input space is discretized. Unlike knowledge distillation, we make no attempt to train a smaller model; the "student" is simply a lookup table with no learned parameters.

The closest analogue is the function tabulation commonly employed in embedded systems and real-time simulation. Our contribution extends this paradigm to ML model outputs, demonstrating that the technique transfers effectively to learned functions trained on empirical data rather than analytical expressions.

5.3 Limitations and Future Work

Several limitations merit acknowledgment. First, the tables capture only the correction function learned by our specific ML model; improvements to the model require regenerating all tables. Second, atmospheric variations (temperature, pressure, humidity) are not currently parameterized; tables assume standard conditions, with atmospheric corrections applied as separate multiplicative factors. Third, the 14% transonic deviation may be unacceptable for applications requiring high precision at extreme range.

Future work may address these limitations through:

Finer transonic sampling with adaptive bin spacing
Additional dimensions for atmospheric parameters
Version 2 tables with drag-model-specific optimization
Exploration of non-linear interpolation schemes

6. Conclusion

This paper has presented a methodology for discretizing continuous machine learning models into offline lookup tables, specifically addressing ballistic coefficient corrections for trajectory prediction. The BC5D table architecture spans five dimensions (weight, BC, muzzle velocity, current velocity, drag model) with caliber-specific instantiation, achieving file sizes of 1.0-1.5 MB per caliber.

Empirical evaluation demonstrates that piecewise-linear interpolation over this discretized space achieves velocity predictions within 5% of the continuous ML model through supersonic and early transonic flight regimes, with predictable degradation to 14% deviation in deep transonic regions where non-linear drag variations exceed the approximation capacity of fixed-interval sampling.

We have argued that this accuracy-connectivity trade-off represents a practical compromise for field deployment, drawing analogy to digital audio sampling where sufficiently fine discretization renders quantization artifacts imperceptible for typical use cases. The transonic regime, exhibiting rapid non-linearities analogous to high-frequency audio content, requires proportionally finer sampling to capture faithfully, a trade-off that can be addressed through adaptive bin spacing in future table versions.

The broader contribution of this work lies in demonstrating that ML model outputs can be effectively tabulated for offline deployment without resorting to model compression techniques that sacrifice learned representations. For application domains where the input space is bounded and query patterns are predictable, lookup table approximation offers a deployment pathway that preserves ML-derived insights while eliminating infrastructure dependencies.

References

[1] McCoy, R. L. (1999). Modern Exterior Ballistics: The Launch and Flight Dynamics of Symmetric Projectiles. Schiffer Publishing.

[2] Carlucci, D. E., & Jacobson, S. S. (2018). Ballistics: Theory and Design of Guns and Ammunition (3rd ed.). CRC Press.

[3] Weinacht, P., Cooper, G. R., & Newill, J. F. (2005). "Analytical Prediction of Projectile Flight." Army Research Laboratory Technical Report ARL-TR-3567.

[4] Silton, S. I. (2005). "Navier-Stokes Computations for a Spinning Projectile from Subsonic to Supersonic Speeds." Journal of Spacecraft and Rockets, 42(2), 223-231.

[5] Litz, B. (2015). Applied Ballistics for Long Range Shooting (3rd ed.). Applied Ballistics LLC.

[6] SAAMI (2015). "Voluntary Industry Performance Standards for Pressure and Velocity of Centerfire Rifle Sporting Ammunition." Sporting Arms and Ammunition Manufacturers' Institute.

[7] Anderson, J. D. (2017). Fundamentals of Aerodynamics (6th ed.). McGraw-Hill Education.

[8] Courtney, M., & Courtney, A. (2012). "Experimental Tests of the Litz Model for Ballistic Coefficient Variation with Velocity." arXiv:1201.3621.

[9] Han, S., Mao, H., & Dally, W. J. (2016). "Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding." ICLR 2016.

[10] Jacob, B., et al. (2018). "Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference." CVPR 2018.

[11] Hinton, G., Vinyals, O., & Dean, J. (2015). "Distilling the Knowledge in a Neural Network." arXiv:1503.02531.

[12] Heckbert, P. S. (1986). "Survey of Texture Mapping." IEEE Computer Graphics and Applications, 6(11), 56-67.

[13] Jeong, K., & Kim, S. (2003). "Lookup Table-Based FPGA Implementation of Trigonometric Functions." Journal of the Korean Physical Society, 43, 843-847.

[14] Fraden, J. (2016). Handbook of Modern Sensors: Physics, Designs, and Applications (5th ed.). Springer.

[15] Rienecker, M. M., et al. (2011). "MERRA: NASA's Modern-Era Retrospective Analysis for Research and Applications." Journal of Climate, 24(14), 3624-3648.

[16] Karis, B. (2013). "Real Shading in Unreal Engine 4." SIGGRAPH 2013 Course Notes.

[17] Ingalls, J. M. (1893). Exterior Ballistics in the Plane of Fire. D. Van Nostrand Company.

[18] Litz, B. (2011). "Ballistic Coefficient Testing of the .308 175gr Sierra Matchking." Applied Ballistics Technical Note.

The author develops ballistics simulation software and maintains the trajectory prediction API at ballistics.7.62x51mm.sh. Source code for the BC5D table generator is available at github.com/ajokela/ballistics-engine.

Running Qwen TTS on AMD Strix Halo: A Complete Guide to Local Text-to-Speech

A.C. Jokela — Sat, 24 Jan 2026 18:00:00 GMT

The rise of high-quality text-to-speech models has opened new possibilities for content creators, accessibility advocates, and developers alike. Qwen3-TTS, developed by Alibaba's Qwen team, represents a significant leap forward in neural TTS technology, offering natural-sounding speech synthesis with multiple speaker voices. In this guide, we'll walk through setting up Qwen3-TTS on AMD's Strix Halo platform (specifically the AI Max+ 395 with its integrated Radeon 8060S graphics) and demonstrate how we use it to generate audio narrations for blog posts right here on TinyComputers.

Why Qwen3-TTS?

The text-to-speech landscape has evolved dramatically over the past few years. While cloud-based services like Amazon Polly, Google Cloud TTS, and ElevenLabs offer impressive quality, they come with ongoing costs, privacy considerations, and internet dependency. Local TTS solutions have historically lagged behind in quality, often producing robotic or unnatural speech.

Qwen3-TTS changes this equation. The model produces remarkably natural speech with proper intonation, pacing, and emphasis. It supports multiple pre-trained speaker voices (including options like Eric, Aiden, Dylan, Serena, and others) each with distinct characteristics suitable for different content types. For technical content like our blog posts, the Eric voice provides clear, professional narration that listeners find easy to follow.

The model we're using, Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice, weighs in at 1.7 billion parameters. While not small, this is manageable on modern hardware and runs efficiently on GPU. The 12Hz designation refers to the audio frame rate used during generation, balancing quality with computational requirements.

The Hardware: AMD AI Max+ 395

AMD's Strix Halo architecture represents their latest push into the high-performance APU market, combining powerful CPU cores with substantial integrated graphics. Our test system features:

CPU: AMD Ryzen AI Max+ 395 with 16 Zen 5 cores (32 threads)
GPU: Integrated Radeon 8060S (RDNA 3.5 architecture)
Memory: 128GB unified DDR5, configured with 96GB VRAM and 32GB system RAM
Compute Units: 40 CUs dedicated to graphics/compute workloads

Our test system is the Bosgame M5 AI Mini Desktop, one of the first mini PCs to ship with AMD's Strix Halo silicon. The GMKtec EVO-X2 is an extremely similar system if you're looking to replicate this setup. The unified memory architecture is particularly relevant for machine learning workloads. Unlike discrete GPUs with their own VRAM, the Radeon 8060S shares system memory with the CPU. This means no PCIe bottleneck for data transfers, and with 96GB allocated as VRAM, even large models fit comfortably.

For our TTS workload, the 8060S provides adequate performance. The 1.7B parameter model fits comfortably in memory, and inference runs entirely on GPU once loaded. We see 100% GPU utilization during speech synthesis, indicating the hardware is being fully leveraged.

Setting Up the Environment

The first challenge with AMD GPUs is getting PyTorch working correctly with ROCm, AMD's open-source GPU compute stack. The Strix Halo uses a newer GPU architecture (gfx1151) that requires ROCm 6.x and some environment variable overrides.

Step 1: Create a Python Virtual Environment

We'll use a dedicated virtual environment to isolate our TTS dependencies:

mkdir -p ~/qwen-tts
cd ~/qwen-tts
python3 -m venv venv
source venv/bin/activate

Step 2: Install PyTorch with ROCm Support

The standard PyTorch installation won't work; we need the ROCm-enabled build. As of this writing, ROCm 6.4 is the latest stable release:

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm6.4

This downloads PyTorch builds compiled specifically for AMD GPUs. The installation is larger than the standard CUDA builds due to the different compute libraries involved.

Step 3: Install Qwen-TTS

With PyTorch in place, install the Qwen TTS package:

pip install qwen-tts soundfile numpy

The soundfile library handles WAV file I/O, while numpy is needed for audio array manipulation.

Step 4: Install xformers for ROCm (Optional but Recommended)

The xformers library provides optimized attention implementations that can improve performance:

pip install xformers --index-url https://download.pytorch.org/whl/rocm6.4

While Qwen-TTS will work without xformers, having it available enables more efficient memory-attention patterns during inference.

Step 5: Configure Environment Variables

The Strix Halo's gfx1151 architecture isn't explicitly recognized by all ROCm components yet. We need to tell the system to treat it as a compatible architecture:

export HSA_OVERRIDE_GFX_VERSION=11.0.0
export GPU_MAX_ALLOC_PERCENT=100
export GPU_MAX_HEAP_SIZE=100
export TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1

Let's break down what these do:

HSA_OVERRIDE_GFX_VERSION=11.0.0: Tells the HSA runtime to report the GPU as gfx1100, which has broader library support
GPU_MAX_ALLOC_PERCENT=100: Allows the GPU to use up to 100% of available memory for allocations
GPU_MAX_HEAP_SIZE=100: Similar memory allocation setting for heap operations
TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1: Enables experimental efficient attention implementations for AMD GPUs

Add these to your .bashrc or create an activation script for convenience.

Step 6: Verify GPU Detection

Before proceeding, confirm PyTorch can see your GPU:

import torch
print(f"CUDA available: {torch.cuda.is_available()}")
print(f"Device count: {torch.cuda.device_count()}")
print(f"Device name: {torch.cuda.get_device_name(0)}")

You should see output like:

CUDA available: True
Device count: 1
Device name: AMD Radeon 8060S

Note that PyTorch uses "CUDA" terminology even for AMD GPUs when using ROCm; this is for API compatibility.

Basic TTS Usage

With the environment configured, let's test basic speech synthesis:

from qwen_tts import Qwen3TTSModel
import soundfile as sf
import torch

# Load model on GPU with bfloat16 precision
model = Qwen3TTSModel.from_pretrained(
    'Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice',
    attn_implementation='sdpa',
    device_map='cuda:0',
    dtype=torch.bfloat16
)

# Check available speakers
print(f"Available speakers: {model.get_supported_speakers()}")

# Generate speech
text = "Hello, and welcome to TinyComputers. Today we're exploring text-to-speech on AMD hardware."
audios, sample_rate = model.generate_custom_voice(
    text,
    speaker='eric',
    language='english'
)

# Save to file
sf.write('output.wav', audios[0], sample_rate)
print(f"Saved audio at {sample_rate}Hz")

A few important notes:

We use attn_implementation='sdpa' for scaled dot-product attention, which works on ROCm
The device_map='cuda:0' explicitly places the model on the GPU
Using dtype=torch.bfloat16 reduces memory usage while maintaining quality
The language parameter must be the full word 'english', not the abbreviation 'en'

Building a Blog-to-Speech Pipeline

For our use case (generating audio versions of blog posts) we need more than basic TTS. Blog posts contain markdown formatting, code blocks, images, and other elements that shouldn't be read aloud. We built a complete pipeline that handles these challenges.

The Blog Cleaner

Our cleaning process strips out non-spoken content while preserving the narrative flow:

import re

def clean_markdown(content):
    # Remove YAML frontmatter
    if content.startswith('---'):
        end = content.find('---', 3)
        if end != -1:
            content = content[end + 3:]

    # Strip HTML tags (audio, video, images)
    content = re.sub(r'<audio[^>]*>[\s\S]*?</audio>', '', content, flags=re.IGNORECASE)
    content = re.sub(r'<video[^>]*>[\s\S]*?</video>', '', content, flags=re.IGNORECASE)
    content = re.sub(r'<img[^>]*/?>', '', content, flags=re.IGNORECASE)
    content = re.sub(r'<[^>]+>', '', content)

    # Remove markdown images and convert links to just text
    content = re.sub(r'!\[[^\]]*\]\([^)]+\)', '', content)
    content = re.sub(r'\[([^\]]+)\]\([^)]+\)', r'\1', content)

    # Remove code blocks
    content = re.sub(r'```[\s\S]*?```', '', content)
    content = re.sub(r'`[^`]+`', '', content)

    # Convert headers to sentences
    content = re.sub(r'^(#{1,6})\s+(.+)$', r'\2.', content, flags=re.MULTILINE)

    # Remove emphasis markers
    content = re.sub(r'\*\*([^*]+)\*\*', r'\1', content)
    content = re.sub(r'\*([^*]+)\*', r'\1', content)

    return content

Unit Conversion for Speech

Technical content often includes abbreviations that sound awkward when read literally. We convert common units to their spoken forms:

def convert_units_for_speech(text):
    text = re.sub(r'(\d+)\s*GB\b', r'\1 gigabytes', text)
    text = re.sub(r'(\d+)\s*MB\b', r'\1 megabytes', text)
    text = re.sub(r'(\d+)\s*GHz\b', r'\1 gigahertz', text)
    text = re.sub(r'(\d+)\s*MHz\b', r'\1 megahertz', text)
    text = re.sub(r'(\d+)\s*KB\b', r'\1 kilobytes', text)
    return text

Chunking Long Content

TTS models work best with moderate-length inputs. Very long passages can cause quality degradation or memory issues. We split content into chunks at sentence boundaries:

def chunk_text(text, max_chars=500):
    chunks = []
    sentences = re.split(r'(?<=[.!?])\s+', text)
    current = ""

    for sentence in sentences:
        if len(current) + len(sentence) < max_chars:
            current += sentence + " "
        else:
            if current:
                chunks.append(current.strip())
            current = sentence + " "

    if current:
        chunks.append(current.strip())

    return chunks

The Complete Script

Putting it all together, here's our blog_to_speech.py script:

#!/usr/bin/env python3
import argparse
import re
import torch
from pathlib import Path
from qwen_tts import Qwen3TTSModel
import soundfile as sf
import numpy as np

def clean_blog_post(source):
    content = Path(source).read_text()
    # Apply cleaning functions...
    return cleaned_text

def synthesize_speech(text, output_file, speaker="eric"):
    model = Qwen3TTSModel.from_pretrained(
        'Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice',
        attn_implementation='sdpa',
        device_map='cuda:0',
        dtype=torch.bfloat16
    )

    chunks = chunk_text(text)
    all_audio = []

    for i, chunk in enumerate(chunks):
        print(f"Processing chunk {i+1}/{len(chunks)}")
        audios, sample_rate = model.generate_custom_voice(
            chunk,
            speaker=speaker,
            language='english'
        )
        all_audio.append(audios[0])

    combined = np.concatenate(all_audio)
    sf.write(output_file, combined, sample_rate)
    duration = len(combined) / sample_rate
    print(f"Saved {duration:.1f}s audio to: {output_file}")

if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument('source', help='Blog post markdown file')
    parser.add_argument('-o', '--output', default='output.wav')
    parser.add_argument('--speaker', default='eric')
    args = parser.parse_args()

    text = clean_blog_post(args.source)
    synthesize_speech(text, args.output, args.speaker)

Choosing the Right Speaker Voice

Qwen3-TTS ships with nine pre-trained speaker voices, each with distinct characteristics:

Speaker	Characteristics	Best For
Eric	Clear, professional male voice with measured pacing	Technical content, tutorials, documentation
Aiden	Younger male voice, slightly more casual	Blog posts, conversational content
Dylan	Deeper male voice with authoritative tone	Formal presentations, announcements
Ryan	Energetic male voice	Marketing content, product demos
Serena	Clear female voice, professional	Corporate content, tutorials
Vivian	Warm female voice	Storytelling, narrative content
Ono Anna	Female voice with distinct character	Creative content
Sohee	Female voice, versatile	General purpose
Uncle Fu	Character voice	Specialized applications

For our technical blog content, we primarily use Eric. His clear enunciation and measured pacing work well for complex technical explanations. The voice handles acronyms, numbers, and technical terminology naturally, making it ideal for content about hardware, programming, and system administration.

You can easily switch voices by changing the speaker parameter:

audios, sample_rate = model.generate_custom_voice(
    text,
    speaker='serena',  # Try different voices
    language='english'
)

Consider matching voice characteristics to content type. A hardware review might work better with Eric's authoritative tone, while a personal essay might benefit from Aiden's more conversational style.

Comparing TTS Options

Before settling on Qwen3-TTS, we evaluated several alternatives. Here's how they compare for our use case:

Cloud Services

Amazon Polly and Google Cloud TTS offer excellent quality with minimal setup. However, costs accumulate quickly for long-form content. At roughly \$4-16 per million characters (depending on voice quality), a 3000-word blog post costs \$0.10-0.40 per generation. For a site with dozens of posts requiring periodic regeneration, this adds up.

ElevenLabs produces arguably the most natural voices available, with impressive emotional range. But their pricing model (based on character quotas) makes it expensive for regular content generation. The quality is exceptional, but overkill for straightforward narration.

Local Alternatives

Coqui TTS (now deprecated) was a popular open-source option but development has stalled. Bark from Suno produces impressive results but runs slowly and lacks fine-grained control. XTTS offers voice cloning but requires more setup and compute resources.

Piper deserves special mention as a lightweight option. It runs quickly even on CPU and produces acceptable quality for many applications. However, the voices sound noticeably synthetic compared to Qwen3-TTS, fine for notifications or short snippets, but fatiguing for 30-minute narrations.

Qwen3-TTS hits a sweet spot: quality approaching cloud services, reasonable compute requirements, and fully local operation. The 1.7B parameter model is large enough for natural prosody but small enough to run on consumer hardware.

Batch Processing for Multiple Posts

When generating audio for multiple blog posts, efficiency matters. Loading the model takes 15-30 seconds, so we keep it loaded while processing multiple files:

#!/usr/bin/env python3
"""Batch TTS processing for multiple blog posts"""

import os
from pathlib import Path
from qwen_tts import Qwen3TTSModel
import torch
import soundfile as sf
import numpy as np

# Load model once
print("Loading model...")
model = Qwen3TTSModel.from_pretrained(
    'Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice',
    attn_implementation='sdpa',
    device_map='cuda:0',
    dtype=torch.bfloat16
)

posts = [
    'post1.md',
    'post2.md',
    'post3.md',
]

for post in posts:
    print(f"\n{'='*50}")
    print(f"Processing: {post}")
    print(f"{'='*50}")

    text = clean_blog_post(post)
    output = f"/tmp/{Path(post).stem}_tts.wav"

    # Process chunks
    chunks = chunk_text(text)
    all_audio = []

    for i, chunk in enumerate(chunks):
        print(f"  Chunk {i+1}/{len(chunks)}")
        audios, sr = model.generate_custom_voice(
            chunk, speaker='eric', language='english'
        )
        all_audio.append(audios[0])

    combined = np.concatenate(all_audio)
    sf.write(output, combined, sr)
    print(f"Saved: {output}")

This approach processes our five-post backlog overnight, with results ready for review in the morning.

Performance Characteristics

On the AI Max+ 395, speech synthesis runs at roughly real-time to 0.5x real-time speed, meaning a 30-minute audio file takes 30-60 minutes to generate. This is slower than high-end discrete GPUs but perfectly acceptable for batch processing.

For reference, here's how different content lengths performed in our testing:

Content	Characters	Chunks	Audio Duration	Generation Time
Short post	5,000	12	~5 min	~15 min
Medium post	15,000	35	~15 min	~45 min
Long post	25,000	55	~27 min	~90 min
Very long	40,000	85	~45 min	~150 min

The relationship between content length and generation time is roughly linear after the initial model warmup.

Some observations from our testing:

First chunk latency: The first chunk takes longer due to GPU kernel compilation and caching
Memory usage: Peak usage around 8-10GB during inference
GPU utilization: Consistent 100% during active synthesis
Quality: Indistinguishable from cloud TTS services for most content

The MIOpen library sometimes logs workspace warnings during execution. These don't affect output quality and can be safely ignored:

MIOpen(HIP): Warning [IsEnoughWorkspace] Solver <GemmFwdRest>, workspace required: 103133184

Integrating Audio into Blog Posts

Once we have the WAV file, we convert to MP3 for web delivery and embed an HTML5 audio player:

ffmpeg -i blog_post.wav -codec:a libmp3lame -qscale:a 2 blog_post.mp3

For reviewing TTS output quality, we recommend using studio monitor headphones that reveal any artifacts or unnatural tones in the generated speech.

The player HTML is straightforward:

<div style="background: #f8f9fa; border: 1px solid #e9ecef;
            border-radius: 8px; padding: 16px 20px; margin: 20px 0;">
  <div class="audio-widget-header">
    <span class="audio-widget-icon">🎧</span>
    <span style="color: #495057; font-weight: 600;">Listen to this article</span>
  </div>
  <audio controls preload="metadata" style="width: 100%;">
    <source src="/audio/blog_post.mp3" type="audio/mpeg">
  </audio>
  <div class="audio-widget-footer">
    27 min · AI-generated narration
  </div>
</div>

Why We're Doing This

Adding audio narration to blog posts serves multiple purposes:

Accessibility: Readers with visual impairments or reading difficulties can consume content aurally
Convenience: Listeners can enjoy posts during commutes, workouts, or other activities
Engagement: Audio content creates a more personal connection with the audience
Reach: Some audiences prefer audio format, expanding our potential readership

Running TTS locally rather than using cloud services gives us:

Cost control: No per-character or per-minute fees
Privacy: Content never leaves our infrastructure
Consistency: Same voice and quality across all posts
Flexibility: Full control over processing pipeline

Troubleshooting Common Issues

"CUDA not available" despite GPU present

Ensure you've installed the ROCm version of PyTorch, not the standard build:

pip uninstall torch torchvision torchaudio
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm6.4

Model runs on CPU instead of GPU

Check that device_map='cuda:0' is specified when loading the model. Also verify the environment variables are set before starting Python.

"Unsupported language 'en'"

Use the full language name: language='english' not language='en'.

Out of memory errors

Try reducing chunk size or using a smaller batch. The model should fit in 16GB, but very long chunks can spike memory usage.

Slow first chunk

This is normal; ROCm compiles GPU kernels on first use. Subsequent chunks process faster.

Future Improvements

Our current pipeline works well but has room for enhancement. Some improvements we're considering:

Voice cloning: Qwen3-TTS supports custom voice training. With sufficient audio samples, we could create a unique voice for TinyComputers rather than using the stock speakers. This would provide brand consistency and differentiation.

Automatic post detection: Currently we manually select posts for TTS generation. A CI/CD integration could automatically generate audio for new posts when they're published, keeping the audio library current without manual intervention.

Chapter markers: For longer posts, embedding chapter markers in the audio file would allow listeners to skip to specific sections. This requires parsing the markdown headers and mapping them to audio timestamps.

Multiple format export: Beyond MP3, offering Opus or AAC formats could reduce file sizes while maintaining quality, benefiting listeners on metered connections.

Speed adjustment: Some listeners prefer 1.25x or 1.5x playback speed. Pre-generating speed-adjusted versions could provide better quality than real-time speed adjustment in the browser.

Conclusion

Running Qwen3-TTS on AMD's Strix Halo platform demonstrates that high-quality local TTS is now accessible beyond NVIDIA hardware. While setup requires some ROCm-specific configuration, the results are impressive: natural-sounding narration suitable for professional content.

The democratization of AI capabilities continues apace. What once required expensive cloud subscriptions or high-end NVIDIA GPUs now runs on integrated graphics. The AI Max+ 395's Radeon 8060S, primarily designed for gaming and general compute tasks, handles a 1.7-billion parameter language model without breaking a sweat.

We're actively using this pipeline to generate audio versions of posts across TinyComputers, making our technical content more accessible and convenient for our readers. As of this writing, we've processed our retrocomputing series, hardware reviews, and technical tutorials, dozens of hours of content generated entirely on local hardware.

The combination of AMD's capable integrated graphics and Qwen's excellent TTS model proves that you don't need expensive discrete GPUs or cloud subscriptions to achieve broadcast-quality speech synthesis. For content creators, educators, and accessibility advocates, this opens new possibilities for enriching written content with audio without ongoing service costs.

If you're running AMD hardware and want to add audio narration to your own content, this guide should get you started. The initial setup investment pays dividends in ongoing cost savings and the satisfaction of running capable AI models entirely on your own infrastructure. And if you encounter issues along the way, the troubleshooting section above addresses the most common pitfalls we discovered during our own setup process.

The audio player at the top of many TinyComputers posts now represents a small but meaningful step toward making technical content more accessible. Every post you can listen to while commuting, exercising, or doing dishes is content that might otherwise go unread. That's the real value of local TTS: not just cost savings, but expanded reach for the ideas we share.

Getting YOLOv8 Training Working on AMD Ryzen™ AI Max+ 395

A.C. Jokela — Wed, 22 Oct 2025 14:54:43 GMT

Introduction

Machine learning on AMD GPUs has always been... interesting. With NVIDIA's CUDA dominating the landscape, AMD's ROCm platform remains the underdog: powerful, but often requiring patience and persistence to get working properly. This is the story of how I got YOLOv8 object detection training working on an AMD Radeon 8060S integrated GPU (gfx1151) in the AMD RYZEN AI MAX+ 395 after encountering batch normalization failures, version mismatches, and a critical bug in MIOpen.

The goal was simple: train a bullet hole detection model for a ballistics application using YOLOv8. The journey? Anything but simple.

The Hardware

System Specifications:

CPU: AMD RYZEN AI MAX+ 395
GPU: AMD Radeon 8060S (integrated, RDNA 3.5 architecture, gfx1151)
VRAM: 96GB shared system memory
ROCm Version: 7.0.2
ROCk module: 6.14.14
PyTorch: 2.8.0+rocm7.0.0.git64359f59
MIOpen: Initially 3.0.5.1 (version code 3005001), later custom build
OS: Linux (conda environment: pt2.8-rocm7)

The AMD Radeon 8060S is an integrated GPU in the AMD RYZEN AI MAX+ 395 based on AMD's RDNA 3.5 architecture (gfx1151). What makes this system particularly interesting for machine learning is the massive 96GB of shared system memory available to the GPU, far more VRAM than typical consumer discrete GPUs. While machine learning support on RDNA 3.5 is still maturing compared to older RDNA 2 architectures, the memory capacity makes it compelling for AI workloads.

But, for about $1,699, you can get up to 96GB of VRAM in a whisper-quiet form factor. This setup beats the pants off of my old GPU rig.

Why YOLOv8 and Ultralytics?

Before diving into the technical challenges, it's worth explaining why we chose YOLOv8 from Ultralytics for this project.

YOLOv8 (You Only Look Once, version 8) is the latest iteration of one of the most popular object detection architectures. Developed and maintained by Ultralytics, it offers several advantages:

Why Ultralytics YOLOv8?

State-of-the-art Accuracy: YOLOv8 achieves excellent detection accuracy while maintaining real-time inference speeds, critical for practical applications.
Ease of Use: Ultralytics provides a clean, well-documented Python API that makes training custom models remarkably straightforward:

from ultralytics import YOLO
model = YOLO("yolov8n.pt")
results = model.train(data="dataset.yaml", epochs=100)

Active Development: Ultralytics is actively maintained with frequent updates, bug fixes, and community support. This proved invaluable during debugging.
Model Variants: YOLOv8 comes in multiple sizes (nano, small, medium, large, extra-large), allowing us to balance accuracy vs. speed for our specific use case.
Built-in Data Augmentation: The framework includes extensive data augmentation capabilities out of the box, essential for training robust detection models with limited training data.
PyTorch Native: Being built on PyTorch meant it should theoretically work with ROCm (AMD's CUDA equivalent)... in theory.

For our bullet hole detection application, YOLOv8's ability to accurately detect small objects (bullet holes in paper targets) while training efficiently made it the obvious choice. Little did I know that "training efficiently" would require a week-long debugging odyssey.

The Initial Setup (ROCm 7.0.0)

I started with ROCm 7.0.0, following AMD's official installation guide. Everything installed cleanly:

$ python -c "import torch; print(torch.cuda.is_available())"
True

$ python -c "import torch; print(torch.cuda.get_device_name(0))"
AMD Radeon Graphics

Perfect! PyTorch recognized the GPU. Time to train some models, right?

The First Failure: Batch Normalization

I loaded a simple YOLOv8 nano model and kicked off training:

from ultralytics import YOLO

model = YOLO("yolov8n.pt")
results = model.train(
    data="data/bullet_hole_dataset_combined/data.yaml",
    epochs=100,
    imgsz=416,
    batch=16,
    device="cuda:0"
)

Within seconds, the training crashed:

RuntimeError: miopenStatusUnknownError

The error was cryptic, but digging deeper revealed the real issue: MIOpen was failing to compile batch normalization kernels with inline assembly errors:

<inline asm>:14:20: error: not a valid operand.
v_add_f32 v4 v4 v4 row_bcast:15 row_mask:0xa
                   ^

Batch normalization. The most common operation in modern deep learning, and it was failing spectacularly on gfx1151. The inline assembly instructions (row_bcast and row_mask) appeared incompatible with the RDNA 3.5 architecture.

What is Batch Normalization?

Batch normalization (BatchNorm) is a technique that normalizes layer inputs across a mini-batch, helping neural networks train faster and more stably. It's used in virtually every modern CNN architecture, including YOLO.

The error message pointed to MIOpen, AMD's equivalent of NVIDIA's cuDNN, a library of optimized deep learning primitives.

Attempt 1: Upgrade to ROCm 7.0.2

My first instinct was to upgrade ROCm. Version 7.0.0 was relatively new, and perhaps 7.0.2 had fixed the batch normalization issues.

# Upgraded PyTorch to ROCm 7.0.2
pip install --upgrade torch --index-url https://download.pytorch.org/whl/rocm7.0

Result? Same error. Batch normalization still failed.

RuntimeError: miopenStatusUnknownError

With the same inline assembly compilation errors about invalid row_bcast and row_mask operands. At this point, I realized this wasn't a simple version mismatch; there was something fundamentally broken with MIOpen's batch normalization implementation for the gfx1151 architecture.

The Revelation: It's MIOpen, Not ROCm

After hours of testing different PyTorch versions, driver configurations, and kernel parameters, I turned to the ROCm community for help.

I posted my issue on Reddit's r/ROCm subreddit, describing the inline assembly compilation failures and miopenStatusUnknownError on gfx1151. Within a few hours, a knowledgeable Redditor responded with a crucial piece of information:

"There's a known issue with MIOpen 3.0.x and gfx1151 batch normalization. The inline assembly instructions use operands that aren't compatible with RDNA 3. A fix was recently merged into the develop branch. Try using a nightly build of MIOpen or build from source."

This was the breakthrough I needed. The issue wasn't with ROCm itself or PyTorch; it was specifically MIOpen version 3.0.5.1 that shipped with ROCm 7.0.x. The maintainers had already fixed the gfx1151 batch normalization bug in a recent pull request, but it hadn't made it into a stable release yet.

The Reddit user suggested two options:

Use a nightly Docker container with the latest MIOpen build
Build MIOpen 3.5.1 from source using the develop branch

Testing the Theory: Docker Nightly Builds

Before committing to building from source, I wanted to verify that a newer MIOpen would actually fix the problem. AMD provides nightly Docker images with bleeding-edge ROCm builds:

docker pull rocm/pytorch-nightly:latest

docker run --rm \
    --device=/dev/kfd \
    --device=/dev/dri \
    --group-add video \
    -v ~/ballistics_training:/workspace \
    -w /workspace \
    rocm/pytorch-nightly:latest \
    bash -c 'pip install ultralytics && python3 test_yolo.py'

The nightly container included MIOpen 3.5.1 from the develop branch.

# test_yolo.py
from ultralytics import YOLO
import torch

print(f"PyTorch: {torch.__version__}")
print(f"CUDA available: {torch.cuda.is_available()}")
print(f"Device: {torch.cuda.get_device_name(0)}")

model = YOLO("yolov8n.pt")
results = model.train(
    data="data_docker.yaml",
    epochs=1,
    imgsz=416,
    batch=2,
    device="cuda:0"
)

Result:

✅ SUCCESS! Nightly build FIXES gfx1151 batch normalization!

It worked! The miopenStatusUnknownError was gone, no more inline assembly compilation failures. Training completed successfully with MIOpen 3.5.1 from the develop branch. The newer version had updated the batch normalization kernels to use instructions compatible with RDNA 3.5's gfx1151 architecture.

This confirmed the Reddit user's tip: the fix was indeed in the newer MIOpen code that hadn't been released in a stable version yet.

The Solution: Building MIOpen from Source

Docker was great for testing, but I needed a permanent solution for my native conda environment. That meant building MIOpen 3.5.1 from source.

Step 1: Clone the Repository

cd ~/ballistics_training
git clone https://github.com/ROCm/MIOpen.git rocm-libraries/projects/miopen
cd rocm-libraries/projects/miopen
git checkout develop  # Latest development branch with gfx1151 fixes

Step 2: Build MIOpen

mkdir build && cd build

cmake \
    -DCMAKE_PREFIX_PATH="/opt/rocm" \
    -DCMAKE_INSTALL_PREFIX="$HOME/ballistics_training/rocm-libraries/projects/miopen/build" \
    -DMIOPEN_BACKEND=HIP \
    -DCMAKE_BUILD_TYPE=Release \
    ..

make -j$(nproc)

[ 98%] Building CXX object src/CMakeFiles/MIOpen.dir/softmax_api.cpp.o
[ 99%] Linking CXX shared library libMIOpen.so
[100%] Built target MIOpen

Success! MIOpen 3.5.1 was built from source.

Step 3: Install Custom MIOpen to Conda Environment

Now came the tricky part: replacing the system MIOpen (version 3.0.5.1) with my custom-built version 3.5.1.

CONDA_LIB=~/anaconda3/envs/pt2.8-rocm7/lib

# Backup the original MIOpen
cp $CONDA_LIB/libMIOpen.so.1.0 $CONDA_LIB/libMIOpen.so.1.0.backup_system

# Install custom MIOpen
cp ~/ballistics_training/rocm-libraries/projects/miopen/build/lib/libMIOpen.so.1.0 $CONDA_LIB/

# Update symlinks
cd $CONDA_LIB
ln -sf libMIOpen.so.1.0 libMIOpen.so.1
ln -sf libMIOpen.so.1 libMIOpen.so

Step 4: Verify the Installation

conda activate pt2.8-rocm7
python -c "import torch; print(f'MIOpen version: {torch.backends.cudnn.version()}')"

Output:

MIOpen version: 3005001

Wait, 3005001? That's version 3.5.1! (MIOpen uses an integer versioning scheme: major1000000 + minor1000 + patch)

The custom MIOpen was successfully loaded.

The Final Test: YOLOv8 Training

Time for the moment of truth. Could I finally train YOLOv8 on my AMD GPU?

from ultralytics import YOLO
import torch

print("=" * 60)
print("Testing YOLOv8 Training with Custom MIOpen 3.5.1")
print("=" * 60)
print(f"PyTorch: {torch.__version__}")
print(f"CUDA available: {torch.cuda.is_available()}")
print(f"MIOpen version: {torch.backends.cudnn.version()}")
print()

model = YOLO("yolov8n.pt")
print("Starting training...")

results = model.train(
    data="data/bullet_hole_dataset_combined/data.yaml",
    epochs=100,
    imgsz=416,
    batch=16,
    device="cuda:0",
    name="bullet_hole_detector"
)

Output:

============================================================
Testing YOLOv8 Training with Custom MIOpen 3.5.1
============================================================
PyTorch: 2.8.0+rocm7.0.0.git64359f59
CUDA available: True
MIOpen version: 3005001

Starting training...

Ultralytics 8.3.217 🚀 Python-3.12.11 torch-2.8.0+rocm7.0.0 CUDA:0 (AMD Radeon Graphics, 98304MiB)

Model summary: 129 layers, 3,011,043 parameters, 3,011,027 gradients, 8.2 GFLOPs

Transferred 319/355 items from pretrained weights
AMP: running Automatic Mixed Precision (AMP) checks...
AMP: checks passed ✅

Starting training for 1 epochs...

      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
        1/1     0.172G      3.022      3.775      1.215         29        416
        1/1     0.174G      2.961      4.034      1.147         46        416
        1/1     0.203G      3.133       4.08      1.251         36        416
        1/1     0.205G       3.14      4.266       1.25         60        416
        1/1     0.205G      3.028      4.194      1.237         18        416
        1/1     0.205G      2.995      4.114      1.235         28        416
        1/1     0.205G      3.029      4.118      1.226         41        416
        1/1     0.205G      2.961      4.031      1.209         26        416
        1/1     0.205G      2.888      3.998      1.193         22        416
        1/1     0.205G      2.861      3.823      1.185         49        416
        1/1     0.205G      2.812      3.657      1.169         46        416
        1/1     0.205G      2.821      3.459      1.149         78        416
        1/1     0.205G      2.776      3.253      1.134         26        416
        1/1     0.217G      2.784      3.207      1.131        122        416
        1/1     0.217G      2.772      3.074      1.121         40        416
        1/1     0.217G      2.774       2.98      1.114         13        416
        1/1     0.217G      2.763      2.914      1.118         37        416
        1/1     0.217G       2.75      2.876      1.113         81        416
        1/1     0.217G      2.731      2.799      1.104         31        416
        1/1     0.217G      2.736      2.732      1.101         30        416: 100% 14.8it/s

                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95)
                   all         60        733      0.653      0.473       0.53      0.191

1 epochs completed in 0.002 hours.

==============================================================
✅ SUCCESS! Training completed without errors!
==============================================================

Speed: 0.0ms preprocess, 1.9ms inference, 0.0ms loss, 0.5ms postprocess per image
Results saved to runs/detect/bullet_hole_detector/

It worked! Batch normalization executed flawlessly. The training progressed smoothly from epoch to epoch, with GPU utilization staying high, memory management remaining stable, and losses converging as expected. The model achieved 53.0% mAP50 and trained without a single error.

After a week of debugging, version wrangling, and source code compilation, I finally had GPU-accelerated YOLOv8 training working on my AMD RDNA 3.5 GPU. The custom MIOpen 3.5.1 build resolved the inline assembly compatibility issues, and training now runs as smoothly on gfx1151 as it would on any other supported GPU.

Performance Notes

With the custom MIOpen build, training performance was excellent:

Training Speed: 70.5 images/second (batch size 16, 416×416 images)
Training Time: 32.6 seconds for 10 epochs (2,300 total images)
Throughput: 9.7-9.9 iterations/second
GPU Utilization: ~95% during training with no throttling
Memory Usage: ~1.2 GB VRAM for YOLOv8n with batch size 16

The GPU utilization stayed consistently high with no performance degradation across epochs. Each epoch averaged approximately 3.3 seconds with solid consistency. For comparison, CPU-only training on the same dataset would be roughly 15-20x slower. The GPU acceleration was well worth the effort.

Lessons Learned

This debugging journey taught me several valuable lessons:

1. The ROCm Community is Invaluable

The Reddit r/ROCm community proved to be the key to solving this issue. When official documentation fails, community knowledge fills the gap. Don't hesitate to ask for help; chances are someone has encountered your exact issue before.

2. MIOpen ≠ ROCm

I initially assumed upgrading ROCm would fix the problem. In reality, MIOpen (the deep learning library) had a separate bug that was independent of the ROCm platform version. Understanding the component architecture of ROCm saved hours of debugging time.

3. RDNA 3.5 (gfx1151) Support is Still Maturing

AMD's latest integrated GPU architecture is powerful, but ML support lags behind older architectures like RDNA 2 (gfx1030) and Vega. If you're doing serious ML work on AMD, consider that newer hardware may require more troubleshooting.

4. Nightly Builds Can Be Production-Ready

There's often hesitation to use nightly/development builds in production. However, in this case, the develop branch of MIOpen was actually more stable than the official release for my specific GPU. Sometimes bleeding-edge code is exactly what you need.

5. Docker is Great for Testing

The ROCm nightly Docker containers were instrumental in proving my hypothesis. Being able to test a newer MIOpen version without committing to a full rebuild saved significant time.

6. Source Builds Give You Control

Building from source is time-consuming and requires understanding the build system, but it gives you complete control over your environment. When binary distributions fail, source builds are your safety net.

Tips for AMD GPU Machine Learning

If you're attempting to do machine learning on AMD GPUs, here are some recommendations:

Environment Setup

Use conda/virtualenv: Isolate your Python environment to avoid system package conflicts
Pin your versions: Lock PyTorch, ROCm, and MIOpen versions once you have a working setup
Keep backups: Always backup working library files before swapping them out

Debugging Strategy

Verify GPU detection first: Ensure torch.cuda.is_available() returns True
Test simple operations: Try basic tensor operations before complex models
Check MIOpen version: torch.backends.cudnn.version() can reveal version mismatches
Monitor logs: ROCm logs (MIOPEN_ENABLE_LOGGING=1) provide valuable debugging info
Try Docker first: Test potential fixes in Docker before modifying your system

Hardware Considerations

RDNA 2 (gfx1030) is more mature than RDNA 3.5 (gfx1151) for ML workloads
Server GPUs (MI series) have better ROCm support than consumer cards
Integrated GPUs with large shared memory (like the Radeon 8060S with 96GB) offer unique advantages for ML
Check compatibility: Always verify your specific GPU (gfx code) is supported before purchasing

Conclusion

Getting YOLOv8 training working on an AMD RDNA 3.5 GPU wasn't easy, but it was achievable. The combination of:

Community support from r/ROCm pointing me to the right solution
Docker testing to verify the fix
Building MIOpen 3.5.1 from source
Carefully replacing system libraries

...resulted in a fully functional GPU-accelerated machine learning training environment.

AMD's ROCm platform still has rough edges compared to NVIDIA's CUDA ecosystem, but it's improving rapidly. With some patience, persistence, and willingness to dig into source code, AMD GPUs can absolutely be viable for machine learning workloads.

The bullet hole detection model trained successfully, achieved excellent accuracy, and now runs in production. Sometimes the journey is as valuable as the destination; I learned more about ROCm internals, library dependencies, and GPU computing in this week than I would have in months of smooth sailing.

If you're facing similar issues with AMD GPUs and ROCm, I hope this guide helps. And remember: when in doubt, check r/ROCm. The community might just have the answer you're looking for.

System Details (for reference):

CPU: AMD RYZEN AI MAX+ 395
GPU: AMD Radeon 8060S (integrated, gfx1151)
VRAM: 96GB shared system memory
ROCm: 7.0.2
ROCk module: 6.14.14
PyTorch: 2.8.0+rocm7.0.0.git64359f59
MIOpen: 3.5.1 (custom build from develop branch)
Conda Environment: pt2.8-rocm7
YOLOv8: Ultralytics 8.3.217

Key Files:

MIOpen source: https://github.com/ROCm/MIOpen
Ultralytics YOLOv8: https://github.com/ultralytics/ultralytics
ROCm installation: https://rocm.docs.amd.com/

Special thanks to the r/ROCm community for pointing me toward the MIOpen develop branch fix!

Transfer Learning for Predictive Custom Drag Modeling: Automated Generation of Drag Coefficient Curves Using Multi-Modal AI

A.C. Jokela — Fri, 10 Oct 2025 18:10:00 GMT

TL;DR

We built a neural network that predicts full drag coefficient curves (41 Mach points from 0.5 to 4.5) for rifle bullets using only basic specifications like weight, caliber, and ballistic coefficient. The system achieves 3.15% mean absolute error and has been serving predictions in production since September 2025. This post walks through the technical implementation details, architecture decisions, and lessons learned building a real-world ML system for ballistic physics.

Read the full whitepaper: Transfer Learning for Predictive Custom Drag Modeling (17 pages)

The Problem: Drag Curves Are Scarce, But Critical

If you've ever built a ballistic calculator, you know the challenge: accurate drag modeling is everything. Standard drag models (G1, G7, G8) work okay for "average" bullets, but modern precision shooting demands better. Custom Drag Models (CDMs), full drag coefficient curves measured with doppler radar, are the gold standard. They capture the unique aerodynamic signature of each bullet design.

The catch? Getting a CDM requires: - Access to a doppler radar range (≈$500K+ equipment) - Firing 50-100 rounds at various velocities - Expert analysis to process the raw data - Cost: $5,000-$15,000 per bullet

For manufacturers like Hornady and Lapua, this is routine. For smaller manufacturers or custom bullet makers? Not happening. We had 641 bullets with real radar-measured CDMs and thousands of bullets with only basic specs. Could we use machine learning to bridge the gap?

The Vision: Transfer Learning from Radar Data

The core insight: bullets with similar physical characteristics have similar drag curves. A 168gr .308 boattail match bullet from Manufacturer A will drag similarly to one from Manufacturer B. We could train a neural network on our 641 radar-measured bullets and use transfer learning to predict CDMs for bullets we've never measured.

But we faced an immediate data problem: 641 samples isn't much for deep learning. Enter synthetic data augmentation.

Part 1: Automating Data Extraction with Claude Vision

Applied Ballistics publishes ballistic data for 704+ bullets as JPEG images. Manual data entry would take 1,408 hours (704 bullets × 2 hours each). We needed automation.

The Vision Processing Pipeline

We built an extraction pipeline using Claude 3.5 Sonnet's vision capabilities:

import anthropic
import base64
from pathlib import Path

def extract_bullet_data(image_path: str) -> dict:
    """Extract bullet specifications from AB datasheet JPEG."""
    client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])

    # Load and encode image
    with open(image_path, "rb") as f:
        image_data = base64.standard_b64encode(f.read()).decode("utf-8")

    # Vision extraction prompt
    message = client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=1024,
        messages=[{
            "role": "user",
            "content": [
                {
                    "type": "image",
                    "source": {
                        "type": "base64",
                        "media_type": "image/jpeg",
                        "data": image_data,
                    },
                },
                {
                    "type": "text",
                    "text": """Extract the following from this Applied Ballistics bullet datasheet:
                    - Caliber (inches, decimal format)
                    - Bullet weight (grains)
                    - G1 Ballistic Coefficient
                    - G7 Ballistic Coefficient
                    - Bullet length (inches, if visible)
                    - Ogive radius (calibers, if visible)

                    Return as JSON with keys: caliber, weight_gr, bc_g1, bc_g7, length_in, ogive_radius_cal"""
                }
            ],
        }]
    )

    # Parse response
    data = json.loads(message.content[0].text)

    # Physics validation
    validate_bullet_physics(data)

    return data

def validate_bullet_physics(data: dict):
    """Sanity checks for extracted data."""
    caliber = data['caliber']
    weight = data['weight_gr']

    # Caliber bounds
    assert 0.172 <= caliber <= 0.50, f"Invalid caliber: {caliber}"

    # Weight-to-caliber ratio (sectional density proxy)
    ratio = weight / (caliber  3)
    assert 0.5 <= ratio <= 2.0, f"Implausible weight for caliber: {weight}gr @ {caliber}in"

    # BC sanity
    assert 0.1 <= data['bc_g1'] <= 1.2, f"Invalid G1 BC: {data['bc_g1']}"
    assert 0.1 <= data['bc_g7'] <= 0.9, f"Invalid G7 BC: {data['bc_g7']}"

Figure 2: Claude Vision extraction pipeline - from JPEG datasheets to structured bullet specifications

Results: - 704/704 successful extractions (100% success rate) - 2.3 seconds per bullet (average) - 27 minutes total vs. 1,408 hours manual - 99.97% time savings

We validated against a manually-verified subset of 50 bullets: - 100% match on caliber - 98% match on weight (±0.5 grain tolerance) - 96% match on BC values (±0.002 tolerance)

The vision model occasionally struggled with hand-drawn or low-quality scans, but the physics validation caught these errors before they corrupted our dataset.

Part 2: Generating Synthetic CDM Curves

Now we had 704 bullets with BC values but no full CDM curves. We needed to synthesize them.

The BC-to-CDM Transformation Algorithm

The relationship between ballistic coefficient and drag coefficient is straightforward:

BC = m / (C_d × d²)

Rearranging:
C_d(M) = m / (BC(M) × d²)

But BC values are typically single scalars, not curves. We developed a 5-step hybrid algorithm combining standard drag model references with BC-derived corrections:

Step 1: Base Reference Curve

Start with the G7 standard drag curve as a baseline (better for modern boattail bullets than G1):

def get_g7_reference_curve(mach_points: np.ndarray) -> np.ndarray:
    """G7 standard drag curve from McCoy (1999)."""
    # Precomputed G7 curve at 41 Mach points
    return interpolate_standard_curve("G7", mach_points)

Step 2: BC-Based Scaling

Scale the reference curve using extracted BC values:

def scale_by_bc(cd_base: np.ndarray, bc_actual: float, bc_reference: float = 0.221) -> np.ndarray:
    """Scale drag curve to match actual BC.

    BC_G7_ref = 0.221 (G7 standard projectile)
    """
    scaling_factor = bc_reference / bc_actual
    return cd_base * scaling_factor

Step 3: Multi-Regime Interpolation

When both G1 and G7 BCs are available, blend them based on Mach regime:

def blend_drag_models(mach: np.ndarray, cd_g1: np.ndarray, cd_g7: np.ndarray) -> np.ndarray:
    """Blend G1 and G7 curves based on flight regime.

    - Supersonic (M > 1.2): Use G1 (better for shock wave region)
    - Transonic (0.8 < M < 1.2): Cubic spline interpolation
    - Subsonic (M < 0.8): Use G7 (better for low-speed)
    """
    cd_blended = np.zeros_like(mach)

    for i, M in enumerate(mach):
        if M > 1.2:
            # Supersonic: G1 better captures shock effects
            cd_blended[i] = cd_g1[i]
        elif M < 0.8:
            # Subsonic: G7 better for boattail bullets
            cd_blended[i] = cd_g7[i]
        else:
            # Transonic: smooth interpolation
            t = (M - 0.8) / 0.4  # Normalize to [0, 1]
            cd_blended[i] = cubic_interpolate(cd_g7[i], cd_g1[i], t)

    return cd_blended

Step 4: Transonic Peak Generation

Model the transonic drag spike using a Gaussian kernel:

def add_transonic_peak(cd_base: np.ndarray, mach: np.ndarray,
                       bc_g1: float, bc_g7: float) -> np.ndarray:
    """Add realistic transonic drag spike.

    Peak amplitude calibrated from BC ratio (G1 worse than G7 in transonic).
    """
    # Estimate peak amplitude from BC discrepancy
    bc_ratio = bc_g1 / bc_g7
    peak_amplitude = 0.15 * (bc_ratio - 1.0)  # Empirically tuned

    # Gaussian centered at critical Mach
    M_crit = 1.0
    sigma = 0.15

    transonic_spike = peak_amplitude * np.exp(-((mach - M_crit)  2) / (2 * sigma  2))

    return cd_base + transonic_spike

Step 5: Monotonicity Enforcement

Apply Savitzky-Golay smoothing to prevent unphysical oscillations:

from scipy.signal import savgol_filter

def enforce_smoothness(cd_curve: np.ndarray, window_length: int = 7, polyorder: int = 3) -> np.ndarray:
    """Smooth drag curve while preserving transonic peak.

    Savitzky-Golay filter preserves peak shape better than moving average.
    """
    # Must have odd window length
    if window_length % 2 == 0:
        window_length += 1

    return savgol_filter(cd_curve, window_length, polyorder, mode='nearest')

Validation Against Ground Truth

We validated synthetic curves against 127 bullets where both BC values and full CDM curves were available:

Metric	Value	Notes
Mean Absolute Error	3.2%	Across all Mach points
Transonic Error	4.8%	Mach 0.8-1.2 (most challenging)
Supersonic Error	2.1%	Mach 1.5-3.0 (best performance)
Shape Correlation	r = 0.984	Pearson correlation

The synthetic curves satisfied all physics constraints: - Monotonic decrease in supersonic regime - Realistic transonic peaks (1.3-2.0× baseline) - Smooth transitions between regimes

Figure 3: Validation of synthetic CDM curves against ground truth radar measurements

Total training data: 1,345 bullets (704 synthetic + 641 real), 2.1x data augmentation.

Part 3: Architecture Exploration

With data ready, we explored four neural architectures:

1. Multi-Layer Perceptron (Baseline)

Simple feedforward network:

import torch
import torch.nn as nn

class CDMPredictor(nn.Module):
    """MLP for CDM prediction: 13 features → 41 Cd values."""

    def __init__(self, dropout: float = 0.2):
        super().__init__()
        self.network = nn.Sequential(
            nn.Linear(13, 256),
            nn.ReLU(),
            nn.Dropout(dropout),

            nn.Linear(256, 512),
            nn.ReLU(),
            nn.Dropout(dropout),

            nn.Linear(512, 512),
            nn.ReLU(),
            nn.Dropout(dropout),

            nn.Linear(512, 256),
            nn.ReLU(),
            nn.Dropout(dropout),

            nn.Linear(256, 41)  # Output: 41 Mach points
        )

    def forward(self, x):
        return self.network(x)

Input Features (13 total):

features = [
    'caliber',           # inches
    'weight_gr',         # grains
    'bc_g1',            # G1 ballistic coefficient
    'bc_g7',            # G7 ballistic coefficient
    'length_in',        # bullet length (imputed if missing)
    'ogive_radius_cal', # ogive radius in calibers
    'meplat_diam_in',   # meplat diameter
    'boat_tail_angle',  # boattail angle (degrees)
    'bearing_length',   # bearing surface length
    'sectional_density', # weight / caliber²
    'form_factor_g1',   # i / BC_G1
    'form_factor_g7',   # i / BC_G7
    'length_to_diameter' # L/D ratio
]

Figure 4: MLP architecture - 13 input features through 4 hidden layers to 41 output Mach points

2. Physics-Informed Neural Network (PINN)

Added physics loss term enforcing drag model constraints:

class PINN_CDMPredictor(nn.Module):
    """Physics-Informed NN with drag equation constraints."""

    def __init__(self):
        super().__init__()
        # Same architecture as MLP
        self.network = build_mlp_network()

    def physics_loss(self, cd_pred: torch.Tensor, features: torch.Tensor, mach: torch.Tensor) -> torch.Tensor:
        """Enforce physics constraints on predictions.

        Constraints:
        1. Drag increases with Mach in subsonic
        2. Transonic peak exists near M=1
        3. Monotonic decrease in supersonic
        """
        # Constraint 1: Subsonic gradient
        subsonic_mask = mach < 0.8
        subsonic_cd = cd_pred[subsonic_mask]
        subsonic_grad = torch.diff(subsonic_cd)
        subsonic_violation = torch.relu(-subsonic_grad).sum()  # Penalize decreases

        # Constraint 2: Transonic peak
        transonic_mask = (mach >= 0.8) & (mach <= 1.2)
        transonic_cd = cd_pred[transonic_mask]
        peak_violation = torch.relu(1.1 - transonic_cd.max()).sum()  # Must exceed 1.1

        # Constraint 3: Supersonic monotonicity
        supersonic_mask = mach > 1.5
        supersonic_cd = cd_pred[supersonic_mask]
        supersonic_grad = torch.diff(supersonic_cd)
        supersonic_violation = torch.relu(supersonic_grad).sum()  # Penalize increases

        return subsonic_violation + peak_violation + supersonic_violation

def total_loss(cd_pred, cd_true, features, mach, lambda_physics=0.1):
    """Combined data + physics loss."""
    data_loss = nn.MSELoss()(cd_pred, cd_true)
    physics_loss = model.physics_loss(cd_pred, features, mach)

    return data_loss + lambda_physics * physics_loss

Result: Over-regularization. Physics loss was too strict, preventing the model from learning subtle variations. Performance degraded to 4.86% MAE.

3. Transformer Architecture

Treated the 41 Mach points as a sequence:

class TransformerCDM(nn.Module):
    """Transformer encoder for sequence-to-sequence CDM prediction."""

    def __init__(self, d_model=128, nhead=8, num_layers=4):
        super().__init__()

        self.feature_embedding = nn.Linear(13, d_model)

        encoder_layer = nn.TransformerEncoderLayer(
            d_model=d_model,
            nhead=nhead,
            dim_feedforward=512,
            dropout=0.1
        )
        self.transformer = nn.TransformerEncoder(encoder_layer, num_layers=num_layers)

        self.output_head = nn.Linear(d_model, 41)

    def forward(self, x):
        # x: [batch, 13]
        embedded = self.feature_embedding(x)  # [batch, d_model]
        embedded = embedded.unsqueeze(1).expand(-1, 41, -1)  # [batch, 41, d_model]

        transformed = self.transformer(embedded)  # [batch, 41, d_model]

        cd_pred = self.output_head(transformed[:, :, :]).squeeze(-1)  # [batch, 41]

        return cd_pred

Result: Mismatch between architecture and problem. CDM prediction isn't a sequence modeling task; Mach points are independent given bullet features. Performance: 6.05% MAE.

4. Neural ODE

Attempted to model drag as a continuous ODE:

from torchdiffeq import odeint

class DragODE(nn.Module):
    """Neural ODE for continuous drag modeling."""

    def __init__(self, hidden_dim=64):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(1 + 13, hidden_dim),  # Mach + features
            nn.Tanh(),
            nn.Linear(hidden_dim, hidden_dim),
            nn.Tanh(),
            nn.Linear(hidden_dim, 1)  # dCd/dM
        )

    def forward(self, t, state):
        # t: current Mach number
        # state: [Cd, features...]
        return self.net(torch.cat([t, state], dim=-1))

def predict_cdm(features, mach_points):
    """Integrate ODE to get Cd curve."""
    initial_cd = torch.tensor([0.5])  # Initial guess
    state = torch.cat([initial_cd, features])

    solution = odeint(ode_func, state, mach_points)

    return solution[:, 0]  # Extract Cd values

Result: Failed to converge due to dimension mismatch errors and extreme sensitivity to initial conditions. Abandoned after 2 days of debugging.

Architecture Comparison Results

Architecture	MAE	Smoothness	Shape Correlation	Status
MLP Baseline	3.66%	90.05%	0.9380	✅ Best
Physics-Informed NN	4.86%	64.02%	0.8234	❌ Over-regularized
Transformer	6.05%	56.83%	0.7891	❌ Poor fit
Neural ODE	---	---	---	❌ Failed to converge

Figure 5: Performance comparison across four neural architectures - MLP baseline wins

Key Insight: Simple MLP with dropout outperformed complex physics-constrained models. The training data already contained sufficient physics signal; explicit constraints hurt generalization.

Part 4: Production System Design

The POC model (3.66% MAE) validated the approach. Now we needed production hardening.

Training Pipeline Improvements

import pytorch_lightning as pl
from torch.utils.data import DataLoader, TensorDataset

class ProductionCDMModel(pl.LightningModule):
    """Production-ready CDM predictor with monitoring."""

    def __init__(self, learning_rate=1e-3, weight_decay=1e-4):
        super().__init__()
        self.save_hyperparameters()

        self.model = CDMPredictor(dropout=0.2)
        self.learning_rate = learning_rate
        self.weight_decay = weight_decay

        # Metrics tracking
        self.train_mae = []
        self.val_mae = []

    def forward(self, x):
        return self.model(x)

    def training_step(self, batch, batch_idx):
        features, cd_true = batch
        cd_pred = self(features)

        # Weighted MSE loss (emphasize transonic region)
        weights = self._get_mach_weights()
        loss = (weights * (cd_pred - cd_true)  2).mean()

        # Metrics
        mae = torch.abs(cd_pred - cd_true).mean()
        self.log('train_loss', loss)
        self.log('train_mae', mae)

        return loss

    def validation_step(self, batch, batch_idx):
        features, cd_true = batch
        cd_pred = self(features)

        loss = nn.MSELoss()(cd_pred, cd_true)
        mae = torch.abs(cd_pred - cd_true).mean()

        self.log('val_loss', loss)
        self.log('val_mae', mae)

        # Physics validation
        smoothness = self._calculate_smoothness(cd_pred)
        transonic_quality = self._check_transonic_peak(cd_pred)

        self.log('smoothness', smoothness)
        self.log('transonic_quality', transonic_quality)

        return loss

    def configure_optimizers(self):
        optimizer = torch.optim.AdamW(
            self.parameters(),
            lr=self.learning_rate,
            weight_decay=self.weight_decay
        )

        scheduler = torch.optim.lr_scheduler.ReduceLROnPlateau(
            optimizer,
            mode='min',
            factor=0.5,
            patience=5,
            verbose=True
        )

        return {
            'optimizer': optimizer,
            'lr_scheduler': scheduler,
            'monitor': 'val_loss'
        }

    def _get_mach_weights(self):
        """Weight transonic region more heavily."""
        weights = torch.ones(41)
        transonic_indices = (self.mach_points >= 0.8) & (self.mach_points <= 1.2)
        weights[transonic_indices] = 2.0  # 2x weight in transonic
        return weights / weights.sum()

    def _calculate_smoothness(self, cd_pred):
        """Measure curve smoothness (low = better)."""
        second_derivative = torch.diff(cd_pred, n=2, dim=-1)
        return 1.0 / (1.0 + second_derivative.abs().mean())

    def _check_transonic_peak(self, cd_pred):
        """Verify transonic peak exists and is realistic."""
        transonic_mask = (self.mach_points >= 0.8) & (self.mach_points <= 1.2)
        peak_cd = cd_pred[:, transonic_mask].max(dim=1)[0]
        baseline_cd = cd_pred[:, 0]  # Subsonic baseline

        return (peak_cd / baseline_cd).mean()  # Should be > 1.0

Training Configuration

# Data preparation
X_train, X_val, X_test = prepare_features()  # 1,039 → 831 / 104 / 104
y_train, y_val, y_test = prepare_targets()

train_dataset = TensorDataset(X_train, y_train)
val_dataset = TensorDataset(X_val, y_val)

train_loader = DataLoader(train_dataset, batch_size=32, shuffle=True, num_workers=4)
val_loader = DataLoader(val_dataset, batch_size=32, shuffle=False, num_workers=4)

# Model training
model = ProductionCDMModel(learning_rate=1e-3, weight_decay=1e-4)

trainer = pl.Trainer(
    max_epochs=100,
    callbacks=[
        pl.callbacks.EarlyStopping(monitor='val_loss', patience=10, mode='min'),
        pl.callbacks.ModelCheckpoint(monitor='val_mae', mode='min', save_top_k=3),
        pl.callbacks.LearningRateMonitor(logging_interval='epoch')
    ],
    accelerator='gpu',
    devices=1,
    log_every_n_steps=10
)

trainer.fit(model, train_loader, val_loader)

Figure 6: Training and validation loss convergence over 60 epochs

Training Results: - Converged at epoch 60 (early stopping) - Final validation loss: 0.0023 - Production model MAE: 3.15% (13.9% improvement over POC) - Smoothness: 88.81% (close to ground truth 89.6%) - Shape correlation: 0.9545

Figure 7: Example predicted CDM curves compared to ground truth measurements

API Integration

# ballistics/ml/cdm_transfer_learning.py

import torch
import pickle
from pathlib import Path

class CDMTransferLearning:
    """Production CDM prediction service."""

    def __init__(self, model_path: str = "models/cdm_transfer_learning/production_mlp.pkl"):
        self.model = self._load_model(model_path)
        self.model.eval()

        # Feature statistics for normalization
        with open(model_path.replace('.pkl', '_stats.pkl'), 'rb') as f:
            self.feature_stats = pickle.load(f)

    def predict(self, bullet_data: dict) -> dict:
        """Predict CDM curve from bullet specifications.

        Args:
            bullet_data: Dict with keys: caliber, weight_gr, bc_g1, bc_g7, etc.

        Returns:
            Dict with mach_numbers, drag_coefficients, validation_metrics
        """
        # Feature engineering
        features = self._extract_features(bullet_data)
        features_normalized = self._normalize_features(features)

        # Prediction
        with torch.no_grad():
            cd_pred = self.model(torch.tensor(features_normalized, dtype=torch.float32))

        # Denormalize
        cd_values = cd_pred.numpy()

        # Validation
        validation = self._validate_prediction(cd_values)

        return {
            'mach_numbers': self.mach_points.tolist(),
            'drag_coefficients': cd_values.tolist(),
            'source': 'ml_transfer_learning',
            'method': 'mlp_prediction',
            'validation': validation
        }

    def _validate_prediction(self, cd_values: np.ndarray) -> dict:
        """Physics validation of predicted curve."""
        return {
            'smoothness': self._calculate_smoothness(cd_values),
            'transonic_quality': self._check_transonic_peak(cd_values),
            'negative_cd_count': (cd_values < 0).sum(),
            'physical_plausibility': self._check_plausibility(cd_values)
        }

REST API Endpoint

# routes/bullets_unified.py

@bp.route('/search', methods=['GET'])
def search_bullets():
    """Search unified bullet database with optional CDM prediction."""
    query = request.args.get('q', '')
    use_cdm_prediction = request.args.get('use_cdm_prediction', 'true').lower() == 'true'

    # Search database
    results = search_database(query)

    cdm_predictions_made = 0

    if use_cdm_prediction:
        cdm_predictor = CDMTransferLearning()

        for bullet in results:
            if bullet.get('cdm_data') is None:
                # Predict CDM if not available
                try:
                    cdm_data = cdm_predictor.predict({
                        'caliber': bullet['caliber'],
                        'weight_gr': bullet['weight_gr'],
                        'bc_g1': bullet.get('bc_g1'),
                        'bc_g7': bullet.get('bc_g7'),
                        'length_in': bullet.get('length_in'),
                        'ogive_radius_cal': bullet.get('ogive_radius_cal')
                    })

                    bullet['cdm_data'] = cdm_data
                    bullet['cdm_predicted'] = True
                    cdm_predictions_made += 1

                except Exception as e:
                    logger.warning(f"CDM prediction failed for bullet {bullet['id']}: {e}")

    return jsonify({
        'results': results,
        'cdm_prediction_enabled': use_cdm_prediction,
        'cdm_predictions_made': cdm_predictions_made
    })

Example Response:

{
  "results": [
    {
      "id": 1234,
      "manufacturer": "Sierra",
      "model": "MatchKing",
      "caliber": 0.308,
      "weight_gr": 168,
      "bc_g1": 0.462,
      "bc_g7": 0.237,
      "cdm_data": {
        "mach_numbers": [0.5, 0.55, 0.6, ..., 4.5],
        "drag_coefficients": [0.287, 0.289, 0.295, ..., 0.312],
        "source": "ml_transfer_learning",
        "method": "mlp_prediction",
        "validation": {
          "smoothness": 91.2,
          "transonic_quality": 1.45,
          "negative_cd_count": 0,
          "physical_plausibility": true
        }
      },
      "cdm_predicted": true
    }
  ],
  "cdm_prediction_enabled": true,
  "cdm_predictions_made": 18
}

Part 5: Deployment and Monitoring

Model Serving Architecture

┌─────────────────┐
│   Client App    │
└────────┬────────┘
         │
         ▼
┌─────────────────────────┐
│  Google Cloud Function  │
│  (Python 3.12)          │
│  - Flask routing        │
│  - Request validation   │
│  - Response formatting  │
└────────┬────────────────┘
         │
         ▼
┌─────────────────────────┐
│  CDMTransferLearning    │
│  - PyTorch model (2.1MB)│
│  - CPU inference (<10ms)│
│  - Feature engineering  │
└────────┬────────────────┘
         │
         ▼
┌─────────────────────────┐
│  Physics Validation     │
│  - Smoothness check     │
│  - Peak detection       │
│  - Plausibility gates   │
└─────────────────────────┘

Performance Characteristics

Model Size: - PyTorch state dict: 2.1 MB - TorchScript (optional): 2.3 MB - ONNX (optional): 1.8 MB

Inference Speed (CPU): - Single prediction: 6-8 ms - Batch of 10: 12-15 ms (1.2-1.5 ms per bullet) - Batch of 100: 80-100 ms (0.8-1.0 ms per bullet)

Cold Start: - Model load time: 150-200 ms - First prediction: 220-280 ms (including load) - Subsequent predictions: 6-8 ms

Memory Footprint: - Model in memory: ~15 MB - Peak during inference: ~30 MB

Figure 8: Production inference performance metrics across different batch sizes

Monitoring and Observability

import newrelic.agent

class MonitoredCDMPredictor:
    """CDM predictor with New Relic monitoring."""

    def __init__(self):
        self.predictor = CDMTransferLearning()
        self.prediction_count = 0
        self.error_count = 0

    @newrelic.agent.function_trace()
    def predict(self, bullet_data: dict) -> dict:
        """Predict with telemetry."""
        self.prediction_count += 1

        try:
            # Track prediction time
            with newrelic.agent.FunctionTrace(name='cdm_prediction'):
                result = self.predictor.predict(bullet_data)

            # Custom metrics
            newrelic.agent.record_custom_metric('CDM/Predictions/Total', self.prediction_count)
            newrelic.agent.record_custom_metric('CDM/Validation/Smoothness',
                                               result['validation']['smoothness'])
            newrelic.agent.record_custom_metric('CDM/Validation/TransonicQuality',
                                               result['validation']['transonic_quality'])

            # Track feature availability
            features_available = sum(1 for k, v in bullet_data.items() if v is not None)
            newrelic.agent.record_custom_metric('CDM/Features/Available', features_available)

            return result

        except Exception as e:
            self.error_count += 1
            newrelic.agent.record_custom_metric('CDM/Errors/Total', self.error_count)
            newrelic.agent.notice_error()
            raise

Key Metrics Tracked: - Prediction latency (p50, p95, p99) - Validation scores (smoothness, transonic quality) - Feature availability (how many inputs provided) - Error rate and types - Cache hit rate (if caching enabled)

Lessons Learned

1. Simple Architectures Often Win

We spent a week exploring Transformers and Neural ODEs, only to find the vanilla MLP performed best. Why?

Data alignment: Our problem is function approximation, not sequence modeling
Inductive bias mismatch: Transformers expect temporal dependencies; drag curves don't have them
Regularization sufficiency: Dropout + weight decay provided enough regularization without physics constraints

Lesson: Start simple. Add complexity only when data clearly demands it.

2. Physics Validation > Physics Loss

Hard-coded physics loss functions became a liability: - Over-constrained the model - Required manual tuning of loss weights - Didn't generalize to all bullet types

Better approach: Validate predictions post-hoc and flag anomalies. Let the model learn physics from data.

3. Synthetic Data Quality Matters More Than Quantity

We generated 704 synthetic CDMs, but spent equal time validating them. Key insight: One bad synthetic sample can poison dozens of real samples during training.

Validation process: 1. Compare synthetic vs. real CDMs (where both exist) 2. Physics plausibility checks 3. Cross-validation with different BC values 4. Manual inspection of outliers

4. Feature Engineering > Model Complexity

The most impactful changes weren't architectural: - Adding sectional_density as a feature: -0.8% MAE - Computing form_factor_g1 and form_factor_g7: -0.6% MAE - Imputing missing features (length, ogive) using physics-based defaults: -0.5% MAE

Figure 9: Feature importance analysis showing impact of each input feature on prediction accuracy

Combined improvement: -1.9% MAE with zero code changes to the model.

5. Production Deployment ≠ POC

Our POC model worked great in notebooks. Production required: - Input validation and sanitization - Graceful degradation when features missing - Physics validation gates - Monitoring and alerting - Model versioning and rollback capability - A/B testing infrastructure

Time split: 30% research, 70% production engineering.

What's Next?

Phase 2: Uncertainty Quantification

Current model outputs point estimates. We're implementing Bayesian Neural Networks to provide confidence intervals:

class BayesianCDMPredictor(nn.Module):
    """Bayesian NN with dropout as approximate inference."""

    def predict_with_uncertainty(self, features, n_samples=100):
        """Monte Carlo dropout for uncertainty estimation."""
        self.train()  # Enable dropout during inference

        predictions = []
        for _ in range(n_samples):
            with torch.no_grad():
                pred = self(features)
                predictions.append(pred)

        predictions = torch.stack(predictions)

        mean = predictions.mean(dim=0)
        std = predictions.std(dim=0)

        return {
            'cd_mean': mean,
            'cd_std': std,
            'cd_lower': mean - 1.96 * std,  # 95% CI
            'cd_upper': mean + 1.96 * std
        }

Use case: Flag predictions with high uncertainty for manual review or experimental validation.

Conclusion

Building a production ML system for ballistic drag prediction required more than just training a model: - Data engineering (Claude Vision automation saved countless hours) - Synthetic data generation (2.1× data augmentation) - Architecture exploration (simple MLP won) - Real-world validation (94% physics check pass rate)

The result: 1,247 bullets now have accurate drag models that didn't exist before. Not bad for a side project.

Read the full technical whitepaper for mathematical derivations, validation details, and complete bibliography: cdm_transfer_learning.pdf

Resources

References: 1. McCoy, R. L. (1999). Modern Exterior Ballistics. Schiffer Publishing. 2. Litz, B. (2016). Applied Ballistics for Long Range Shooting (3rd ed.).