The SpacemiT K3 on Mainline Ubuntu: KVM Works, and the BSDs Cannot Boot

In the first piece on this board I said the SpacemiT K3 Pico-ITX was genuinely competitive silicon shipped with a kernel that could not load KVM. I bought it to run virtual machines. It cost \$1,175.09 delivered and modprobe kvm oopsed in jump_label_module_notify.

If you did not read that one, the short version: the K3 Pico-ITX is a RISC-V single board computer with 32 GB of RAM and sixteen cores, split into two groups of eight. Eight X100 cores do ordinary work. Eight A100 cores are an AI engine rated at 60 TOPS. It is the first RISC-V board I have tested that competes with ARM on speed rather than apologising for itself, and at \$1,175.09 it needs to.

Two words will come up constantly here, so they are worth pinning down. The vendor kernel is the Linux that SpacemiT ships, which they maintain themselves and which knows every quirk of their hardware. Mainline is the Linux everyone else gets, where support has been accepted into the official kernel tree. Vendor kernels usually work better and age worse. The interesting moment for any new board is when mainline catches up, because that is when the board stops depending on one company's attention.

Ubuntu 26.10 was the first release where this board's support is upstream rather than carried in SpacemiT's vendor tree. So I flashed EDK2 over the vendor U-Boot, installed Ubuntu, and re-ran everything.

KVM works. That is the headline and it is a real result:

$ sudo modprobe kvm && lsmod | grep kvm
kvm    466944  10
$ ls -l /dev/kvm
crw-rw---- 1 root kvm 10, 232 /dev/kvm

A Debian 13 riscv64 guest boots under -cpu host and inherits the X100's vector and vector-crypto extensions. That h on the end of the ISA string rv64imafdcvh is the hypervisor extension, the optional block of silicon that lets a RISC-V chip run virtual machines at full speed instead of simulating them. The K3 has always had it. Until now no kernel on this board could use it. The board does the thing I bought it for.

Everything else I found is worse than that sentence suggests. Including, first, the part where I could not install it.

Three failed installs and a serial cable

The supported path replaces SpacemiT's bootloader with EDK2, an open implementation of UEFI, the same firmware interface a PC uses to find and start an operating system. Swapping it in is what lets the board boot an ordinary Ubuntu USB stick instead of something hand-assembled for this hardware. The swap happens over fastboot, with the board held in a special flashing mode. The flash is the easy part. Hold the FDL button, apply power to the first USB-C connector, release, and fastboot devices answers. Six partitions get written and edk2.itb lands in the one confusingly named uboot. That worked first time.

The install did not. Three times the board booted the installer, appeared on the network as ubuntu-server, and then sat there. For hours. No reboot, no progress, no way in. The DHCP lease renewed, ICMP answered, port 22 was open and refused every credential. From outside, a working install and a dead one look identical.

I spent a long time constructing theories from that non-evidence. The storage match in my autoinstall config must be wrong. The kernel must not see the UFS. The firmware must have a dangling boot entry. Each theory was plausible, each led to another blind retry, and every one of them was wrong.

What broke the deadlock was a three-wire USB-to-TTL serial adapter on the header silkscreened DEBUG, next to the USB-C connectors. Any 3.3 V adapter works; mine is an older PL2303 and a CP2102 is the better buy today. Whatever you use, it must be 3.3 V TTL and not a true RS-232 cable, which swings to plus and minus twelve volts and will destroy the UART. The board's own device tree tells you where the console goes:

chosen  { stdout-path = "serial0:115200n8"; }
aliases { serial0 = "/soc/serial@d4017000"; }

Thirty seconds after attaching it, the answer was on screen. Two answers, actually.

The first was the UFS error storm described below, saturating an 11.5 KB/s UART with 117 KB of messages every ten seconds. The second was better:

Installer update available
[ Update to the new installer ]
[ Continue without updating   ]

That prompt only appears when autoinstall is not active. Autoinstall is Ubuntu's unattended install mechanism: you leave a configuration file on the media, the installer reads it, and it answers every question for you without a human present. It is how you install a machine that has no screen. My configuration had never been applied on any attempt. Every one of those multi-hour "hangs" was the plain interactive installer, sitting at a prompt, waiting for a keypress from a machine with no keyboard and no screen.

There is a second trap stacked on the first. Because the device tree pins the console to serial, subiquity starts in serial mode and opens with a dialog asking whether you want rich mode or basic mode. It waits. An unattended install on a board whose console is a UART will stop dead at that question, and nothing about it reaches the network.

So I drove the installer over the serial line instead, one keystroke at a time, which is how Ubuntu actually got onto this board. It is worth knowing before you buy a headless RISC-V machine and assume autoinstall will carry you.

Mainline sees half the cores, and not the half you'd hope

Bianbu listed sixteen processors and let a process use one cluster of eight or the other, never both. I assumed a mainline kernel would either fix the partition or at least expose the same hardware.

It does neither:

possible : 0-7
present  : 0-7
online   : 0-7
uarch    : spacemit,x100  (x8)
/proc/set_ai_thread : absent

The A100 cluster is simply not enumerated. Bianbu at least showed all sixteen in /proc/cpuinfo and gave you /proc/set_ai_thread to reach the AI cores. Mainline does not know they exist. The vendor hook is gone with it.

So the AI cores are less accessible on Ubuntu than on Bianbu, not more.

The practical consequence is that the 110-second sixteen-core compile I speculated about in the first piece is not merely blocked, it is unreachable. There is no cluster to schedule onto. The A100 figure from that piece, 342.78 s against the X100's 161.29 s on the same tree, was measured through the vendor's /proc/set_ai_thread hook. That hook does not exist on mainline, so on Ubuntu there is no way to run anything on those cores at all, fast or slow.

One trap worth naming, because I fell into it and reported the wrong answer for a few minutes: taskset -c 0-15 succeeds here. It returns zero and looks like a win. It intersects the mask you asked for with the CPUs that exist and quietly gives you 0-7. Check the resulting affinity, not the exit code.

The compile got slower, and it is not the clock

Same board, same tree at 2821bba, same rustc 1.98.1. Mainline is 13.4% slower on the compile.

My first instinct was thermal throttling, because the three mainline runs climbed steadily (181.0, 184.2, 185.1) where Bianbu's were flat. That instinct was wrong. Single-threaded sysbench is a decent clock proxy and it is identical, within 0.2%. The eight-thread figure is actually 0.9% faster on mainline. Raw CPU throughput has not changed.

So something other than the CPU costs 13% on a compile. The most likely candidate is storage: root moved from the internal UFS to NVMe behind LVM, and a clean cargo build is I/O heavy. I have not proven it, and I am not going to claim a kernel regression I cannot demonstrate.

Two things make this comparison less clean than I would like, and both are worth stating rather than burying:

• Mainline exposes no cpufreq policy and no thermal zones for this SoC. On Bianbu the governor was userspace pinned at 2.2 GHz and there were seven thermal zones. On Ubuntu the clock cannot be read, pinned, or checked for throttling.
• OpenSSL went from 3.5.5 to 4.0.1 between the two installs, so the crypto numbers are not comparable at all. I am not publishing a vector-crypto delta that is really a library version delta.

The internal storage does not work

The K3 has 119 GB of onboard UFS storage, which is the faster successor to the eMMC chips most single board computers solder down. On mainline it does not exist:

ufshcd-spacemit c0e00000.ufshc: freq-table-hz property not specified
ufshcd-spacemit c0e00000.ufshc: ufshcd_populate_vreg: Unable to find vcc-supply regulator, assuming enabled
ufshcd-spacemit c0e00000.ufshc: DL error event, INT errors:0x4, DL_ERR:0x80000002
  ... ~1600 more error lines ...
ufshcd-spacemit c0e00000.ufshc: ufshcd_async_scan failed: -19

The device tree is a data file the firmware hands the kernel describing what hardware is present and how it is wired: which controller sits at which address, which power rails feed it, what clock speeds it supports. Linux has no other way to discover any of this on a board like the K3. The K3's device tree declares the storage controller at ufshc@c0e00000 but supplies no vcc/vccq regulator entries and no freq-table-hz. The driver assumes the rails are live, the device never answers, and link training fails at the Data Link and PHY Adapter layers until the controller gives up in eh_fatal.

The driver binds correctly, and the alias matches the board exactly (of:N*T*Cspacemit,k3-ufshc against the DTB's compatible = "spacemit,k3-ufshc"). The DTB is simply incomplete. Ubuntu ships and manages that DTB as a package, and I watched update-initramfs print Installing new k3-pico-itx.dtb while blacklisting the driver, so this is fixable upstream without new silicon and without a vendor release.

Until then the internal storage is unusable and root has to live on NVMe. Mine is a WD_BLACK SN850X, which is overkill for a boot disk and was already in the board from the first review. I blacklisted ufs_spacemit, which reclaims about forty seconds of boot and 1,600 error lines.

That error storm is worth dwelling on, because it nearly cost me the whole install. The K3's device tree pins the kernel console to serial0 at 115200, which is about 11.5 KB/s. The UFS failure generates roughly 117 KB of error messages every ten seconds. The machine spends essentially all of its time writing errors to a UART whether or not anything is listening. From the outside it looks exactly like a hang.

The finding I did not expect

I wanted to see the board do real work, so I set out to build four RISC-V guests: Debian, FreeBSD, NetBSD and OpenBSD.

Debian booted under KVM and ran fine. All three BSDs panicked about one second in.

FreeBSD:

panic: Illegal instruction 0xc0002573 at 0xffffffc00042e362
#5 0xffffffc00042e03e at arc4rand+0x8e

NetBSD, independently:

panic: cpu_trap: fatal kernel trap  (va=0xc0002773)

Those two values are the same instruction with a different destination register. Pulling 0xc0002573 apart: the low seven bits are the opcode, 0x73, which on RISC-V means a system instruction. The middle field says it is a CSR read. The top twelve bits are the register number, 0xC00. That register is cycle, a free-running count of CPU clock ticks, and the instruction that reads it is spelled rdcycle.

Both kernels were dying on the same one-instruction question: how many cycles have elapsed?

A detour into who is allowed to do what

To explain why that question is illegal, I need one piece of RISC-V background, and it is the piece that makes everything else in this post make sense.

RISC-V software runs at one of three privilege levels. Machine mode is the most privileged and is where firmware lives: on this board that is SpacemiT's OpenSBI, burned into flash, running before Linux and continuing to sit underneath it forever. Supervisor mode is where the operating system kernel runs. User mode is where your programs run. Each level can see and do strictly less than the one below it.

Crucially, the lower level decides what the upper levels are permitted to touch. Machine-mode firmware holds a register called mcounteren, the counter-enable register, with one bit per hardware counter. Set the bit and everyone above may read that counter. Leave it clear and any attempt to read it is an illegal instruction, exactly as if the CPU did not implement it at all.

So rdcycle failing is not a bug in the CPU, the kernel, or the hypervisor. It is firmware declining to grant permission, and there is nothing any layer above can do about it.

My first theory was still the wrong one. I assumed RISC-V's KVM does not grant guests direct counter access, and that a guest is simply more restricted than its host. Wrong. The host cannot do it either:

$ cat rdcycle.c
int main(void){ unsigned long c; __asm__ volatile("rdcycle %0":"=r"(c)); return 0; }
$ ./rdcycle
Illegal instruction (core dumped)

That is plain userspace on the host, no virtualization anywhere. Testing all three counters:

The K3's M-mode firmware sets mcounteren.TM but not CY or IR. Nothing above M-mode can read the cycle or instruction counters: not host userspace, not the host kernel, not any guest.

FreeBSD's arc4rand and NetBSD's early init both execute rdcycle. So they panic on this board whether virtualized or not. This is not a KVM limitation and not a BSD bug. Linux is unaffected only because riscv64 Linux goes through SBI and the time CSR instead of reading cycle directly.

I confirmed it the other way too. QEMU can run a guest two ways. KVM uses the host's own CPU to execute guest code directly, which is fast but means the guest inherits the host's hardware, firmware restrictions included. TCG ignores the hardware and simulates a RISC-V processor in software, which is slow but means the simulated CPU behaves the way the specification says rather than the way this board does. Under TCG, which emulates its own machine mode and its own counters, FreeBSD 15.1 and NetBSD 11.99.8 both boot to a root shell without complaint. The images are fine. The board is not.

It is worth being precise about what this is not. NetBSD runs perfectly well on RISC-V: I have installed it on the Milk-V Mars and chased a firmware bug through its page tables on that board. So I ran the same three-counter test on the Pine64 Star64, a JH7110 board sitting on the same bench:

Same operating systems, same architecture, same room. The JH7110's firmware grants all three counters and NetBSD boots on it. The K3's grants one. That is the whole difference, and it is a decision made in M-mode firmware rather than anything about RISC-V, NetBSD, or KVM.

One practical note if you try this yourself: BSD guests need virtio-blk-device (MMIO) rather than if=virtio, which gives virtio-blk-PCI. NetBSD does not attach the PCI variant and drops to root device: with only vioif0 offered.

What a guest actually gets

Since the whole point was virtualization, it is worth saying what running a guest on this board is actually like.

The first thing that bites is firmware layering, and it is not documented anywhere obvious. Under KVM the host's OpenSBI already owns M-mode, so the guest must start in supervisor mode. Passing QEMU an M-mode firmware fails outright:

qemu-system-riscv64: Machine mode firmware is not supported in combination with KVM.

The invocation that works is -bios none plus the S-mode U-Boot as the kernel, which then chainloads the guest's own bootloader over EFI. Under TCG the opposite is true: QEMU emulates the entire machine including M-mode, so OpenSBI has to be loaded with -bios default. The same VM definition cannot serve both. I spent a while watching a TCG guest produce absolutely no output because I had carried the KVM arguments over unchanged, and with no M-mode firmware there was nothing to run the S-mode payload.

What the guest sees is better than I expected. With -cpu host, the Debian guest's ISA string comes through as:

rv64imafdcv_zicbom_..._zvkned_zvknha_zvknhb_zvksed_zvksh_zvkt

Those last few are the vector crypto extensions, dedicated instructions for AES, SHA-2, SM4 and SM3. They are why this board pushes 2.37 GB/s of AES-GCM where an older RISC-V board manages 21.7 MB/s. The guest is not seeing a sanitised baseline RISC-V core, it is seeing the X100's actual feature set, which means the AES-GCM throughput that made this board interesting in the first place is available inside a VM. That is the part of this that genuinely works well.

The guest ISA does not carry h, so there is no nested virtualization. That is expected and I would not have used it.

I cannot tell you much about how many guests this board holds, because I never found out. Three reached a shell: Debian under KVM, FreeBSD and NetBSD under TCG. OpenBSD got as far as extracting base78.tgz and never finished, because by then it was clear that every BSD here was going to be an emulated guest and I stopped all three rather than let them grind. With those four QEMU processes up, one of which was only an installer, the host sat at load 2.07 and 5 GB of 31 GB. That is not a capacity test and I am not going to dress it up as one. It says only that the board was nowhere near its limits when I gave up on the experiment.

Which kernel should you actually run

That leaves a question the first piece could not ask, because there was only one option. Now there are two, and neither wins outright.

Bianbu 4.0.4 with the vendor 6.18.3 BSP compiles 13.4% faster, drives the internal 119 GB UFS, exposes cpufreq so the clock can be pinned at 2.2 GHz, reports seven thermal zones, and enumerates all sixteen cores with a documented way to reach the A100 cluster. It cannot load KVM.

Ubuntu 26.10 with mainline 7.3.0 loads KVM and runs accelerated guests. It cannot use the internal storage, shows no clock or temperature at all, and pretends eight of the cores do not exist.

If you bought this board to compile things, the vendor kernel is still the better machine and it is not close. If you bought it to run VMs, which I did, mainline is the only option and you pay for it in storage, visibility and cores.

What makes that trade-off worse than it sounds is that the losses are not symmetric. A slower compile is an inconvenience. Losing the internal storage means the NVMe slot is no longer an expansion option, it is a requirement, and the board's advertised 128 GB becomes decorative. Losing cpufreq and thermal zones means you cannot tell whether a long job is throttling. And losing the A100 cluster removes the 60 TOPS that justify a large part of the \$1,175.09.

I keep coming back to that number. The first piece argued the price was defensible because you get 32 GB, a hypervisor extension and an AI engine. Two of those three now work. The AI engine is reachable only from a kernel that cannot virtualize, which means on any given boot you can have the feature you paid for or the feature you paid for, but not both.

So where does that leave it

I stopped the BSDs. Emulating RISC-V on a RISC-V host defeats the entire point of buying hardware with the hypervisor extension, and a TCG guest on this board is slower than the same guest on a laptop. They were worth building only to find out why they would not run, and that turned out to be the most interesting thing I learned all week.

The honest summary is narrower than "KVM works":

KVM works, for Linux guests. The K3 is a capable Linux hypervisor with a lot of memory. It is also, right now, a hostile host for the BSDs at the firmware level, it cannot use its own internal storage on a mainline kernel, and it hides half its cores from the only kernel that can virtualize properly.

Every one of those is a software or firmware problem rather than a silicon one, which is the same conclusion I reached the first time. The difference is that the list is now specific enough to fix: grant CY and IR in mcounteren, fill in the UFS regulators in the device tree, and describe the A100 cluster upstream.

One more piece to come

Two and a half weeks before this board arrived, I costed a RISC-V rental business in public: two K3 Pico-ITX nodes in a 1U at a Miami colo, roughly \$47 to \$57 per node per month, and a question to readers about whether anyone would actually pay for it. The central pitch was that the hypervisor extension is the one thing you cannot rent from Scaleway or anybody else today.

The best news is that the pitch survives. KVM works, so the differentiator is real rather than theoretical, and the estimate I published without owning the hardware turned out to describe the right machine. The design held up better than I did, too: that plan already budgeted a Raspberry Pi as the BMC holding a USB-UART console to each board, and NVMe drives at about \$120 the pair. Both of those turned out to be exactly the things this week proved you cannot run this board without.

What changes is subtler and mostly unflattering. Renting virtual machines forces the mainline kernel, and on mainline the 128 GB of onboard storage in that \$399 board specification is dead weight, because the UFS controller never links. You are paying for storage you cannot use. The BSD finding removes a customer segment that a RISC-V rental should have been well placed to serve. And mainline reports no clock and no temperature at all, which means a tenant asking whether their node is throttling gets an honest shrug. A BMC can power-cycle a wedged board, but it cannot report numbers the kernel refuses to collect.

There is one operational gap worth designing around. The BMC switches 12 V and watches the console, which covers a hung node. It does not cover firmware recovery, because entering flash mode on this board means holding a physical button while power is applied. That is a GPIO line and a bit of wiring rather than a flight to Miami, but it is not in the current design and I only know it matters because I spent this week needing it.

I want a few more weeks of uptime before I redo those numbers. The next piece will be that arithmetic revisited by someone who now owns the hardware instead of estimating it.

Caveats. The compile comparison has an unresolved 13% gap with a confirmed-identical CPU throughput on both sides; root storage differs between the two installs and is the leading suspect. Clocks could not be pinned or verified on mainline because no cpufreq policy is exposed. Crypto figures are not compared across installs because the OpenSSL version changed. The A100 cluster was not benchmarked because mainline does not enumerate it. The virtualization experiment was abandoned rather than completed: three guests reached a shell, OpenBSD's install was stopped part-way, and no attempt was made to find out how many guests the board will hold. All raw data is in the benchmark repository.