TinyComputers.io (Posts about open source hardware)

Milk-V Mars Review: SiFive's RISC-V Enters the SBC Arena

A.C. Jokela — Fri, 13 Feb 2026 21:30:00 GMT

Introduction

The Milk-V Mars is a compact RISC-V single board computer built around StarFive's JH7110 system-on-chip, featuring four SiFive U74-MC application cores and 8GB of LPDDR4 RAM. Priced affordably and shipping in a Raspberry Pi-like form factor, the Mars targets developers and enthusiasts who want to explore the RISC-V ecosystem on real hardware without investing in high-end development boards.

This review arrives at an interesting moment for RISC-V single board computers. We've already benchmarked the Orange Pi RV2 with its 8-core Ky(R) X1 processor, and the Milk-V Mars gives us a second data point in the RISC-V SBC landscape - this time with SiFive's cores, the company widely regarded as the pioneer of commercial RISC-V silicon. How does a board built on SiFive's mature U74 core design compare to the newer Ky X1, and where does it land against the ARM and x86 competition? The answers are illuminating, if not entirely encouraging.

Hardware Architecture: SiFive U74-MC on StarFive JH7110

At the heart of the Milk-V Mars sits the StarFive JH7110, a 28nm SoC that pairs four SiFive U74-MC application cores with a single SiFive S7 monitor core. The U74 is SiFive's 64-bit application processor core implementing the RV64GC instruction set (rv64imafdc) - the standard RISC-V profile with integer multiplication, atomic operations, single and double-precision floating point, and compressed instructions. The cores feature an in-order, dual-issue, 8-stage pipeline with separate L1 instruction and data caches, backed by a shared 2MB L2 cache. For readers wanting to understand the ISA itself, The RISC-V Reader is an excellent and concise introduction to the architecture and its design rationale.

The SiFive U74 has an important place in RISC-V history. As one of the first commercially available application-class RISC-V cores, it powered the HiFive Unmatched development board that many early RISC-V adopters used to explore the architecture. The U74-MC variant in the JH7110 is a cost-optimized version without the memory management features of the full U74, targeting embedded and SBC applications rather than server or workstation use.

However, the U74's in-order pipeline design places it at a fundamental architectural disadvantage compared to modern out-of-order ARM cores. While the Cortex-A76 in a Raspberry Pi 5 can speculatively execute instructions, reorder them for optimal throughput, and predict branches with sophisticated algorithms refined over years of iteration, the U74 executes instructions strictly in program order. This simplicity reduces silicon area and power consumption but significantly limits single-threaded performance - the metric that matters most for many real-world workloads.

The JH7110 is fabricated on TSMC's 28nm process, which is two to three generations behind the 8nm and 6nm processes used in contemporary ARM SoCs like the Rockchip RK3588. This older process node limits clock speeds, increases power consumption per transistor, and constrains the amount of logic that can be economically integrated into the die.

Board Specifications

Our test unit came configured with:

SoC: StarFive JH7110 (28nm process)
CPU: 4x SiFive U74-MC @ 1.5 GHz (RV64GC)
Monitor Core: 1x SiFive S7
RAM: 8GB LPDDR4
Storage: microSD card (29GB card installed, expandable)
Networking: Gigabit Ethernet
OS: Debian GNU/Linux Bookworm (kernel 5.15.0)
USB: USB 3.0 and USB 2.0 ports
Video: HDMI output
GPIO: 40-pin header (Raspberry Pi compatible layout)

The 8GB of RAM is generous for a board in this class - more than many ARM SBCs ship with by default - and proved more than adequate for our Rust compilation benchmarks. The microSD card storage is adequate but not exceptional; the board does support eMMC modules for better storage performance, though our test unit used a standard SD card.

One notable issue we encountered during setup: the factory image shipped with a root partition of only 3.8GB on a 29GB SD card. This is common with SBC images but particularly problematic here because installing the Rust toolchain and build dependencies requires significant disk space. We expanded the partition using parted and resize2fs before proceeding - a standard operation, but one that new users might find daunting.

Software Environment

The Milk-V Mars runs Debian Bookworm with a Linux 5.15.0 kernel, a relatively old kernel version compared to the 6.x kernels shipping on more recent ARM and RISC-V boards. The kernel includes StarFive-specific patches for JH7110 hardware support, but the older base means newer kernel features and optimizations aren't available.

The software experience is functional but bare. The system ships as a minimal Debian installation - no build tools, no git, no development packages preinstalled. Setting up the board for development required installing build-essential, git, curl, and bc via apt before we could proceed with Rust installation. The Debian repositories worked without issues, and rustup installed cleanly for the riscv64gc-unknown-linux-gnu target, delivering Rust 1.93.1 - the latest stable release at the time of testing.

This is actually a positive sign for the RISC-V ecosystem. Rust's official support for RISC-V has matured to the point where rustup "just works" on riscv64gc targets. Compare this to just a few years ago when RISC-V Rust development required building the compiler from source. The toolchain infrastructure has come a long way, even if the hardware performance hasn't yet caught up to ARM. If you're new to Rust and want to understand the language driving these benchmarks, The Rust Programming Language by Klabnik and Nichols is the definitive starting point.

Performance Testing: Rust Compilation Benchmarks

To assess the Milk-V Mars's CPU performance, we ran our standard benchmark: compiling the ballistics-engine Rust project in release mode, three times from a clean build state. This workload exercises all CPU cores through parallel compilation units, stresses the memory subsystem with large intermediate data structures, and measures real-world compiler and linker performance - the kind of task a developer would encounter daily.

Milk-V Mars Compilation Times

Run 1: 939.13 seconds (15 minutes 38 seconds)
Run 2: 933.45 seconds (15 minutes 33 seconds)
Run 3: 935.96 seconds (15 minutes 35 seconds)
Average: 936.18 seconds (15 minutes 36 seconds)
Standard Deviation: 2.85 seconds (0.30%)

The first thing that stands out is the remarkable consistency. A standard deviation of just 2.85 seconds across three 15-minute runs indicates stable thermal behavior and no throttling - the board maintained consistent performance throughout the entire test sequence. This suggests good power delivery and adequate thermal management, even without active cooling.

The second thing that stands out is the absolute time: over 15 minutes per compilation. This is the slowest result in our entire benchmark fleet.

Comparative Analysis

Here's how the Milk-V Mars stacks up against every board we've benchmarked:

Rank	System	Architecture	CPU	Cores	Average Time	vs. Mars
1	Orange Pi 5 Max	ARM64	Cortex-A55/A76	8 (4+4)	62.31s	15.0x faster
2	Raspberry Pi CM5	ARM64	Cortex-A76	4	71.04s	13.2x faster
3	LattePanda IOTA	x86_64	Intel N150	4	72.21s	13.0x faster
4	Raspberry Pi 5	ARM64	Cortex-A76	4	76.65s	12.2x faster
5	Banana Pi CM5-Pro	ARM64	Cortex-A53/A72	8 (4+4)	167.15s	5.6x faster
6	Horizon X3 CM	ARM64	Cortex-A53	4	378.81s	2.5x faster
7	Orange Pi RV2	RISC-V	Ky(R) X1	8	650.60s	1.4x faster
8	Milk-V Mars	RISC-V	SiFive U74-MC	4	936.18s	baseline

The Milk-V Mars is 15x slower than the fastest board in our fleet (Orange Pi 5 Max) and 12.2x slower than the ubiquitous Raspberry Pi 5. Even the much-maligned Horizon X3 CM with its ancient Cortex-A53 cores is 2.5x faster.

Understanding the Performance Gap

The 15x gap between the Milk-V Mars and the Orange Pi 5 Max isn't surprising when you understand the architectural differences at play:

In-order vs. out-of-order execution. The U74's in-order pipeline means the CPU stalls whenever it encounters a cache miss, branch misprediction, or data dependency. Modern out-of-order cores like the Cortex-A76 can continue executing independent instructions while waiting for stalled operations to complete. For a workload like Rust compilation, which involves complex data structures, frequent branching, and irregular memory access patterns, out-of-order execution provides enormous benefits. Hennessy and Patterson's Computer Architecture: A Quantitative Approach covers the engineering trade-offs between in-order and out-of-order pipelines in detail.

Clock speed. The U74 runs at 1.5 GHz compared to 2.4 GHz for the Cortex-A76 cores in the Pi 5 and Orange Pi 5 Max. This 1.6x frequency disadvantage compounds with the architectural differences to create a much larger effective performance gap.

Process technology. The JH7110's 28nm fabrication limits the transistor budget available for performance-enhancing features like larger caches, more complex branch predictors, and wider execution units. Modern ARM SoCs on 8nm or smaller processes can pack significantly more logic into the same power envelope.

Compiler maturity. While LLVM's RISC-V backend has improved substantially, it still lacks many of the target-specific optimizations that the ARM backend has accumulated over years of development. Code generation for RISC-V may not always exploit the microarchitecture as effectively as ARM code generation exploits Cortex-A76 features.

Core count. The Mars has four cores versus eight on the Orange Pi 5 Max. For parallel compilation workloads, more cores translate directly to faster builds, all else being equal.

RISC-V vs. RISC-V: Mars vs. Orange Pi RV2

Perhaps the most interesting comparison is between the two RISC-V boards in our fleet. The Orange Pi RV2 with its 8-core Ky(R) X1 processor compiled our benchmark in 650.60 seconds - 1.44x faster than the Milk-V Mars's 936.18 seconds.

This 44% performance advantage for the Orange Pi RV2 can be attributed to two factors:

Double the cores. The Ky X1 has eight cores versus the U74's four. For parallel compilation, this provides a near-linear speedup for the parallel portions of the build, though Amdahl's Law limits the overall benefit due to serial bottlenecks in linking and code generation.

Newer core design. The Ky X1 represents a more recent RISC-V core design than the U74, likely incorporating microarchitectural improvements that boost instructions-per-clock (IPC). While details about the Ky X1's pipeline are scarce, its performance-per-core appears to be modestly better than the U74's.

However, both RISC-V boards exist in the same performance tier - dramatically slower than ARM and x86 alternatives. The 286-second gap between the Mars and the RV2 is notable but pales in comparison to the 874-second gap between the Mars and the Orange Pi 5 Max. Both RISC-V platforms are firmly in "pioneer hardware" territory, demonstrating architectural viability rather than competitive performance.

The SiFive Legacy: Historical Context

It's worth acknowledging what the SiFive U74 represents historically. SiFive, founded in 2015 by the creators of the RISC-V instruction set at UC Berkeley, was the first company to offer commercial RISC-V cores. The U74 was their first serious application-class core, designed at a time when RISC-V was still primarily an academic project.

The HiFive Unmatched, which used the same JH7110-predecessor (the FU740), was one of the first RISC-V boards capable of running a full Linux desktop. It proved that RISC-V could work for general-purpose computing. The Milk-V Mars, using the JH7110 with U74-MC cores, is essentially a cost-reduced descendant of that pioneering effort.

In this light, the U74's performance is less a failure and more a measure of how far the RISC-V ecosystem has come - and how far it still needs to go. The U74 was designed for correctness and compatibility, not performance leadership. Newer SiFive cores like the P670 and P870 incorporate out-of-order execution, branch prediction, and other modern features that should dramatically close the gap with ARM. We haven't yet seen these cores in affordable SBC form factors, but when they arrive, the comparison should be far more competitive.

Practical Considerations

Who Should Buy the Milk-V Mars?

RISC-V learners and enthusiasts. If you want to understand RISC-V at a hardware level - boot Linux, explore the ISA, compile software, and experiment with the architecture - the Mars is a low-cost entry point. The 8GB of RAM is generous, and the Debian software environment is functional enough for exploration.

Embedded RISC-V developers. If you're developing software that will eventually run on RISC-V embedded systems, the Mars provides a Linux-capable development and testing platform. Cross-compilation workflows (develop on x86, test on RISC-V) are viable, and native compilation works - it just takes patience.

SBC collectors and architecture enthusiasts. The Mars represents an important moment in RISC-V history. As SiFive's application-class cores in an affordable SBC, it has both practical and historical value for anyone tracking the evolution of processor architectures.

Who Should Look Elsewhere?

Anyone needing competitive performance. If compilation speed, application responsiveness, or computational throughput matter for your use case, the Raspberry Pi 5 delivers 12x better performance at a similar price point. There is no workload where the Mars outperforms modern ARM alternatives.

Production deployments. The combination of slow CPU performance, older kernel, and limited software ecosystem makes the Mars unsuitable for production use cases. Even RISC-V-specific production deployments would benefit from waiting for next-generation hardware.

AI and machine learning. The Mars lacks any AI acceleration hardware, and its CPU performance makes even lightweight inference workloads impractical. Our experience running TinyLlama on the Orange Pi RV2 at 0.44 tokens per second suggests the Mars would be even slower given its lower core count and comparable per-core performance.

The Bigger Picture: RISC-V's Two-Board Story

Having now benchmarked two RISC-V single board computers, a pattern emerges. Both the Milk-V Mars (SiFive U74, 4 cores, 936s) and the Orange Pi RV2 (Ky X1, 8 cores, 651s) occupy a performance tier roughly 10-15x slower than mainstream ARM platforms. This isn't a coincidence - it reflects the current state of RISC-V application processor development.

The good news is that this gap isn't architectural. Nothing about the RISC-V instruction set prevents high-performance implementations. SiFive's P-series cores, Ventana's Veyron, and Alibaba's Xuantie C920 all promise ARM-competitive performance. The gap exists because the affordable SBC market is currently served by first-generation cores on older process nodes with immature compiler support.

The bad news is that closing this gap requires simultaneous progress on multiple fronts: newer core designs with out-of-order execution, migration to modern process nodes (7nm and below), compiler optimizations specific to new microarchitectures, and operating system tuning for RISC-V hardware features. This is years of work, and it's unclear whether the SBC market generates enough volume to justify the investment.

For now, RISC-V single board computers remain in the "interesting to explore, impractical to deploy" category. The Milk-V Mars embodies this perfectly: a board that works, runs real software, and delivers real results - just not quickly enough to compete with the ARM and x86 boards sitting next to it on the shelf.

Conclusion

The Milk-V Mars is a functional, stable, and affordable RISC-V single board computer that delivers on its basic promise: a Linux-capable platform built on SiFive's pioneering U74 cores. Its 8GB of RAM, Gigabit Ethernet, and standard Debian environment make it a reasonable development platform for RISC-V exploration.

Its performance, however, tells the honest story of where RISC-V stands in early 2026. At 936 seconds average for our Rust compilation benchmark, the Mars is the slowest board we've tested - 15x slower than the Orange Pi 5 Max, 12.2x slower than the Raspberry Pi 5, and 1.4x slower than the Orange Pi RV2. These numbers reflect the U74's in-order pipeline, 1.5 GHz clock speed, and 28nm process node, compounded by RISC-V's still-maturing compiler toolchain.

The remarkable consistency across benchmark runs (0.3% standard deviation) shows that the hardware is well-behaved and thermally stable - it's simply not fast. The board does exactly what it's supposed to do; it just does it slowly by modern standards.

For RISC-V enthusiasts, the Milk-V Mars offers an affordable window into the architecture that may eventually reshape computing. For everyone else, the Raspberry Pi 5 remains the obvious choice for general-purpose single board computing. The Mars is a board for the curious and the patient - those willing to trade performance today for a front-row seat to an architectural revolution that's still finding its footing.

The question isn't whether RISC-V will eventually match ARM and x86 performance in affordable SBCs. It will. The question is when, and whether boards like the Milk-V Mars will be remembered as charming relics of the early days or as stepping stones that helped build the ecosystem that made RISC-V competitive. Either way, they deserve a place in the story.

Specifications Summary

Processor:

StarFive JH7110 (28nm process)
4x SiFive U74-MC @ 1.5 GHz (RV64GC: rv64imafdc)
1x SiFive S7 monitor core
MMU: Sv39

Memory & Storage:

8GB LPDDR4 RAM (2GB and 4GB options also available)
microSD card slot
eMMC module support

Video:

HDMI output
Hardware video decoding support

Connectivity:

Gigabit Ethernet
USB 3.0 and USB 2.0 ports
40-pin GPIO header (Raspberry Pi compatible)
I2C, SPI, UART

Physical:

Compact SBC form factor
Passive cooling adequate for sustained workloads

Benchmark Performance:

Rust compilation: 936.18 seconds average
15.0x slower than Orange Pi 5 Max (ARM64, RK3588)
12.2x slower than Raspberry Pi 5 (ARM64, BCM2712)
5.6x slower than Banana Pi CM5-Pro (ARM64, RK3576)
2.5x slower than Horizon X3 CM (ARM64, Sunrise X3)
1.4x slower than Orange Pi RV2 (RISC-V, Ky X1)
Standard deviation: 2.85 seconds (0.30% - excellent consistency)

Software:

Debian GNU/Linux Bookworm
Linux kernel 5.15.0 (StarFive patches)
Rust 1.93.1 (official rustup support for riscv64gc)

Recommendation: Suitable for RISC-V exploration and development; not competitive with ARM or x86 alternatives for performance-sensitive workloads.

The Orange Pi RV2: RISC-V Comes to the Single Board Computer Arena

A.C. Jokela — Tue, 21 Oct 2025 22:55:14 GMT

The Orange Pi RV2: Cost-effective 8-core RISC-V development board

When the Orange Pi RV2 arrived for testing, it represented something fundamentally different from the dozens of ARM and x86 single board computers that have crossed my desk over the years. This wasn't just another Cortex-A76 board with slightly tweaked specifications or a new Intel Atom variant promising better performance-per-watt. The Orange Pi RV2, powered by the Ky(R) X1 processor, represents one of the first commercially available RISC-V single board computers aimed at the hobbyist and developer market. It's a glimpse into a future where processor architecture diversity might finally break the ARM-x86 duopoly that has dominated single board computing as of late.

But is RISC-V ready for prime time? Can it compete with the mature ARM ecosystem that powers everything from smartphones to supercomputers, or the x86 architecture that has dominated desktop and server computing for over four decades? I put the Orange Pi RV2 through the same rigorous benchmarking suite I use for all single board computers, comparing it directly against established platforms including the Raspberry Pi 5, Raspberry Pi Compute Module 5, Orange Pi 5 Max, and LattePanda IOTA. The results tell a fascinating story about where RISC-V stands today and where it might be heading.

What is RISC-V and Why Does it Matter?

Before diving into performance numbers, it's worth understanding what makes RISC-V different. Unlike ARM or x86, RISC-V is an open instruction set architecture. This means anyone can implement RISC-V processors without paying licensing fees or negotiating complex agreements with chip vendors. The specification is maintained by RISC-V International, a non-profit organization, and the core ISA is frozen and will never change.

This openness has led to an explosion of academic research and commercial implementations. Companies like SiFive, Alibaba, and now apparently Ky have developed RISC-V cores targeting everything from embedded microcontrollers to high-performance application processors. The promise is compelling: a truly open architecture that could democratize processor design and break vendor lock-in.

However, openness alone doesn't guarantee performance or ecosystem maturity. The RISC-V software ecosystem is still catching up to ARM and x86, with toolchains, operating systems, and applications at various stages of optimization. The Orange Pi RV2 gives us a real-world test of where this ecosystem stands in late 2024 and early 2025.

The Orange Pi RV2: Specifications and Setup

Top view showing the Ky X1 RISC-V processor and 8GB RAM

The Orange Pi RV2 features the Ky(R) X1 processor, an 8-core RISC-V chip running at up to 1.6 GHz. The system ships with Orange Pi's custom Linux distribution based on Ubuntu Noble, running kernel 6.6.63-ky. The board includes 8GB of RAM, sufficient for most development tasks and light server workloads.

Side view showing USB 3.0 ports, Gigabit Ethernet, and HDMI connectivity

Setting up the Orange Pi RV2 proved straightforward. The board boots from SD card and includes SSH access out of the box. Installing Rust, the language I use for compilation benchmarks, required building from source rather than using rustup, as RISC-V support in rustup is still evolving. Once installed, I had rustc 1.90.0 and cargo 1.90.0 running successfully.

The system presents itself as:

Linux orangepirv2 6.6.63-ky #1.0.0 SMP PREEMPT Wed Mar 12 09:04:00 CST 2025 riscv64 riscv64 riscv64 GNU/Linux

One immediate observation: this kernel was compiled in March 2025, suggesting very recent development. This is typical of the RISC-V SBC space right now - these boards are so new that kernel and userspace support is being actively developed, sometimes just weeks or months before the hardware ships.

Bottom view showing eMMC connector and M.2 key expansion

The Competition: ARM64 and x86_64 Platforms

To properly evaluate the Orange Pi RV2, I compared it against four other single board computers representing the current state of ARM and x86 in this form factor.

The Raspberry Pi 5 and Raspberry Pi Compute Module 5 both feature the Broadcom BCM2712 with four Cortex-A76 cores running at 2.4 GHz. These represent the current flagship for the Raspberry Pi Foundation, widely regarded as the gold standard for hobbyist and education-focused SBCs. The standard Pi 5 averaged 76.65 seconds in compilation benchmarks, while the CM5 came in slightly faster, demonstrating the maturity of ARM's Cortex-A76 architecture.

The Orange Pi 5 Max takes a different approach with its Rockchip RK3588 SoC, featuring a big.LITTLE configuration with four Cortex-A76 cores and four Cortex-A55 efficiency cores, totaling eight cores. This heterogeneous architecture allows the system to balance performance and power consumption. In my testing, the Orange Pi 5 Max posted the fastest compilation times among the ARM platforms, leveraging all eight cores effectively.

On the x86 side, the LattePanda IOTA features Intel's N150 processor, a quad-core Alder Lake-N chip. This represents Intel's current low-power x86 offering, designed to compete directly with ARM in the SBC and mini-PC market. The N150 delivered solid performance with an average compilation time of 72.21 seconds, demonstrating that x86 can still compete in this space when properly optimized.

Compilation Performance: The Rust Test

Comprehensive compilation performance comparison across all platforms

My primary benchmark involves compiling a Rust project - specifically, a ballistics engine with significant computational complexity and numerous dependencies. This real-world workload stresses the CPU, memory subsystem, and compiler toolchain in ways that synthetic benchmarks often miss. I perform three clean compilation runs on each system and average the results.

The results were striking:

Orange Pi 5 Max (ARM64, RK3588, 8 cores): 62.35 seconds average
LattePanda IOTA (x86_64, Intel N150, 4 cores): 72.21 seconds average
Raspberry Pi 5 (ARM64, BCM2712, 4 cores): 76.65 seconds average
Raspberry Pi CM5 (ARM64, BCM2712, 4 cores): ~74 seconds average
Orange Pi RV2 (RISC-V, Ky X1, 8 cores): 650.60 seconds average

The Orange Pi RV2's compilation times of 661.25, 647.39, and 643.16 seconds averaged out to 650.60 seconds - more than ten times slower than the Orange Pi 5 Max and nearly nine times slower than the Raspberry Pi 5. Despite having eight cores compared to the Pi 5's four, the RISC-V platform lagged dramatically behind.

This performance gap isn't simply about clock speeds or core counts. The Orange Pi RV2 runs at 1.6 GHz compared to the Pi 5's 2.4 GHz, but that 1.5x difference in frequency doesn't explain a 10x difference in compilation time. Instead, we're seeing the combined effect of several factors:

Processor microarchitecture maturity - ARM's Cortex-A76 represents over a decade of iterative improvement, while the Ky X1 is a first-generation design
Compiler optimization - LLVM's ARM backend has been optimized for years, while RISC-V support is much newer
Memory subsystem performance - the Ky X1's memory controller and cache hierarchy appear significantly less optimized
Single-threaded performance - compilation is often limited by single-threaded tasks, where the ARM cores have a significant advantage

It's worth noting that the Orange Pi RV2 showed good consistency across runs, with only about 2.8 percent variation between the fastest and slowest compilation. This suggests the hardware itself is stable; it's simply not competitive with current ARM or x86 offerings for this workload.

The Ecosystem Challenge: Toolchains and Software

Beyond raw performance, the RISC-V ecosystem faces significant maturity challenges. This became evident when attempting to run llama.cpp, the popular framework for running large language models locally. Following Jeff Geerling's guide for building llama.cpp on RISC-V, I immediately hit toolchain issues.

The llama.cpp build system detected RISC-V vector extensions and attempted to compile with -march=rv64gc_zfh_v_zvfh_zicbop, enabling hardware support for floating-point operations and vector processing. However, the GCC 13.3.0 compiler shipping with Orange Pi's Linux distribution didn't fully support these extensions, producing errors about unexpected ISA strings.

The workaround was to disable RISC-V vector support entirely:

cmake -B build -DLLAMA_CURL=OFF -DGGML_RVV=OFF -DGGML_NATIVE=OFF

By compiling with basic rv64gc instructions only - essentially the baseline RISC-V instruction set without advanced SIMD capabilities - the build succeeded. But this immediately highlights a key ecosystem problem: the mismatch between hardware capabilities, compiler support, and software assumptions.

On ARM or x86 platforms, these issues were solved years ago. When you compile llama.cpp on a Raspberry Pi 5, it automatically detects and uses NEON SIMD instructions. On x86, it leverages AVX2 or AVX-512 if available. The toolchain, runtime detection, and fallback mechanisms all work seamlessly because they've been tested and refined over countless deployments.

RISC-V is still working through these growing pains. The vector extensions exist in the specification and are implemented in hardware on some processors, but compiler support varies, software doesn't always detect capabilities correctly, and fallback paths aren't always reliable. This forced me to compile llama.cpp in its least optimized mode, guaranteeing compatibility but leaving significant performance on the table.

Running LLMs on RISC-V: TinyLlama Performance

Despite the toolchain challenges, I successfully built llama.cpp and downloaded TinyLlama 1.1B in Q4_K_M quantization - a relatively small language model suitable for testing on resource-constrained devices. Running inference revealed exactly what you'd expect given the compilation benchmarks: functional but slow performance.

Prompt processing achieved 0.87 tokens per second, taking 1,148 milliseconds per token to encode the input. Token generation during the actual response was even slower at 0.44 tokens per second, or 2,250 milliseconds per token. To generate a 49-token response to "What is RISC-V?" took 110 seconds total.

For context, the same TinyLlama model on a Raspberry Pi 5 typically achieves 5-8 tokens per second, while the LattePanda IOTA manages 8-12 tokens per second depending on quantization. High-end ARM boards like the Orange Pi 5 Max can exceed 15 tokens per second with this model. The Orange Pi RV2's 0.44 tokens per second puts it roughly 11-34x slower than comparable ARM and x86 platforms.

The LLM did produce correct output, successfully explaining RISC-V as "a software-defined architecture for embedded and real-time systems" before noting it was "open-source and community-driven." The accuracy of the output confirms that the RISC-V platform is functionally correct - it's running the same model with the same weights and producing equivalent results. But the performance makes interactive use impractical for anything beyond basic testing and development.

What makes this particularly interesting is that we disabled vector instructions entirely. On ARM and x86 platforms, SIMD instructions provide massive speedups for the matrix multiplications that dominate LLM inference. The Orange Pi RV2 theoretically has vector extensions that could provide similar acceleration, but the immature toolchain forced us to leave them disabled. When RISC-V compiler support matures and llama.cpp can reliably use these hardware capabilities, we might see 2-4x performance improvements - though that would still leave RISC-V trailing ARM significantly.

The State of RISC-V SBCs: Pioneering Territory

It's important to contextualize these results within the broader RISC-V SBC landscape. These boards are extraordinarily new to the market. While ARM-based SBCs have evolved over 12+ years since the original Raspberry Pi, and x86 SBCs have existed even longer, RISC-V platforms aimed at developers and hobbyists have only emerged in the past two years.

The Orange Pi RV2 is essentially a first-generation product in a first-generation market. For comparison, the original Raspberry Pi from 2012 featured a single-core ARM11 processor running at 700 MHz and struggled with basic desktop tasks. Nobody expected it to compete with contemporary x86 systems; it was revolutionary simply for existing at a $35 price point and running Linux.

RISC-V is in a similar position today. The existence of an eight-core RISC-V SBC that can boot Ubuntu, compile complex software, and run large language models is itself remarkable. Five years ago, RISC-V was primarily found in microcontrollers and academic research chips. The progress to application-class processors running general-purpose operating systems has been rapid.

The ecosystem is growing faster than most observers expected. Major distributions like Debian, Fedora, and Ubuntu now provide official RISC-V images. The Rust programming language has first-class RISC-V support in its compiler. Projects like llama.cpp, even with their current limitations, are actively working on RISC-V optimization. Hardware vendors beyond SiFive and Chinese manufacturers are beginning to show interest, with Qualcomm and others investigating RISC-V for specific use cases.

What we're seeing with the Orange Pi RV2 isn't a mature product competing with established platforms - it's a pioneer platform demonstrating what's possible and revealing where work remains. The 10x performance gap versus ARM isn't a fundamental limitation of the RISC-V architecture; it's a measure of how much optimization work ARM has received over the past decade that RISC-V hasn't yet enjoyed.

Where RISC-V Goes From Here

The question isn't whether RISC-V will improve, but how quickly and how much. Several factors suggest significant progress in the near term:

Compiler maturity will improve rapidly as RISC-V gains adoption. LLVM and GCC developers are actively optimizing RISC-V backends, and major software projects are adding RISC-V-specific optimizations. The vector extension issues I encountered will be resolved as compilers catch up with hardware capabilities.

Processor implementations will evolve quickly. The Ky X1 in the Orange Pi RV2 is an early design, but Chinese semiconductor companies are investing heavily in RISC-V, and Western companies are beginning to follow. Second and third-generation designs will benefit from lessons learned in these first products.

Software ecosystem development is accelerating. Critical applications are being ported and optimized for RISC-V, from machine learning frameworks to databases to web servers. As this software matures, RISC-V systems will become more practical for real workloads.

The standardization of extensions will help. RISC-V's modular approach allows vendors to pick and choose which extensions to implement, but this creates fragmentation. As the ecosystem consolidates around standard profiles - baseline feature sets that software can depend on - compatibility and optimization will improve.

However, RISC-V faces challenges that ARM and x86 don't. The lack of a dominant vendor means fragmentation is always a risk. The openness that makes RISC-V attractive also means there's no single company with ARM or Intel's resources pushing the architecture forward. Progress depends on collective ecosystem development rather than centralized decision-making.

For hobbyists and developers today, RISC-V boards like the Orange Pi RV2 serve a specific purpose: experimentation, learning, and contributing to ecosystem development. If you want the fastest compilation times, most compatible software, or best performance per dollar, ARM or x86 remain superior choices. But if you want to be part of an emerging architecture, contribute to open-source development, or simply understand an alternative approach to processor design, RISC-V offers unique opportunities.

Conclusion: A Promising Start

The Orange Pi RV2 demonstrates both the promise and the current limitations of RISC-V in the single board computer space. It's a functional, stable platform that successfully runs complex workloads - just not quickly compared to established alternatives. The 650-second compilation times and 0.44 tokens-per-second LLM inference are roughly 10x slower than comparable ARM platforms, but they work correctly and consistently.

This performance gap isn't surprising or condemning. It reflects where RISC-V is in its maturity curve: early, promising, but not yet optimized. The architecture itself has no fundamental limitations preventing it from reaching ARM or x86 performance levels. What's missing is time, optimization work, and ecosystem development.

For anyone considering the Orange Pi RV2 or similar RISC-V boards, set expectations appropriately. This isn't a Raspberry Pi 5 competitor in raw performance. It's a development platform for exploring a new architecture, contributing to open-source projects, and learning about processor design. If those goals align with your interests, the Orange Pi RV2 is a fascinating platform. If you need maximum performance for compilation, machine learning, or general computing, stick with ARM or x86 for now.

But watch this space. RISC-V is moving faster than most expected, and platforms like the Orange Pi RV2 are pushing the boundaries of what's possible with open processor architectures. The 10x performance gap today might be 3x in two years and negligible in five. We're witnessing the early days of a potential revolution in processor architecture, and being able to participate in that development is worth more than a few minutes of faster compile times.

The future of computing might not be exclusively ARM or x86. If RISC-V continues its current trajectory, we could see a genuinely competitive third architecture in the mainstream within this decade. The Orange Pi RV2 is an early step on that journey - imperfect, slow by current standards, but undeniably significant.