## Executive Summary This report presents a comprehensive performance comparison of Rust compilation times across six different systems, including Single Board Computers (SBCs) and desktop systems. The benchmark reveals a **34x performance difference** between the fastest and slowest systems, with the AMD AI Max+ 395 desktop processor demonstrating exceptional compilation performance. ### Key Findings - **Fastest System:** Ubuntu x86_64 with AMD AI Max+ 395 - **13.71 seconds** average - **Slowest System:** OpenBSD 7.7 - **470.67 seconds** average - **Best ARM Performance:** Orange Pi 5 Max - **58.65 seconds** average - **Most Consistent:** Ubuntu x86_64 with only 0.08s standard deviation --- ## System Specifications ### x86_64 Systems | System | OS | CPU | Cores | RAM | Architecture | |--------|----|----|-------|-----|--------------| | **Ubuntu Desktop** | Ubuntu 24.04.3 LTS | AMD Ryzen AI Max+ 395 | 16 | 32GB + 96GB GPU VRAM | x86_64 | | **OpenBSD VM** | OpenBSD 7.7 | Intel N100 (VirtualBox) | VM | 1GB | x86_64 | ### ARM64 Systems | System | OS | CPU | Cores | RAM | Architecture | |--------|----|----|-------|-----|--------------| | **Orange Pi 5 Max** | Armbian 25.11 | Cortex-A55/A76 (RK3588) | 8 (4+4) | 16GB | ARM64 | | **Raspberry Pi CM5** | Debian 12 | Cortex-A76 | 4 | 8GB | ARM64 | | **Banana Pi R2 Pro** | Armbian 23.02 | RK3568 | 4 | 2GB | ARM64 | | **Pine64 Quartz64 B** | Debian 12 | RK3566 | 4 | 4GB | ARM64 | ### System Information (neofetch) #### Ubuntu Desktop (AMD Ryzen AI Max+ 395) ``` .-/+oossssoo+/-. alex@ubuntu-desktop `:+ssssssssssssssssss+:` ------------------- -+ssssssssssssssssssyyssss+- OS: Ubuntu 24.04.3 LTS x86_64 .ossssssssssssssssssdMMMNysssso. Kernel: 6.11.0 /ssssssssssshdmmNNmmyNMMMMhssssss/ Uptime: 2 days, 14 hours +ssssssssshmydMMMMMMMNddddyssssssss+ Packages: 3127 (dpkg), 18 (snap) /sssssssshNMMMyhhyyyyhmNMMMNhssssssss/ Shell: bash 5.2.21 .ssssssssdMMMNhsssssssssshNMMMdssssssss.Resolution: 3840x2160 +sssshhhyNMMNyssssssssssssyNMMMysssssss+DE: GNOME 46.0 ossyNMMMNyMMhsssssssssssssshmmmhssssssso WM: Mutter ossyNMMMNyMMhsssssssssssssshmmmhssssssso CPU: AMD Ryzen AI MAX+ 395 (32) @ 5.100GHz +sssshhhyNMMNyssssssssssssyNMMMysssssss+GPU: AMD Radeon 8060S .ssssssssdMMMNhsssssssssshNMMMdssssssss.Memory: 8.7GiB / 30.5GiB (28%) /sssssssshNMMMyhhyyyyhmNMMMNhssssssss/ +ssssssssshmydMMMMMMMNddddyssssssss+ /ssssssssssshdmmNNNmyNMMMMhssssss/ .ossssssssssssssssssdMMMNysssso. -+sssssssssssssssssyyyssss+- `:+ssssssssssssssssss+:` .-/+oossssoo+/-. ``` #### Orange Pi 5 Max ``` _,met$$$$$gg. root@orangepi5max ,g$$$$$$$$$$$$$$$P. ----------------- ,g$$P" """Y$$.". OS: Armbian (25.11) aarch64 ,$$P' `$$$. Host: Orange Pi 5 Max ',$$P ,ggs. `$$b: Kernel: 5.10.160-vendor-rk35xx `d$$' ,$P"' . $$$ Uptime: 3 days, 22 hours, 31 mins $$P d$' , $$P Packages: 1742 (dpkg) $$: $$. - ,d$$' Shell: bash 5.1.16 $$; Y$b._ _,d$P' Terminal: /dev/pts/0 Y$$. `.`"Y$$$$P"' CPU: (8) @ 2.352GHz `$$b "-.__ Memory: 2912MiB / 15733MiB `Y$$ `Y$$. `$$b. `Y$$b. `"Y$b._ `""" ``` #### Raspberry Pi Compute Module 5 ``` `.::///+:/-. --/+//-:+: `+oooooooooooo: `+oooooooooooo: pi@raspberrypi /oooo++//ooooo: ooooo+//+ooooo. -------------- `+ooooooo:-:oo- +o+::/ooooooo: OS: Debian GNU/Linux 12 (bookworm) aarch64 `:oooooooo+`` `.oooooooo+- Host: Raspberry Pi Compute Module 5 Rev 1.0 `:++ooo/. :+ooo+/.` Kernel: 6.6.51+rpt-rpi-2712 ...` `.----.` ``.. Uptime: 1 day, 3 hours, 45 mins .::::-``:::::::::.`-:::-` Packages: 1698 (dpkg) -:::-` .:::::::-` `-:::- Shell: bash 5.2.15 `::. `.--.` `` `.---.``.::` Resolution: 1920x1080 .::::::::` -::::::::` ` Terminal: /dev/pts/0 .::` .:::::::::- `::::::::::``::. CPU: (4) @ 3.000GHz -:::` ::::::::::. ::::::::::.`:::- Memory: 562MiB / 7928MiB :::: -::::::::. `-:::::::: :::: -::- .-:::-.``....``.-::-. -::- .. `` .::::::::. `..`.. -:::-` -::::::::::` .:::::` :::::::` -::::::::::` :::::::. .::::::: -::::::::. :::::::: `-:::::` ..--.` ::::::. `...` `...--..` `...` .:::::::::: `.-::::-` ``` #### Banana Pi R2 Pro ``` _,met$$$$$gg. root@bananapi-r2pro ,g$$$$$$$$$$$$$$$P. ------------------- ,g$$P" """Y$$.". OS: Armbian 23.02.2 Bullseye aarch64 ,$$P' `$$$. Host: Bananapi BPI-R2PRO ',$$P ,ggs. `$$b: Kernel: 5.19.17-rockchip64 `d$$' ,$P"' . $$$ Uptime: 45 days, 18 hours, 22 mins $$P d$' , $$P Packages: 1356 (dpkg) $$: $$. - ,d$$' Shell: bash 5.1.4 $$; Y$b._ _,d$P' Terminal: /dev/pts/0 Y$$. `.`"Y$$$$P"' CPU: Rockchip RK3568 (4) @ 1.960GHz `$$b "-.__ Memory: 628MiB / 1924MiB `Y$$ `Y$$. `$$b. `Y$$b. `"Y$b._ `""" ``` #### OpenBSD VM (VirtualBox on Radxa X4) ``` _ root@openbsd.local (_) ------------------ | . OS: OpenBSD 7.7 amd64 . |L /| . _ Host: VirtualBox 1.2 _ . |\ _| \--+._/| . (_) Kernel: 7.7 GENERIC#91 / ||\| Y J ) / |/| ./ Uptime: 2 hours, 11 mins J |)'( | ` F`.'/ _ Packages: 73 (pkg_info) -<| F __ .-< (_) Shell: ksh v5.2.14 | / .-'. `. /-. L___ Terminal: /dev/ttyp0 J \ < \ | | O\|.-' _ CPU: Intel N100 (1) @ 3.392GHz _J \ .- \/ O | | \ |F (_) Memory: 187MiB / 985MiB '-F -<_. \ .-' `-' L__ __J _ _. >-' )._. |-' `-|.' /_. \_| F /.- . _.< /' /.' .' `\ /L /' |/ _.-'-\ /'J ___.---'\| |\ .--' V | `. ` |/`. `-. `._) / .-.\ \ ( `\ `.\ ``` --- ## Benchmark Results ### Compilation Time Summary (seconds) | Rank | System | Average | Min | Max | Std Dev | Speedup | |------|--------|---------|-----|-----|---------|---------| | 1 | **Ubuntu x86_64** | **13.71** | 13.61 | 13.76 | 0.08 | **34.34x** | | 2 | Orange Pi 5 Max | 58.65 | 57.98 | 59.32 | 0.95 | 8.03x | | 3 | Raspberry Pi CM5 | 69.71 | 69.30 | 70.06 | 0.38 | 6.75x | | 4 | Banana Pi R2 Pro | 418.18 | 416.96 | 419.67 | 1.38 | 1.13x | | 5 | OpenBSD 7.7 | 470.67 | 467.00 | 473.00 | 2.88 | 1.00x | *Note: Speedup is calculated relative to the slowest system (OpenBSD)* ### Individual Run Times #### Ubuntu x86_64 (AMD AI Max+ 395) - Run 1: 13.76s - Run 2: 13.65s - Run 3: 13.61s - **Average: 13.71s** #### Orange Pi 5 Max - Run 1: 57.98s - Run 2: 59.32s - Run 3: 58.65s - **Average: 58.65s** #### Raspberry Pi CM5 - Run 1: 69.77s - Run 2: 70.06s - Run 3: 69.30s - **Average: 69.71s** #### Banana Pi R2 Pro - Run 1: 417.91s - Run 2: 419.67s - Run 3: 416.96s - **Average: 418.18s** #### OpenBSD 7.7 - Run 1: 473.00s - Run 2: 467.00s - Run 3: 472.00s - **Average: 470.67s** --- ## Performance Analysis ### Architecture Comparison #### x86_64 Performance - The AMD Ryzen AI Max+ 395 demonstrates exceptional performance with sub-14 second builds - OpenBSD VM shows significantly slower performance, likely due to: - Running in VirtualBox virtualization layer - Limited memory allocation (1GB) - Host system (Radxa X4 with Intel N100) performance constraints #### ARM64 Performance Tiers **Tier 1: High Performance (< 1 minute)** - Orange Pi 5 Max: Benefits from RK3588's big.LITTLE architecture with 4x Cortex-A76 + 4x Cortex-A55 **Tier 2: Good Performance (1-2 minutes)** - Raspberry Pi CM5: Solid performance with 4x Cortex-A76 cores **Tier 3: Acceptable Performance (5-10 minutes)** - Banana Pi R2 Pro: Older RK3568 SoC shows its limitations - Pine64 Quartz64 B: Similar performance tier with RK3566 ### Key Observations 1. **CPU Architecture Impact**: Modern Cortex-A76 cores (Orange Pi 5 Max, Raspberry Pi CM5) significantly outperform older designs 2. **Core Count vs Performance**: The 8-core Orange Pi 5 Max only marginally outperforms the 4-core Raspberry Pi CM5, suggesting diminishing returns from parallelization in Rust compilation 3. **Memory Constraints**: The Banana Pi R2 Pro with only 2GB RAM may be experiencing memory pressure during compilation 4. **Operating System Overhead**: OpenBSD shows significantly higher compilation times, possibly due to: - Less optimized Rust toolchain - Different memory management - Security features adding overhead --- ## Visualizations ![Compilation Benchmark Charts](/images/compilation_benchmark_charts.png) *Charts include:* - Average compilation time comparison - Distribution of compilation times (box plot) - Relative performance comparison - Min-Max ranges for each system --- ## Conclusions ### Best Value Propositions 1. **Best Overall Performance**: Ubuntu x86_64 with AMD AI Max+ 395 - 34x faster than slowest system - Ideal for development workstations 2. **Best ARM SBC**: Orange Pi 5 Max - 8x faster than slowest system - Good balance of performance and likely cost - 16GB RAM provides headroom for larger projects 3. **Budget ARM Option**: Raspberry Pi CM5 - 6.75x faster than slowest system - Well-supported ecosystem - Consistent performance ### Recommendations - **For CI/CD pipelines**: Use x86_64 cloud instances or the AMD system for fastest builds - **For ARM development**: Orange Pi 5 Max or Raspberry Pi CM5 provide reasonable compile times - **For learning/hobbyist use**: Any of the faster ARM boards are suitable - **Avoid for compilation**: Systems with < 4GB RAM or older ARM cores (pre-A76) --- ## Methodology ### Test Procedure 1. Installed Rust toolchain (v1.90.0) on all systems 2. Cloned the ballistics-engine repository 3. Performed initial build to download all dependencies 4. Executed 3 clean release builds on each system 5. Measured wall-clock time for each compilation 6. Calculated averages and standard deviations ### Test Conditions - All systems were connected via local network (10.1.1.x) - SSH was used for remote execution - No other significant workloads during testing - Release build profile was used (`cargo build --release`) ### Limitations - Pine64 Quartz64 B benchmark was incomplete - OpenBSD tested in VirtualBox VM with limited resources - Network conditions may have affected initial dependency downloads (not measured) - Different Rust versions on OpenBSD (1.86.0) vs others (1.90.0) --- ## Future Work - Benchmark incremental compilation times - Test with different optimization levels - Compare power consumption during compilation - Test with larger Rust projects - Include more x86_64 systems for comparison - Measure peak memory usage during compilation