TinyComputers.io (Posts about gpu computing)

Upgrading ROCm 7.0 to 7.2 on AMD Strix Halo (gfx1151)

A.C. Jokela — Wed, 18 Feb 2026 16:00:00 GMT

Introduction

If you're running AMD's Strix Halo hardware -- specifically the Ryzen AI MAX+ 395 with its integrated Radeon 8060S GPU -- you already know the software ecosystem is a moving target. The gfx1151 architecture sits in an awkward spot: powerful hardware that isn't officially listed on AMD's ROCm support matrix, yet functional enough to run real workloads with the right driver stack. When ROCm 7.2 landed in early 2026, upgrading from 7.0.2 was a priority. The newer stack brings an updated HSA runtime, a refreshed amdgpu kernel module, and broader compatibility improvements that matter on bleeding-edge silicon.

This post documents the complete upgrade procedure from ROCm 7.0.2 to 7.2 on a production Ubuntu 24.04 system. It's not a theoretical exercise -- this was performed on a live server running QEMU virtual machines and network services, with the expectation that everything would come back online after a single reboot.

AMD's official documentation states that in-place ROCm upgrades are not supported. The recommended path is a full uninstall followed by a clean reinstall. That's exactly what we did, and the entire process took about 20 minutes of wall-clock time (excluding the reboot).

System Overview

The target system is a Bosgame mini PC running the Ryzen AI MAX+ 395 APU. If you've read the earlier review of this hardware, you'll be familiar with the specs. For context on this upgrade, here's what matters:

Hardware

CPU: AMD Ryzen AI MAX+ 395, 16 cores / 32 threads, Zen 5
GPU: Integrated Radeon 8060S, 40 Compute Units, RDNA 3.5 (gfx1151)
Memory: 32 GB DDR5, unified architecture with 96 GB allocatable to GPU
Peak GPU Clock: 2,900 MHz

Software (Pre-Upgrade)

OS: Ubuntu 24.04.3 LTS (Noble Numbat)
Kernel: 6.14.0-37-generic (HWE, pinned)
ROCm: 7.0.2
amdgpu-dkms: 6.14.14 (from repo.radeon.com/amdgpu/30.10.2)
ROCk Module: 6.14.14

Running Services

The system was actively serving several roles during the upgrade:

Five QEMU virtual machines (three x86, two aarch64)
A PXE boot server (dnsmasq) for the local network
Docker daemon with various containers

None of these services are tied to the GPU driver stack, so the plan was to perform the upgrade and reboot without shutting them down first. The VMs and network services would come back automatically after the reboot.

Why Upgrade

ROCm 7.0.2 worked on this hardware. Models loaded, inference ran, rocminfo detected the GPU. So why bother upgrading?

Three reasons:

Driver maturity for gfx1151: The amdgpu kernel module jumped from 6.14.14 to 6.16.13 between the two releases. That's not a minor revision -- it represents months of kernel driver development. On hardware that isn't officially supported, newer drivers tend to bring meaningful stability improvements as AMD's internal teams encounter and fix issues on adjacent architectures.
HSA Runtime improvements: ROCm 7.2 ships HSA Runtime Extension version 1.15, up from 1.11 in ROCm 7.0.2. The HSA (Heterogeneous System Architecture) runtime is the lowest layer of the ROCm software stack -- it handles device discovery, memory management, and kernel dispatch. Improvements here affect everything built on top of it.
Ecosystem alignment: PyTorch wheels, Ollama builds, and other ROCm-dependent tools increasingly target 7.2 as the baseline. Running 7.0.2 was becoming an exercise in version pinning and compatibility workarounds.

The Kernel Hold: Why It Matters

Before diving into the procedure, a note on kernel management. This system runs the Ubuntu HWE (Hardware Enablement) kernel, which provides newer kernel versions on LTS releases. At the time of this upgrade, the HWE kernel was 6.14.0-37-generic. The upstream kernel had already moved to 6.17, but we didn't want the ROCm upgrade to pull in a kernel that AMD's DKMS module might not build against.

The solution is apt-mark hold:

sudo apt-mark hold linux-generic-hwe-24.04 linux-headers-generic-hwe-24.04 linux-image-generic-hwe-24.04

This prevents apt from upgrading the kernel meta-packages, effectively pinning the system to 6.14.0-37-generic. The hold was already in place before the upgrade and remained untouched throughout. After the upgrade, we confirmed it was still active:

apt-mark showhold

linux-generic-hwe-24.04
linux-headers-generic-hwe-24.04
linux-image-generic-hwe-24.04

If you're running Strix Halo or any other hardware where kernel compatibility with amdgpu-dkms is uncertain, kernel holds are essential. A kernel upgrade that breaks the DKMS build means no GPU driver after reboot.

Upgrade Procedure

Step 1: Uninstall the Current ROCm Stack

AMD provides the amdgpu-uninstall script for exactly this purpose. It removes all ROCm userspace packages and the amdgpu-dkms kernel module in a single operation:

sudo amdgpu-uninstall -y

This command removed approximately 120 packages, including the full HIP runtime, rocBLAS, MIOpen, MIGraphX, ROCm SMI, the LLVM-based compiler toolchain, and the Mesa graphics drivers that ship with ROCm. The DKMS module was purged, which means the amdgpu kernel module was removed from the 6.14.0-37-generic kernel's module tree.

After the ROCm stack was removed, we purged the amdgpu-install meta-package itself:

sudo apt purge -y amdgpu-install

This also cleaned up the APT repository entries that amdgpu-install had configured in /etc/apt/sources.list.d/. The old repos -- repo.radeon.com/amdgpu/30.10.2, repo.radeon.com/rocm/apt/7.0.2, and repo.radeon.com/graphics/7.0.2 -- were all removed automatically.

Step 2: Clean Up Leftover Files

The package removal was thorough but not perfect. A few leftover directories remained in /opt/:

ls /opt/ | grep rocm

rocm-7.0.0
rocm-7.0.2
rocm-7.9.0

The rocm-7.0.0 directory was from a previous installation attempt. The rocm-7.9.0 was from an earlier experiment with a release candidate build. The rocm-7.0.2 directory contained a single orphaned shared library (libamdhip64.so.6) that dpkg couldn't remove because the directory wasn't empty. All three were cleaned up manually:

sudo rm -rf /opt/rocm-7.0.0 /opt/rocm-7.0.2 /opt/rocm-7.9.0

It's worth checking for stale ROCm directories after any uninstall. They consume negligible disk space but can confuse build systems and scripts that scan /opt/rocm* for active installations.

Step 3: Install the ROCm 7.2 Installer

AMD distributes ROCm through a meta-package called amdgpu-install. Each ROCm release has its own version of this package, which configures the appropriate APT repositories. The 7.2 installer was downloaded directly from AMD's repository:

cd /tmp
wget https://repo.radeon.com/amdgpu-install/7.2/ubuntu/noble/amdgpu-install_7.2.70200-1_all.deb
sudo apt install -y ./amdgpu-install_7.2.70200-1_all.deb
sudo apt update

After installation and apt update, three new repositories were active:

https://repo.radeon.com/amdgpu/30.30/ubuntu noble -- the kernel driver and Mesa components
https://repo.radeon.com/rocm/apt/7.2 noble -- the ROCm userspace stack
https://repo.radeon.com/graphics/7.2/ubuntu noble -- graphics libraries

The version numbering can be confusing. The amdgpu-install package version is 30.30.0.0.30300000-2278356.24.04, which maps to the amdgpu driver release 30.30. The ROCm version is 7.2.0. These are different version tracks that AMD maintains in parallel.

Step 4: Install ROCm 7.2

With the repositories configured, the actual installation was a single command:

sudo amdgpu-install -y --usecase=graphics,rocm

The --usecase=graphics,rocm flag tells the installer to include both the Mesa graphics drivers and the full ROCm compute stack. This is the right choice for a system that needs both display output and GPU compute capabilities.

The installation took approximately 10 minutes and included:

amdgpu-dkms 6.16.13: The kernel module, compiled via DKMS against the running kernel
Full ROCm 7.2 stack: HIP runtime, hipcc compiler, rocBLAS, rocFFT, MIOpen, MIGraphX, RCCL, ROCm SMI, ROCProfiler, and dozens of other libraries
Mesa graphics: Updated EGL, OpenGL, and Vulkan drivers from the amdgpu Mesa fork
ROCm LLVM toolchain: The LLVM-based compiler infrastructure that HIP uses for kernel compilation

The DKMS build is the critical step. During installation, DKMS compiled the amdgpu module against the kernel headers for 6.14.0-37-generic. The output confirmed a successful build:

depmod...
update-initramfs: Generating /boot/initrd.img-6.14.0-37-generic

The initramfs was regenerated to include the new module, ensuring it would be loaded at boot.

Step 5: Verify DKMS

Before rebooting, we confirmed the DKMS status:

dkms status

amdgpu/6.16.13-2278356.24.04, 6.14.0-37-generic, x86_64: installed
virtualbox/7.0.16, 6.14.0-36-generic, x86_64: installed
virtualbox/7.0.16, 6.14.0-37-generic, x86_64: installed
virtualbox/7.0.16, 6.8.0-100-generic, x86_64: installed

The new amdgpu module (6.16.13) was built and installed for 6.14.0-37-generic. Note that it only built for the currently running kernel, unlike VirtualBox which had modules built for older kernels as well. This is expected -- DKMS builds against available kernel headers, and the old kernel headers for 6.14.0-36 and 6.8.0-100 were still present from earlier installations.

Step 6: Reboot

sudo reboot

The server came back online in approximately 50 seconds.

Post-Reboot Verification

rocminfo

The first check after reboot was rocminfo, which queries the HSA runtime for available agents:

rocminfo

ROCk module version 6.16.13 is loaded
=====================
HSA System Attributes
=====================
Runtime Version:         1.18
Runtime Ext Version:     1.15
...
==========
HSA Agents
==========
Agent 1: AMD RYZEN AI MAX+ 395 w/ Radeon 8060S (CPU)
Agent 2: gfx1151 (GPU)
  Marketing Name:          AMD Radeon Graphics
  Compute Unit:            40
  Max Clock Freq. (MHz):   2900
  Memory Properties:       APU
  ISA 1: amdgcn-amd-amdhsa--gfx1151
  ISA 2: amdgcn-amd-amdhsa--gfx11-generic

Key observations:

ROCk module 6.16.13: The new kernel module loaded successfully.
Runtime Ext Version 1.15: Upgraded from 1.11 in ROCm 7.0.2.
gfx1151 detected: The GPU was recognized with its correct ISA identifier.
gfx11-generic ISA: ROCm 7.2 also exposes a generic gfx11 ISA, which allows software compiled for the broader RDNA 3 family to run on this device without gfx1151-specific builds.
APU memory: The memory properties correctly identify this as an APU with unified memory.

ROCm SMI

rocm-smi

Device  Node  Temp    Power     SCLK  MCLK     Fan  Perf  VRAM%  GPU%
0       1     33.0C   9.087W    N/A   1000Mhz  0%   auto  0%     0%

The GPU was visible and reporting telemetry. The 0% VRAM reading is expected on an APU -- rocm-smi reports dedicated VRAM usage, but on a unified memory architecture, GPU memory allocations come from system RAM and aren't reflected in this counter.

ROCm Version

cat /opt/rocm/.info/version

7.2.0

DKMS

dkms status

Confirmed amdgpu/6.16.13 remained installed for 6.14.0-37-generic after reboot.

PyTorch Validation

With the driver stack verified, the next step was confirming that PyTorch could see and use the GPU. ROCm 7.2 ships with prebuilt PyTorch wheels on AMD's repository.

Installing PyTorch for ROCm 7.2

We set up a Python virtual environment and installed the ROCm-specific wheels:

python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip

The PyTorch wheel for ROCm 7.2 requires a matching ROCm-specific build of Triton. Both are available from AMD's manylinux repository. The order matters -- Triton must be installed first, since the PyTorch wheel declares it as a dependency with a specific version that doesn't exist on PyPI:

pip install https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/triton-3.5.1%2Brocm7.2.0.gita272dfa8-cp312-cp312-linux_x86_64.whl
pip install https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/torch-2.9.1%2Brocm7.2.0.lw.git7e1940d4-cp312-cp312-linux_x86_64.whl
pip install https://repo.radeon.com/rocm/manylinux/rocm-rel-7.2/torchvision-0.24.0%2Brocm7.2.0.gitb919bd0c-cp312-cp312-linux_x86_64.whl

These are the ROCm 7.2 builds for Python 3.12. AMD also provides wheels for Python 3.10, 3.11, and 3.13.

Smoke Test

import torch
print("PyTorch:", torch.__version__)
print("CUDA available:", torch.cuda.is_available())
print("Device:", torch.cuda.get_device_name(0))
print("VRAM:", round(torch.cuda.get_device_properties(0).total_memory / 1e9, 1), "GB")

PyTorch: 2.9.1+rocm7.2.0.git7e1940d4
CUDA available: True
Device: AMD Radeon Graphics
VRAM: 103.1 GB

PyTorch detected the GPU through ROCm's HIP-to-CUDA translation layer. The 103.1 GB figure represents the total addressable memory on this unified-memory APU, which includes both the 96 GB GPU allocation and additional system memory accessible through the HSA runtime.

Note the use of torch.cuda despite this being an AMD GPU. ROCm's HIP runtime presents itself through PyTorch's CUDA interface, so all CUDA API calls in PyTorch (device selection, memory management, kernel launches) work transparently with AMD hardware.

Before and After Summary

Component	ROCm 7.0.2	ROCm 7.2.0
ROCm Version	7.0.2	7.2.0
amdgpu-dkms	6.14.14	6.16.13
ROCk Module	6.14.14	6.16.13
HSA Runtime Ext	1.11	1.15
amdgpu Repo	30.10.2	30.30
PyTorch	N/A	2.9.1+rocm7.2.0
Triton	N/A	3.5.1+rocm7.2.0
Kernel	6.14.0-37-generic	6.14.0-37-generic
Kernel Holds	In place	In place

Notes on gfx1151 Support

It's worth being explicit about the support situation. As of February 2026, gfx1151 (Strix Halo) is not listed on AMD's official ROCm support matrix. The supported RDNA 3 targets are gfx1100 (Navi 31, RX 7900 XTX) and gfx1101 (Navi 32). Strix Halo's gfx1151 is an RDNA 3.5 derivative that shares much of the ISA with gfx1100 but has architectural differences in the memory subsystem and compute unit layout.

In practice, ROCm 7.2 works on gfx1151. The kernel driver loads, rocminfo detects the GPU, and PyTorch can allocate tensors and dispatch compute kernels. The gfx11-generic ISA target in ROCm 7.2 is particularly helpful -- it provides a compatibility path for software that hasn't been explicitly compiled for gfx1151.

However, "works" and "fully supported" are different things. There are known quirks:

rocm-smi VRAM reporting: Always shows 0% on the APU since it only tracks discrete VRAM
No official PyTorch gfx1151 builds: The ROCm PyTorch wheels target gfx1100. They run on gfx1151 through ISA compatibility, but performance may not be optimal
Large model loading latency: Moving large models to the GPU device can be slow on the unified memory architecture, as the HSA runtime handles page migration differently than discrete GPU DMA transfers

If you're considering this hardware for production AI workloads, treat ROCm support as "functional but experimental." It works well enough for development, testing, and moderate inference workloads. For production training or latency-sensitive deployment, stick with hardware on AMD's official support list.

Rollback Plan

If the upgrade fails -- the DKMS module doesn't build, the GPU isn't detected after reboot, or something else goes wrong -- the rollback path is straightforward:

Uninstall ROCm 7.2:

sudo amdgpu-uninstall -y
sudo apt purge -y amdgpu-install

Reinstall ROCm 7.0.2:

wget https://repo.radeon.com/amdgpu-install/30.10.2/ubuntu/noble/amdgpu-install_30.10.2.0.30100200-2226257.24.04_all.deb
sudo apt install -y ./amdgpu-install_30.10.2.0.30100200-2226257.24.04_all.deb
sudo apt update
sudo amdgpu-install -y --usecase=graphics,rocm
sudo reboot

The entire rollback takes about 15 minutes. Keep the old amdgpu-install deb URL handy -- it's not linked from AMD's current download pages once a newer version is published.

Conclusion

Upgrading ROCm on hardware that isn't officially supported always carries some risk, but this upgrade from 7.0.2 to 7.2 on gfx1151 was uneventful. The procedure follows AMD's documented uninstall-reinstall approach with no deviations. The kernel hold strategy kept the kernel stable, the DKMS module built cleanly against 6.14.0-37-generic, and all post-reboot checks passed.

The improvements in ROCm 7.2 -- particularly the HSA runtime bump to 1.15 and the introduction of the gfx11-generic ISA target -- represent meaningful progress for Strix Halo users. The ecosystem is slowly catching up to the hardware. It's not there yet, but each release closes the gap.

For anyone running a Ryzen AI MAX+ 395 or similar Strix Halo hardware on Ubuntu 24.04, this upgrade is worth doing. The procedure is well-defined, the rollback path is clear, and the newer driver stack brings tangible benefits. Just remember to hold your kernel first.

Recommended Resources

Hardware

Bosgame M5 AI Mini PC (Ryzen AI MAX+ 395) - The system used in this post
GMKtec EVO X2 (Ryzen AI MAX+ 395) - Another Strix Halo mini PC option on Amazon

Books

Deep Learning with PyTorch by Stevens, Antiga, Huang, Viehmann - Comprehensive guide to building, training, and tuning neural networks with PyTorch
Programming PyTorch for Deep Learning by Ian Pointer - Practical guide to creating and deploying deep learning applications
Understanding Deep Learning by Simon Prince - Modern treatment of deep learning fundamentals

Image Editing on 10-Year-Old GPUs: NVIDIA P40 vs AMD Strix Halo

A.C. Jokela — Tue, 17 Feb 2026 18:00:00 GMT

Introduction

There's a certain satisfaction in making old hardware do new tricks. When NVIDIA released the Tesla P40 in 2016, deep learning was still finding its footing. ImageNet classification was the benchmark everyone cared about, GANs were generating blurry faces, and the idea of a 57-billion-parameter image editing model would have seemed like science fiction.

Around the middle of 2017, when the P40 would have been seeing peak adoption in datacenters, I found myself in an advanced pattern recognition course, my final credits needed for a masters in computer science (the name hadn't been updated to reflect more contemporary terminology like "machine learning," let alone "deep learning"). The textbook was Bishop's Pattern Recognition and Machine Learning, a book that managed to make Bayesian inference feel both rigorous and approachable. We spent the last two weeks of the course looking at deep learning using TensorFlow, but we didn't even have GPU infrastructure available. Everything ran on CPU. It would have been great to have experienced the P40 in its prime, when 24 GB of VRAM and 3,840 CUDA cores made it one of the most capable inference GPUs money could buy. Instead, I'm getting acquainted with it a decade later, asking it to do things its designers never imagined.

Fast forward to 2026, and here I am, running a 57-billion-parameter model on four of these decade-old GPUs, and comparing the results against AMD's latest Strix Halo APU, a chip that didn't exist until 2025.

The model in question is FireRed-Image-Edit-1.0 from FireRedTeam, a 57.7GB diffusion model built on the QwenImageEditPlusPipeline architecture. It takes an input image and a text prompt, then produces an edited version. The kind of thing that would have required a massive cloud GPU a couple of years ago.

This post documents the full journey: the precision pitfalls of running modern diffusion models on Pascal-era GPUs, the quantization trade-offs that make or break image quality, and the head-to-head performance comparison that produced some genuinely surprising results. All of the inference scripts and output images are available on GitHub.

The Hardware

NVIDIA Tesla P40 (2016)

The P40 was NVIDIA's inference-focused datacenter GPU from the Pascal generation. The key specs for our purposes:

Architecture: Pascal (sm_6.1)
CUDA Cores: 3,840
Memory: 24 GB GDDR5X
Memory Bandwidth: 346 GB/s
FP32 Performance: 12 TFLOPS
FP16 Performance: Limited, no native FP16 tensor cores
BF16 Support: None
Price today: ~$100-200 per card on the secondary market

I have four of these cards in a server, giving me 96 GB of total VRAM, but spread across four separate memory spaces, which introduces its own challenges.

AMD Ryzen AI MAX+ 395 / Strix Halo (2025)

AMD's Strix Halo is a different beast entirely. It's an APU (CPU and GPU on the same die, sharing the same memory pool):

GPU Architecture: RDNA 3.5 (gfx1151)
Compute Units: 40 CUs
Memory: 128 GB unified LPDDR5X (32 GB for CPU, 96 GB for VRAM)
Memory Bandwidth: ~256 GB/s (shared)
BF16 Support: Yes
FP16 Support: Yes (Fast F16 Operation)
ROCm: 7.9.0
Price: ~$2,000+ for the complete system

The unified memory architecture means all 96 GB is accessible to the GPU without any PCIe transfer overhead, and the entire model can live in a single memory space.

The Model: FireRed-Image-Edit-1.0

FireRed-Image-Edit is a diffusion-based image editing model with three major components:

Component	Size	Description
Transformer	40.9 GB	QwenImageTransformer2DModel, 60 layers
Text Encoder	16.6 GB	Qwen2.5-VL 7B vision-language model
VAE	~0.3 GB	AutoencoderKL for encoding/decoding images

Total: 57.7 GB of model weights. The scheduler is FlowMatchEulerDiscreteScheduler, and the pipeline uses true classifier-free guidance (CFG), which roughly doubles the memory needed during inference since it runs both conditional and unconditional passes.

The test task: take this input image and apply the prompt "Add a red hat on the cat"; the model draws a cat wearing a red hat onto the book cover, rendered in the style of the O'Reilly animal illustrations.

The P40 Challenge: When FP16 Breaks Everything

The Precision Problem

The first, and biggest, challenge with the P40s is numerical precision. Modern diffusion models are designed for BF16 (bfloat16), which has the same exponent range as FP32 (8 exponent bits, range ±3.4×10³⁸) but with reduced mantissa precision. The P40, being a Pascal-era GPU, supports neither BF16 nor proper FP16 tensor operations.

FP16 has only 5 exponent bits, giving it a range of ±65,504. This might seem sufficient, but the diffusion scheduler's internal sigma values and the VAE's convolution operations routinely produce intermediate values that overflow this range. The FlowMatchEulerDiscreteScheduler, in particular, works with sigma schedules that can produce large intermediate values during the noise prediction and scaling steps. When these overflow FP16's limited range, they become NaN or Inf, and these corrupt values propagate through every subsequent operation (matrix multiplications, attention computations, residual connections) until the entire tensor is garbage.

The result: NaN propagation that silently corrupts the entire pipeline, producing an all-black output image.

This was the most time-consuming discovery in the entire project. The model would load, the progress bar would advance through all 40 denoising steps without any indication of trouble, and then the output would be perfectly black: mean=0.0, min=0, max=0. No error messages. No warnings. No NaN detection exceptions. Just silent numerical corruption that only becomes visible when you look at the final image.

The debugging process was particularly frustrating because the corruption happens gradually. Partial NaN contamination in early steps doesn't crash anything; the attention mechanisms and residual connections continue to produce tensor outputs of the expected shapes. The model appears to be working normally right up until the final image is decoded from all-zero latents.

The FP32 Solution (and a Speed Surprise)

The fix was to run the entire pipeline in FP32: scheduler, VAE, and all non-quantized transformer layers. The quantized weights themselves stay compressed (INT8 or NF4), but every arithmetic operation uses full 32-bit precision.

It wasn't enough to just set the quantization compute dtype to FP32; that only fixes the dequantized matmul operations inside the quantized layers. The scheduler's sigma arithmetic, the VAE's convolution operations, and the non-quantized components (layer norms, biases, attention scaling) all needed FP32 as well. Similarly, loading the pipeline with torch_dtype=torch.float32 but leaving the transformer's non-quantized layers in FP16 caused a dtype mismatch in the attention mechanism; PyTorch's scaled dot-product attention requires query, key, and value tensors to share the same dtype. Every component in the computational chain needed to be FP32.

The one exception is the text encoder, which runs once before the denoising loop begins. It stays in FP16 on its own GPU, and its output embeddings are upcast to FP32 when transferred to the main device. This is safe because the text encoder doesn't participate in the iterative process where precision errors compound.

Here's where things got interesting: FP32 was actually faster than FP16 on the P40. The first attempts with FP16 ran at approximately 9 minutes per denoising step. After switching to FP32, the same operations completed in about 2.4 minutes per step with NF4, and 1.5 minutes per step with INT8. The P40's FP32 throughput is its native strength; it was designed for FP32 datacenter inference, after all. FP16 on Pascal is handled through slower pathways that add overhead rather than saving it.

Multi-GPU Device Orchestration

With 57.7 GB of model weights and only 24 GB per GPU, some form of model sharding or quantization is mandatory. After extensive testing, the optimal configuration for the P40s turned out to be:

GPU 0: INT8-quantized transformer (~22 GB)
GPU 1: Text encoder in FP16 (~16.6 GB)
GPU 2: VAE in FP32 (~6.6 GB including decode workspace)
GPU 3: Unused

This layout requires significant monkey-patching of the diffusers pipeline. The _execution_device property must be overridden to ensure latents are created on the correct GPU. The encode_prompt method needs patching to route inputs to the text encoder's GPU and move the resulting embeddings back. And for the INT8 configuration, the VAE's encode and decode methods need wrappers to handle cross-device tensor transfers.

The text encoder stays in FP16 because it fits on a single GPU and its outputs are immediately upcast to FP32 when moved to the main device. This is safe because the text encoder runs once at the beginning; it doesn't participate in the iterative denoising loop where precision matters most.

Quantization Quality: INT8 vs NF4

With the FP32 pipeline in place, I tested both INT8 (8-bit) and NF4 (4-bit) quantization for the transformer:

NF4 (4-bit quantization): The NF4 approach uses bitsandbytes' normalized float 4-bit quantization with double quantization enabled. The transformer compresses from 40.9 GB to roughly 10 GB, easily fitting on a single P40 alongside the VAE. However, the output quality was significantly degraded, with heavy noise and grain throughout the image, even at the full 40 denoising steps. Each denoising step introduces small numerical errors from the 4-bit weight approximations, and these errors compound across 40 iterations.

INT8 (8-bit quantization): INT8 produced dramatically better results. The output was clean and sharp, visually comparable to what you'd expect from full-precision inference on a modern GPU. The 8-bit precision preserves enough information in the weights that the per-step errors don't accumulate into visible artifacts.

The trade-off is memory: the INT8 transformer occupies ~22 GB, nearly filling an entire P40. This is why the VAE had to move to a third GPU; there wasn't enough headroom on GPU 0 for the VAE's convolution workspace during the decode phase. An early attempt that kept the VAE on GPU 0 ran all 40 denoising steps successfully, only to crash with an out-of-memory error at the very last operation.

The Strix Halo Experience: Simplicity Wins

BF16 Full Precision

Running the same model on the Strix Halo was refreshingly simple. With 96 GB of unified VRAM and native BF16 support, the entire pipeline loads in a few lines:

pipe = QwenImageEditPlusPipeline.from_pretrained(
    model_path,
    torch_dtype=torch.bfloat16,
)
pipe.to("cuda:0")

No quantization. No multi-GPU patching. No device transfer hooks. No FP32 workarounds. The model loads in BF16 and runs natively.

During inference, the pipeline consumed approximately 75 GB of VRAM (the true CFG doubles the workspace requirements), well within the 96 GB budget.

The first run did take about 35 minutes of JIT kernel compilation before producing any inference steps; ROCm compiles HIP kernels for the gfx1151 architecture on first use. During this phase, the GPU sits at 100% utilization with no visible progress, which can be alarming if you're not expecting it. The GPU temperature climbed from 31°C idle to 69°C, and power draw went from 9W to 119W as the compiler worked through the hundreds of unique kernel configurations needed by a 60-layer transformer. These compiled kernels are cached, so subsequent runs skip this overhead entirely.

Quantization on Strix Halo: Does It Help?

Given the surprising performance parity between the two systems at full precision, I tested whether quantization could speed up the Strix Halo by reducing memory traffic. The theory was that if the workload is memory-bandwidth-limited, smaller model weights should mean faster inference.

The results were definitive:

Configuration	Per-Step Time	40-Step Estimate	VRAM Used
BF16 (full precision)	82.6s	55 min	~75 GB
NF4 (4-bit)	83.5s	56 min	~30 GB
INT8 (8-bit)	94.9s	63 min	~44 GB

NF4 quantization produced virtually identical speed to full BF16. The model shrank from 75 GB to 30 GB of VRAM usage, but inference time didn't improve at all. INT8 was actually slower; the bitsandbytes INT8 matmul path adds dequantization overhead that more than offsets any memory bandwidth savings.

This tells us something important about the Strix Halo's performance profile for this workload: it's compute-bound, not memory-bound. The RDNA 3.5 GPU's 40 compute units are the bottleneck, not the LPDDR5X memory bandwidth. Reducing the model size doesn't help because the GPU is already busy with arithmetic, not waiting on memory.

This contrasts with LLM inference workloads (text generation), where the Strix Halo's large memory pool is a genuine advantage. LLM token generation is almost entirely memory-bound, making quantization directly translate to speed improvements. Each token generation pass reads the entire model's weights but performs relatively little computation per weight. Diffusion models are the opposite: each denoising step runs a full forward pass through 60 transformer layers with dense matrix multiplications, attention computations, and residual connections. The arithmetic intensity is much higher, putting the pressure squarely on the GPU's compute units rather than its memory subsystem.

Head-to-Head: The Numbers

Here's the complete performance comparison across all tested configurations:

System	Configuration	Per-Step	40 Steps	Image Quality
Strix Halo	BF16 full precision	82.6s	55 min	Clean, sharp
Strix Halo	NF4 (4-bit)	83.5s	56 min	Clean (10-step test)
Strix Halo	INT8 (8-bit)	94.9s	63 min	Clean (10-step test)
4× P40	INT8 + FP32 pipeline	87.5s	58 min	Clean, sharp
4× P40	NF4 + FP32 pipeline	145.9s	97 min	Heavy noise/grain

The headline result: a single AMD Strix Halo APU from 2025 is about 6% faster per step than four NVIDIA P40s from 2016 running INT8-quantized inference. That's not exactly the generational leap you might expect from a decade of GPU evolution.

To be fair, the comparison isn't entirely apples-to-apples. The P40 is running an 8-bit quantized model (less computation per step but with dequantization overhead), while the Strix Halo runs the full BF16 model. The P40's dedicated GDDR5X provides 346 GB/s of bandwidth to a single GPU, while the Strix Halo's LPDDR5X shares its ~256 GB/s between the CPU and GPU. And the P40 setup requires three GPUs working in concert, while the Strix Halo uses a single unified memory space.

Lessons Learned

Old GPUs Are Surprisingly Capable

Four P40s at ~$500 total produce inference quality and speed that's competitive with a $2,000+ modern APU system. The P40's 346 GB/s memory bandwidth per card and strong FP32 throughput remain relevant even for models that were designed for hardware two generations newer. The main challenge is software engineering: working around the precision limitations and multi-GPU complexity takes significant effort.

Precision Matters More Than Speed

The single most impactful discovery in this project was that FP16 silently corrupts diffusion model outputs on Pascal GPUs. There are no error messages, no NaN warnings during inference, just a black image at the end. The fix (using FP32 everywhere) actually improved performance, which was counterintuitive. The lesson: when dealing with older hardware, always validate your numerical precision assumptions before optimizing for speed.

Quantization Is Not Free

On the P40s, INT8 quantization was essential (the model simply wouldn't fit otherwise) and produced excellent results. NF4 was too aggressive; the 4-bit precision degraded output quality visibly.

On the Strix Halo, quantization was unnecessary and even counterproductive. INT8 added overhead without any speed benefit, and NF4 didn't save time despite dramatically reducing memory usage. The takeaway: quantization's value depends entirely on your bottleneck. If you're compute-bound, smaller weights don't help.

Unified Memory Is Underrated

The Strix Halo's greatest advantage wasn't raw performance; it was simplicity. Loading a 57.7 GB model into a single 96 GB memory space eliminates an entire category of engineering problems: no device placement, no cross-GPU tensor transfers, no monkey-patching encode/decode methods, no VAE OOM surprises at the decode step. The inference script for the Strix Halo is about 50 lines. The P40 version is over 150, most of it careful device orchestration code.

For anyone who values development velocity and code maintainability over squeezing the last dollar of cost-efficiency out of used datacenter hardware, unified memory APUs have a compelling argument even when they don't win on raw throughput.

What About Newer NVIDIA GPUs?

It's worth putting these numbers in context. An NVIDIA RTX 4090 with 24 GB of VRAM and native BF16/FP16 tensor core support would likely run this model (with INT8 quantization) at roughly 10-15 seconds per step, 5-8x faster than either system tested here. An A100 with 80 GB could run it unquantized in BF16 at similar or better speeds. The P40 and Strix Halo are both firmly in the "budget/accessible" tier of AI hardware.

The more interesting comparison is cost-per-step. Four P40s from eBay cost about $500 total (plus a server that can host them). The Strix Halo system runs about $2,000+. Both produce essentially the same result at the same speed. The P40 route demands more engineering knowledge; the Strix Halo route demands more money.

Conclusion

Both systems successfully ran a 57.7 GB diffusion model that would have been considered impossibly large for consumer hardware just a few years ago. The P40s did it through clever quantization and multi-GPU orchestration. The Strix Halo did it by brute force: 96 GB of memory and native BF16 support.

The performance story is more nuanced than "newer is always better." For diffusion model inference, the NVIDIA P40 (a card you can buy for $100 on eBay) remains remarkably competitive when properly configured. It requires more engineering effort, and you need to know the precision pitfalls, but the results speak for themselves.

The Strix Halo's strength lies not in raw speed but in its unified memory architecture and modern instruction set support. It eliminates the multi-GPU complexity entirely, runs native BF16 without precision hacks, and provides a development experience that's orders of magnitude simpler. For iterating on models, testing new architectures, or just avoiding the headaches of cross-device tensor management, that simplicity has real value.

If you're considering hardware for running large diffusion models locally, the choice comes down to how you value your time versus your budget. Four P40s and a weekend of debugging will get you to roughly the same place as a Strix Halo system that just works out of the box. Both paths lead to a cat in a red hat.

Recommended Resources

Hardware

NVIDIA Tesla P40 24GB - The GPU used in this post. Available on eBay for a fraction of the original price. You'll need a server with PCIe x16 slots and adequate cooling.
GMKtec EVO-X2 (AMD Ryzen AI MAX+ 395) - A compact Strix Halo mini PC with 128GB unified LPDDR5X 8000MHz, WiFi 7, and USB4. A representative platform for running large models on Strix Halo.

Books

Pattern Recognition and Machine Learning by Christopher M. Bishop - The classic that introduced many to Bayesian methods and kernel machines. Still one of the best foundations for understanding the statistical principles behind modern ML.
Hands-On Generative AI with Transformers and Diffusion Models by Omar Sanseviero, Pedro Cuenca, Apolinário Passos, and Jonathan Whitaker - A practical guide to building and fine-tuning diffusion models using the Hugging Face ecosystem, including the diffusers library used in this post.
Understanding Deep Learning by Simon J.D. Prince - A thorough modern treatment of deep learning fundamentals through diffusion models, with excellent visualizations and mathematical rigor.

Getting PyTorch Working with AMD Radeon Pro W7900 (MAX+ 395): A Comprehensive Guide

A.C. Jokela — Sat, 11 Oct 2025 23:08:14 GMT

Your browser does not support the audio element.

Getting PyTorch Working with AMD Radeon Pro W7900 (MAX+ 395): A Comprehensive Guide

Introduction

The AMD Radeon Pro W7900 represents a significant leap forward in professional GPU computing. With 96GB of unified memory and 20 compute units, this workstation-class GPU brings serious computational power to tasks like machine learning, scientific computing, and data analysis. However, getting deep learning frameworks like PyTorch to work with AMD GPUs has historically been more challenging than with NVIDIA's CUDA ecosystem.

Here's a complete walkthrough of setting up PyTorch with ROCm support on the AMD MAX+ 395, including installation, verification, and real-world testing. By the end, you'll have a fully functional PyTorch environment capable of leveraging your AMD GPU's computational power.

Understanding ROCm and PyTorch

What is ROCm?

ROCm (Radeon Open Compute) is AMD's open-source software platform for GPU computing. It serves as AMD's answer to NVIDIA's CUDA, providing:

Low-level GPU programming interfaces
Optimized libraries for linear algebra, FFT, and other operations
Deep learning framework support
Compatibility with CUDA-based code through HIP (Heterogeneous-compute Interface for Portability)

PyTorch and ROCm Integration

PyTorch has officially supported ROCm since version 1.8, and support has matured significantly over subsequent releases. The ROCm version of PyTorch uses the same API as the CUDA version, making it straightforward to port existing PyTorch code to AMD GPUs. In fact, most PyTorch code written for CUDA will work without modification on ROCm, as the framework abstracts away the underlying GPU platform.

System Specifications

Testing was performed on a system with the following specifications:

GPU: AMD Radeon Pro W7900 (MAX+ 395)
GPU Memory: 96 GB
Compute Units: 20
CUDA Capability: 11.5 (ROCm compatibility level)
Operating System: Linux
Python: 3.12.11
PyTorch Version: 2.8.0+rocm7.0.0
ROCm Version: 7.0.0

Installation and Setup

This section provides detailed, step-by-step instructions for bootstrapping a complete ROCm 7.0 + PyTorch 2.8 environment on Ubuntu 24.04.3 LTS. These instructions are based on successful installations on the AMD Ryzen AI Max+395 platform.

Prerequisites

Ubuntu 24.04.3 LTS (Server or Desktop)
Administrator/sudo access
Internet connection for downloading packages

Step 1: Update Linux Kernel

ROCm 7.0 works best with Linux kernel 6.14 or later. Update your kernel:

sudo apt-get install linux-generic-hwe-24.04

Verify the installation:

cat /proc/version

You should see output similar to:

Linux version 6.14.0-33-generic (buildd@lcy02-amd64-026)...

Reboot to load the new kernel:

sudo reboot

Step 2: Install AMDGPU Driver

First, set up the AMD repository:

# Create keyring directory if it doesn't exist
sudo mkdir --parents --mode=0755 /etc/apt/keyrings

# Download and install AMD GPG key
wget https://repo.radeon.com/rocm/rocm.gpg.key -O - | \
  gpg --dearmor | sudo tee /etc/apt/keyrings/rocm.gpg > /dev/null

# Add AMDGPU repository
sudo tee /etc/apt/sources.list.d/amdgpu.list << EOF
deb [arch=amd64,i386 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/amdgpu/latest/ubuntu noble main
EOF

Install the AMDGPU DKMS driver:

sudo apt update
sudo apt install amdgpu-dkms
sudo reboot

Verify the driver installation:

sudo dkms status

You should see output like:

amdgpu/6.14.14-2212064.24.04, 6.14.0-33-generic, x86_64: installed

Step 3: Install ROCm 7.0

Install prerequisites:

sudo apt install python3-setuptools python3-wheel
sudo apt update

Download and install the AMD GPU installer:

wget https://repo.radeon.com/amdgpu-install/7.0/ubuntu/noble/amdgpu-install_7.0.70000-1_all.deb
sudo apt install ./amdgpu-install_7.0.70000-1_all.deb

Install ROCm with the compute use case (choose Y when prompted to overwrite amdgpu.list):

amdgpu-install -y --usecase=rocm
sudo reboot

Add your user to the required groups:

sudo usermod -a -G render,video $LOGNAME
sudo reboot

Verify ROCm installation:

rocminfo

You should see your GPU listed as an agent with detailed properties.

Step 4: Configure ROCm Libraries

Configure the system to find ROCm shared libraries:

# Add ROCm library paths
sudo tee --append /etc/ld.so.conf.d/rocm.conf <<EOF
/opt/rocm/lib
/opt/rocm/lib64
EOF

sudo ldconfig

# Set library path environment variable (add to ~/.bashrc for persistence)
export LD_LIBRARY_PATH=/opt/rocm-7.0.0/lib:$LD_LIBRARY_PATH

Install and verify OpenCL runtime:

sudo apt install rocm-opencl-runtime
clinfo

The clinfo command should display information about your AMD GPU.

Step 5: Install PyTorch with ROCm Support

Create a conda environment and install PyTorch:

# Create conda environment
conda create -n pt2.8-rocm7 python=3.12
conda activate pt2.8-rocm7

# Install PyTorch 2.8.0 with ROCm 7.0 from AMD's repository
pip install https://repo.radeon.com/rocm/manylinux/rocm-rel-7.0/pytorch_triton_rocm-3.2.0%2Brocm7.0.0.4d510c3a44-cp312-cp312-linux_x86_64.whl
pip install https://repo.radeon.com/rocm/manylinux/rocm-rel-7.0/torch-2.8.0%2Brocm7.0.0-cp312-cp312-linux_x86_64.whl
pip install https://repo.radeon.com/rocm/manylinux/rocm-rel-7.0/torchvision-0.23.0%2Brocm7.0.0-cp312-cp312-linux_x86_64.whl
pip install https://repo.radeon.com/rocm/manylinux/rocm-rel-7.0/torchaudio-2.8.0%2Brocm7.0.0-cp312-cp312-linux_x86_64.whl

# Install GCC 12.1 (required for some operations)
conda install -c conda-forge gcc=12.1.0

Important Notes: - The URLs above are for Python 3.12 (cp312). Adjust for your Python version if different. - These wheels are built specifically for ROCm 7.0 and may not work with other ROCm versions. - The LD_LIBRARY_PATH must be set correctly, or PyTorch won't find ROCm libraries.

Verifying Installation

After installation, perform a quick verification:

import torch
print(f"PyTorch version: {torch.__version__}")
print(f"CUDA available: {torch.cuda.is_available()}")
print(f"Device count: {torch.cuda.device_count()}")
if torch.cuda.is_available():
    print(f"Device name: {torch.cuda.get_device_name(0)}")

Note that despite using ROCm, PyTorch still refers to the GPU API as "CUDA" for compatibility reasons. This is intentional and allows CUDA-based code to run on AMD GPUs without modification.

Comprehensive GPU Testing

To thoroughly validate that PyTorch is working correctly with the MAX+ 395, we developed a comprehensive test suite that exercises various aspects of GPU computing.

Test Suite Overview

Our test suite includes five major components:

Installation Verification: Confirms PyTorch version and GPU detection
ROCm Availability Check: Validates GPU properties and capabilities
Tensor Operations: Tests basic tensor creation and mathematical operations
Neural Network Operations: Validates deep learning functionality
Memory Management: Tests GPU memory allocation and deallocation

Test Script

Here's the complete test script we developed:

#!/usr/bin/env python3
"""
ROCm PyTorch GPU Test POC
Tests if ROCm PyTorch can successfully detect and use AMD GPUs
"""

import torch
import sys

def print_section(title):
    """Print a formatted section header"""
    print(f"\n{'='*60}")
    print(f" {title}")
    print(f"{'='*60}")

def test_pytorch_installation():
    """Test basic PyTorch installation"""
    print_section("PyTorch Installation Info")
    print(f"PyTorch Version: {torch.__version__}")
    print(f"Python Version: {sys.version}")

def test_rocm_availability():
    """Test ROCm/CUDA availability"""
    print_section("ROCm/CUDA Availability")

    cuda_available = torch.cuda.is_available()
    print(f"CUDA Available: {cuda_available}")

    if cuda_available:
        print(f"CUDA Device Count: {torch.cuda.device_count()}")
        print(f"Current Device: {torch.cuda.current_device()}")
        print(f"Device Name: {torch.cuda.get_device_name(0)}")

        props = torch.cuda.get_device_properties(0)
        print(f"\nDevice Properties:")
        print(f"  - Total Memory: {props.total_memory / 1024**3:.2f} GB")
        print(f"  - Multi Processor Count: {props.multi_processor_count}")
        print(f"  - CUDA Capability: {props.major}.{props.minor}")
    else:
        print("No CUDA/ROCm devices detected!")
        return False

    return True

def test_tensor_operations():
    """Test basic tensor operations on GPU"""
    print_section("Tensor Operations Test")

    try:
        cpu_tensor = torch.randn(1000, 1000)
        print(f"CPU Tensor created: {cpu_tensor.shape}")
        print(f"CPU Tensor device: {cpu_tensor.device}")

        gpu_tensor = cpu_tensor.cuda()
        print(f"\nGPU Tensor created: {gpu_tensor.shape}")
        print(f"GPU Tensor device: {gpu_tensor.device}")

        print("\nPerforming matrix multiplication on GPU...")
        result = torch.matmul(gpu_tensor, gpu_tensor)
        print(f"Result shape: {result.shape}")
        print(f"Result device: {result.device}")

        cpu_result = result.cpu()
        print(f"Moved result back to CPU: {cpu_result.device}")

        print("\n✓ Tensor operations successful!")
        return True

    except Exception as e:
        print(f"\n✗ Tensor operations failed: {e}")
        return False

def test_simple_neural_network():
    """Test a simple neural network operation on GPU"""
    print_section("Neural Network Test")

    try:
        model = torch.nn.Sequential(
            torch.nn.Linear(100, 50),
            torch.nn.ReLU(),
            torch.nn.Linear(50, 10)
        )

        print("Model created on CPU")
        print(f"Model device: {next(model.parameters()).device}")

        model = model.cuda()
        print(f"Model moved to GPU: {next(model.parameters()).device}")

        input_data = torch.randn(32, 100).cuda()
        print(f"\nInput data shape: {input_data.shape}")
        print(f"Input data device: {input_data.device}")

        print("Performing forward pass...")
        output = model(input_data)
        print(f"Output shape: {output.shape}")
        print(f"Output device: {output.device}")

        print("\n✓ Neural network test successful!")
        return True

    except Exception as e:
        print(f"\n✗ Neural network test failed: {e}")
        return False

def test_memory_management():
    """Test GPU memory management"""
    print_section("GPU Memory Management Test")

    try:
        if torch.cuda.is_available():
            print(f"Allocated Memory: {torch.cuda.memory_allocated(0) / 1024**2:.2f} MB")
            print(f"Cached Memory: {torch.cuda.memory_reserved(0) / 1024**2:.2f} MB")

            tensors = []
            for i in range(5):
                tensors.append(torch.randn(1000, 1000).cuda())

            print(f"\nAfter allocating 5 tensors:")
            print(f"Allocated Memory: {torch.cuda.memory_allocated(0) / 1024**2:.2f} MB")
            print(f"Cached Memory: {torch.cuda.memory_reserved(0) / 1024**2:.2f} MB")

            del tensors
            torch.cuda.empty_cache()

            print(f"\nAfter clearing cache:")
            print(f"Allocated Memory: {torch.cuda.memory_allocated(0) / 1024**2:.2f} MB")
            print(f"Cached Memory: {torch.cuda.memory_reserved(0) / 1024**2:.2f} MB")

            print("\n✓ Memory management test successful!")
            return True
        else:
            print("No GPU available for memory test")
            return False

    except Exception as e:
        print(f"\n✗ Memory management test failed: {e}")
        return False

def main():
    """Run all tests"""
    print("\n" + "="*60)
    print(" ROCm PyTorch GPU Test POC")
    print("="*60)

    test_pytorch_installation()

    if not test_rocm_availability():
        print("\n" + "="*60)
        print(" FAILED: No ROCm/CUDA devices available")
        print("="*60)
        sys.exit(1)

    results = []
    results.append(("Tensor Operations", test_tensor_operations()))
    results.append(("Neural Network", test_simple_neural_network()))
    results.append(("Memory Management", test_memory_management()))

    print_section("Test Summary")
    all_passed = True
    for test_name, passed in results:
        status = "✓ PASSED" if passed else "✗ FAILED"
        print(f"{test_name}: {status}")
        if not passed:
            all_passed = False

    print("\n" + "="*60)
    if all_passed:
        print(" SUCCESS: All tests passed! ROCm GPU is working.")
    else:
        print(" PARTIAL SUCCESS: Some tests failed.")
    print("="*60 + "\n")

    return 0 if all_passed else 1

if __name__ == "__main__":
    sys.exit(main())

Test Results and Analysis

Running our comprehensive test suite on the MAX+ 395 yielded excellent results across all categories.

GPU Detection and Properties

The first test confirmed that PyTorch successfully detected the AMD GPU:

CUDA Available: True
CUDA Device Count: 1
Current Device: 0
Device Name: AMD Radeon Graphics

Device Properties:
  - Total Memory: 96.00 GB
  - Multi Processor Count: 20
  - CUDA Capability: 11.5

The 96GB of memory is particularly impressive, far exceeding what's available on most consumer or even professional NVIDIA GPUs. This massive memory capacity opens up possibilities for:

Training larger models without splitting across multiple GPUs
Processing high-resolution images or long sequences
Handling larger batch sizes for improved training efficiency
Running multiple models simultaneously

Tensor Operations Performance

Basic tensor operations executed flawlessly:

CPU Tensor created: torch.Size([1000, 1000])
CPU Tensor device: cpu

GPU Tensor created: torch.Size([1000, 1000])
GPU Tensor device: cuda:0

Performing matrix multiplication on GPU...
Result shape: torch.Size([1000, 1000])
Result device: cuda:0
Moved result back to CPU: cpu

✓ Tensor operations successful!

The seamless movement of tensors between CPU and GPU memory, along with successful matrix multiplication, confirms that the fundamental PyTorch operations work correctly on ROCm.

Neural Network Operations

Our neural network test validated that PyTorch's high-level APIs work correctly:

Model created on CPU
Model device: cpu
Model moved to GPU: cuda:0

Input data shape: torch.Size([32, 100])
Input data device: cuda:0
Performing forward pass...
Output shape: torch.Size([32, 10])
Output device: cuda:0

✓ Neural network test successful!

This test confirms that: - Models can be moved to GPU with the .cuda() method - Forward passes execute correctly on GPU - All layers (Linear, ReLU) are properly accelerated

Memory Management

The memory management test showed efficient allocation and deallocation:

Allocated Memory: 32.00 MB
Cached Memory: 54.00 MB

After allocating 5 tensors:
Allocated Memory: 52.00 MB
Cached Memory: 54.00 MB

After clearing cache:
Allocated Memory: 32.00 MB
Cached Memory: 32.00 MB

PyTorch's memory management on ROCm works identically to CUDA, with proper caching behavior and the ability to manually clear cached memory when needed.

Performance Considerations

Memory Bandwidth

The MAX+ 395's 96GB of memory is a significant advantage, but memory bandwidth is equally important for deep learning workloads. The W7900's memory subsystem provides substantial bandwidth for data transfers between GPU memory and compute units.

Compute Performance

With 20 compute units, the MAX+ 395 provides substantial parallel processing capability. While direct comparisons to NVIDIA GPUs depend on the specific workload, ROCm's optimization for AMD architectures ensures efficient utilization of available compute resources.

Software Maturity

ROCm has matured significantly over recent years. Most PyTorch operations that work on CUDA now work seamlessly on ROCm. However, some edge cases and newer features may still have better support on CUDA, so testing your specific workload is recommended.

Practical Tips and Best Practices

Code Portability

To write code that works on both CUDA and ROCm:

# Use device-agnostic code
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)
inputs = inputs.to(device)

Monitoring GPU Utilization

Use rocm-smi to monitor GPU utilization:

watch -n 1 rocm-smi

This provides real-time information about GPU usage, memory consumption, temperature, and power draw.

Optimizing Memory Usage

With 96GB available, you might be tempted to use very large batch sizes. However, optimal batch size depends on many factors:

# Experiment with batch sizes
for batch_size in [32, 64, 128, 256]:
    # Train and measure throughput
    # Find the sweet spot between memory usage and performance

Debugging

Enable PyTorch's anomaly detection during development:

torch.autograd.set_detect_anomaly(True)

Troubleshooting Common Issues

GPU Not Detected

If torch.cuda.is_available() returns False:

Verify ROCm installation: rocm-smi
Check PyTorch was installed with ROCm support: print(torch.__version__) should show +rocm
Ensure ROCm drivers match PyTorch's ROCm version

Out of Memory Errors

Even with 96GB, you can run out of memory:

# Clear cache periodically
torch.cuda.empty_cache()

# Use gradient checkpointing for large models
from torch.utils.checkpoint import checkpoint

Performance Issues

If training is slower than expected:

Profile your code: torch.profiler.profile()
Check for CPU-GPU transfer bottlenecks
Verify data loading isn't the bottleneck
Consider using mixed precision training with torch.cuda.amp

Conclusion

The AMD Radeon Pro W7900 (MAX+ 395) with ROCm provides a robust, capable platform for PyTorch-based machine learning workloads. Our comprehensive testing demonstrated that:

PyTorch 2.8.0 with ROCm 7.0.0 works seamlessly with the MAX+ 395
All tested operations (tensors, neural networks, memory management) function correctly
The massive 96GB memory capacity enables unique use cases
Code written for CUDA generally works without modification

For organizations invested in AMD hardware or looking for alternatives to NVIDIA's ecosystem, the MAX+ 395 with ROCm represents a viable option for deep learning workloads. The open-source nature of ROCm and PyTorch's strong support for the platform ensure that AMD GPUs are first-class citizens in the deep learning community.

As ROCm continues to evolve and PyTorch support deepens, AMD's GPU offerings will only become more compelling for machine learning practitioners. The MAX+ 395, with its exceptional memory capacity and solid compute performance, stands ready to tackle demanding deep learning tasks.

Acknowledgments

The detailed ROCm 7.0 installation procedure is based on Wei Lu's excellent article "Ultralytics YOLO/SAM with ROCm 7.0 on AMD Ryzen AI Max+395 'Strix Halo'" published on Medium in October 2025. Wei Lu's pioneering work in documenting the complete bootstrapping process for ROCm 7.0 on the Max+395 platform made this possible.

Resources

Based on real-world testing performed on October 10, 2025, using PyTorch 2.8.0 with ROCm 7.0.0 on an AMD Radeon Pro W7900 GPU with 96GB memory. Installation instructions adapted from Wei Lu's documentation of the AMD Ryzen AI Max+395 platform.

AMD AI Max+ 395 System Review: A Comprehensive Analysis

A.C. Jokela — Sun, 21 Sep 2025 20:25:28 GMT

Executive Summary

The AMD AI Max+ 395 system represents AMD's latest entry into the high-performance computing and AI acceleration market, featuring the company's cutting-edge Strix Halo architecture. This comprehensive review examines the system's performance characteristics, software compatibility, and overall viability for AI workloads and general computing tasks. While the hardware shows impressive potential with its 16-core CPU and integrated Radeon 8060S graphics, significant software ecosystem challenges, particularly with PyTorch/ROCm compatibility for the gfx1151 architecture, present substantial barriers to immediate adoption for AI development workflows.

Note: An Orange Pi 5 Max was photobombing this photograph

System Specifications and Architecture Overview

CPU Specifications

Processor: AMD RYZEN AI MAX+ 395 w/ Radeon 8060S
Architecture: x86_64 with Zen 5 cores
Cores/Threads: 16 cores / 32 threads
Base Clock: 599 MHz (minimum)
Boost Clock: 5,185 MHz (maximum)
Cache Configuration:
L1d Cache: 768 KiB (16 instances, 48 KiB per core)
L1i Cache: 512 KiB (16 instances, 32 KiB per core)
L2 Cache: 16 MiB (16 instances, 1 MiB per core)
L3 Cache: 64 MiB (2 instances, 32 MiB per CCX)
Instruction Set Extensions: Full AVX-512, AVX-VNNI, BF16 support

Memory Subsystem

Total System Memory: 32 GB DDR5
Memory Configuration: Unified memory architecture with shared GPU/CPU access
Memory Bandwidth: Achieved ~13.5 GB/s in multi-threaded tests

Graphics Processing Unit

GPU Architecture: Strix Halo (RDNA 3.5 based)
GPU Designation: gfx1151
Compute Units: 40 CUs (80 reported in ROCm, likely accounting for dual SIMD per CU)
Peak GPU Clock: 2,900 MHz
VRAM: 96 GB shared system memory (103 GB total addressable) - Note: This allocation was intentionally configured to maximize GPU memory for large language model inference
Memory Bandwidth: Shared with system memory
OpenCL Compute Units: 20 (as reported by clinfo)

Platform Details

Operating System: Ubuntu 24.04.3 LTS (Noble)
Kernel Version: 6.8.0-83-generic
Architecture: x86_64
Virtualization: AMD-V enabled

Performance Benchmarks

Figure 1: Comprehensive performance analysis and compatibility overview of the AMD AI Max+ 395 system

CPU Performance Analysis

Single-Threaded Performance

The sysbench CPU benchmark with prime number calculation revealed strong single-threaded performance:

Events per second: 6,368.92
Average latency: 0.16 ms
95th percentile latency: 0.16 ms

This performance places the AMD AI Max+ 395 in the upper tier of modern processors for single-threaded workloads, demonstrating the effectiveness of the Zen 5 architecture's IPC improvements and high boost clocks.

Multi-Threaded Performance

Multi-threaded testing across all 32 threads showed excellent scaling:

Events per second: 103,690.35
Scaling efficiency: 16.3x improvement over single-threaded (theoretical maximum 32x)
Thread fairness: Excellent distribution with minimal standard deviation

The scaling efficiency of approximately 51% indicates good multi-threading performance, though there's room for optimization in workloads that can fully utilize all available threads.

Memory Performance

Memory Bandwidth Testing

Memory performance testing using sysbench revealed:

Single-threaded bandwidth: 9.3 GB/s
Multi-threaded bandwidth: 13.5 GB/s (16 threads)
Latency characteristics: Sub-millisecond access times

The memory bandwidth results suggest the system is well-balanced for most workloads, though AI applications requiring extremely high memory bandwidth may find this a limiting factor compared to discrete GPU solutions with dedicated VRAM.

GPU Performance and Capabilities

Hardware Specifications

The integrated Radeon 8060S GPU presents impressive specifications on paper:

Architecture: RDNA 3.5 (Strix Halo)
Compute Units: 40 CUs with 2 SIMDs each
Memory Access: Full 96 GB of shared system memory
Clock Speed: Up to 2.9 GHz

OpenCL Capabilities

OpenCL enumeration reveals solid compute capabilities:

Device Type: GPU with full OpenCL 2.1 support
Max Compute Units: 20 (OpenCL reporting)
Max Work Group Size: 256
Image Support: Full 2D/3D image processing capabilities
Memory Allocation: Up to 87 GB maximum allocation

Network Performance Testing

Network infrastructure testing using iperf3 demonstrated excellent localhost performance:

Loopback Bandwidth: 122 Gbits/sec sustained
Latency: Minimal retransmissions (0 retries)
Consistency: Stable performance across 10-second test duration

This indicates robust internal networking capabilities suitable for distributed computing scenarios and high-bandwidth data transfer requirements.

PyTorch/ROCm Compatibility Analysis

Current State of ROCm Support

We installed ROCm 7.0 and related components: - ROCm Version: 7.0.0 - HIP Version: 7.0.51831 - PyTorch Version: 2.5.1+rocm6.2

gfx1151 Compatibility Issues

The most significant finding of this review centers on the gfx1151 architecture compatibility with current AI software stacks. Testing revealed critical limitations:

PyTorch Compatibility Problems

rocBLAS error: Cannot read TensileLibrary.dat: Illegal seek for GPU arch : gfx1151
List of available TensileLibrary Files:
- TensileLibrary_lazy_gfx1030.dat
- TensileLibrary_lazy_gfx906.dat
- TensileLibrary_lazy_gfx908.dat
- TensileLibrary_lazy_gfx942.dat
- TensileLibrary_lazy_gfx900.dat
- TensileLibrary_lazy_gfx90a.dat
- TensileLibrary_lazy_gfx1100.dat

This error indicates that PyTorch's ROCm backend lacks pre-compiled optimized kernels for the gfx1151 architecture. The absence of gfx1151 in the TensileLibrary files means:

No Optimized BLAS Operations: Matrix multiplication, convolutions, and other fundamental AI operations cannot leverage GPU acceleration
Training Workflows Broken: Most deep learning training pipelines will fail or fall back to CPU execution
Inference Limitations: Even basic neural network inference is compromised

Root Cause Analysis

The gfx1151 architecture represents a newer GPU design that hasn't been fully integrated into the ROCm software stack. While the hardware is detected and basic OpenCL operations function, the optimized compute libraries essential for AI workloads are missing.

Workaround Attempts

Testing various workarounds yielded limited success:

HSA_OVERRIDE_GFX_VERSION=11.0.0: Failed to resolve compatibility issues
CPU Fallback: PyTorch operates normally on CPU, but defeats the purpose of GPU acceleration
Basic GPU Operations: Simple tensor allocation succeeds, but compute operations fail

Software Ecosystem Gaps

Beyond PyTorch, the gfx1151 compatibility issues extend to:

TensorFlow: Likely similar rocBLAS dependency issues
JAX: ROCm backend compatibility uncertain
Scientific Computing: NumPy/SciPy GPU acceleration unavailable
Machine Learning Frameworks: Most frameworks dependent on rocBLAS will encounter issues

AMD GPU Software Support Ecosystem Analysis

Current State Assessment

AMD's GPU software ecosystem has made significant strides but remains fragmented compared to NVIDIA's CUDA platform:

Strengths

Open Source Foundation: ROCm's open-source nature enables community contributions
Standard API Support: OpenCL 2.1 and HIP provide industry-standard interfaces
Linux Integration: Strong kernel-level support through AMDGPU drivers
Professional Tools: rocm-smi and related utilities provide comprehensive monitoring

Weaknesses

Fragmented Architecture Support: New architectures like gfx1151 lag behind in software support
Limited Documentation: Less comprehensive than CUDA documentation
Smaller Developer Community: Fewer third-party tools and optimizations
Compatibility Matrix Complexity: Different software versions support different GPU architectures

Long-term Viability Concerns

The gfx1151 compatibility issues highlight broader ecosystem challenges:

Release Coordination Problems

Hardware releases outpace software ecosystem updates
Critical libraries (rocBLAS, Tensile) require architecture-specific optimization
Coordination between AMD hardware and software teams appears insufficient

Market Adoption Barriers

Developers hesitant to adopt platform with uncertain software support
Enterprise customers require guaranteed compatibility
Academic researchers need stable, well-documented platforms

Recommendations for AMD

Accelerated Software Development: Prioritize gfx1151 support in rocBLAS and related libraries
Pre-release Testing: Ensure software ecosystem readiness before hardware launches
Better Documentation: Comprehensive compatibility matrices and migration guides
Community Engagement: More responsive developer relations and support channels

Network Infrastructure and Connectivity

The system demonstrates excellent network performance characteristics suitable for modern computing workloads:

Internal Performance

Memory-to-Network Efficiency: 122 Gbps loopback performance indicates minimal bottlenecks
System Integration: Unified memory architecture benefits network-intensive applications
Scalability: Architecture suitable for distributed computing scenarios

External Connectivity Assessment

While specific external network testing wasn't performed, the system's infrastructure suggests:

Support for high-speed Ethernet (2.5GbE+)
Low-latency interconnects suitable for cluster computing
Adequate bandwidth for data center deployment scenarios

Power Efficiency and Thermal Characteristics

Limited thermal data was available during testing:

Idle Temperature: 29°C (GPU sensor)
Idle Power: 8.059W (GPU subsystem)
Thermal Management: Appears well-controlled under light loads

The unified architecture's power efficiency represents a significant advantage over discrete GPU solutions, particularly for mobile and edge computing applications.

Competitive Analysis

Comparison with Intel Arc

Intel's Arc GPUs face similar software ecosystem challenges, though Intel has made more aggressive investments in AI software stack development. The Arc series benefits from Intel's deeper software engineering resources but still lags behind NVIDIA in AI framework support.

Comparison with NVIDIA

NVIDIA maintains a substantial advantage in:

Software Maturity: CUDA ecosystem is mature and well-supported
AI Framework Integration: Native support across all major frameworks
Developer Tools: Comprehensive profiling and debugging tools
Documentation: Extensive, well-maintained documentation

AMD's advantages include:

Open Source Approach: More flexible licensing and community development
Unified Memory: Simplified programming model for certain applications
Cost: Potentially more cost-effective solutions

Market Positioning

The AMD AI Max+ 395 occupies a unique position as a high-performance integrated solution, but software limitations significantly impact its competitiveness in AI-focused markets.

Use Case Suitability Analysis

Recommended Use Cases

General Computing: Excellent performance for traditional computational workloads
Development Platforms: Strong for general software development (non-AI)
Edge Computing: Unified architecture benefits power-constrained deployments
Future AI Workloads: When software ecosystem matures

Not Recommended For

Current AI Development: gfx1151 compatibility issues are blocking
Production AI Inference: Unreliable software support
Machine Learning Research: Limited framework compatibility
Time-Critical Projects: Uncertain timeline for software fixes

Large Language Model Performance and Stability

Ollama LLM Inference Testing

Testing with Ollama reveals a mixed picture for LLM inference on the AMD AI Max+ 395 system. The platform successfully runs various models through CPU-based inference, though GPU acceleration faces significant challenges.

Performance Metrics

Testing with various model sizes revealed the following performance characteristics:

GPT-OSS 20B Model Performance:

Prompt evaluation rate: 61.29 tokens/second
Text generation rate: 8.99 tokens/second
Total inference time: ~13 seconds for 117 tokens
Memory utilization: ~54 GB VRAM usage

Llama 4 (67B) Model:

Successfully loads and runs
Generation coherent and accurate

The system demonstrates adequate performance for smaller models (20B parameters and below) when running through Ollama, though performance significantly lags behind NVIDIA GPUs with proper CUDA acceleration. The large unified memory configuration (96 GB VRAM, deliberately maximized for this testing) allows loading of substantial models that would typically require multiple GPUs or extensive system RAM on other platforms. This conscious decision to allocate maximum memory to the GPU was specifically made to evaluate the system's potential for large language model workloads.

Critical Stability Issues with Large Models

Driver Crashes with Advanced AI Workloads

Testing revealed severe stability issues when attempting to run larger models or when using AI-accelerated development tools:

Affected Scenarios:

Large Model Loading: GPT-OSS 120B model causes immediate amdgpu driver crashes
AI Development Tools: Continue.dev with certain LLMs triggers GPU reset
OpenAI Codex Integration: Consistent driver failures with models exceeding 70B parameters

GPU Reset Events

System logs reveal frequent GPU reset events during AI workload attempts:

[ 1030.960155] amdgpu 0000:c5:00.0: amdgpu: GPU reset begin!
[ 1033.972213] amdgpu 0000:c5:00.0: amdgpu: MODE2 reset
[ 1034.002615] amdgpu 0000:c5:00.0: amdgpu: GPU reset succeeded, trying to resume
[ 1034.003141] [drm] VRAM is lost due to GPU reset!
[ 1034.037824] amdgpu 0000:c5:00.0: amdgpu: GPU reset(1) succeeded!

These crashes result in:

Complete loss of VRAM contents
Application termination
Potential system instability requiring reboot
Interrupted workflows and data loss

Root Cause Analysis

The driver instability appears to stem from the same underlying issue as the PyTorch/ROCm incompatibility: immature driver support for the gfx1151 architecture. The drivers struggle with:

Memory Management: Large model allocations exceed driver's tested parameters
Compute Dispatch: Complex kernel launches trigger unhandled edge cases
Power State Transitions: Rapid load changes cause driver state machine failures
Synchronization Issues: Multi-threaded inference workloads expose race conditions

Implications for AI Development

The combination of LLM testing results and driver stability issues reinforces that the AMD AI Max+ 395 system, despite impressive hardware specifications, remains unsuitable for production AI workloads. The platform shows promise for future AI applications once driver maturity improves, but current limitations include:

Unreliable Large Model Support: Models over 70B parameters risk system crashes
Limited Tool Compatibility: Popular AI development tools cause instability
Workflow Interruptions: Frequent crashes disrupt development productivity
Data Loss Risk: VRAM resets can lose unsaved work or model states

Future Outlook and Development Roadmap

Short-term Expectations (3-6 months)

ROCm updates likely to address gfx1151 compatibility
PyTorch/TensorFlow support should improve
Community-driven workarounds may emerge

Medium-term Prospects (6-18 months)

Full AI framework support expected
Optimization improvements for Strix Halo architecture
Better documentation and developer resources

Long-term Considerations (18+ months)

AMD's commitment to open-source ecosystem should pay dividends
Potential for superior price/performance ratios
Growing developer community around ROCm platform

Conclusions and Recommendations

The AMD AI Max+ 395 system represents impressive hardware engineering with its unified memory architecture, strong CPU performance, and substantial GPU compute capabilities. However, critical software ecosystem gaps, particularly the gfx1151 compatibility issues with PyTorch and ROCm, severely limit its immediate utility for AI and machine learning workloads.

Key Findings Summary

Hardware Strengths:

Excellent CPU performance with 16 Zen 5 cores
Innovative unified memory architecture with 96 GB addressable
Strong integrated GPU with 40 compute units
Efficient power management and thermal characteristics

Software Limitations:

Critical gfx1151 architecture support gaps in ROCm ecosystem
PyTorch integration completely broken for GPU acceleration
Limited AI framework compatibility across the board
Insufficient documentation for troubleshooting

Market Position:

Competitive hardware specifications
Unique integrated architecture advantages
Significant software ecosystem disadvantages versus NVIDIA
Uncertain timeline for compatibility improvements

Purchasing Recommendations

Buy If: - Primary use case is general computing or traditional HPC workloads - Willing to wait 6-12 months for AI software ecosystem maturity - Value open-source software development approach - Need power-efficient integrated solution

Avoid If:

Immediate AI/ML development requirements
Production AI inference deployments planned
Time-critical project timelines
Require guaranteed software support

Final Verdict

The AMD AI Max+ 395 system shows tremendous promise as a unified computing platform, but premature software ecosystem development makes it unsuitable for current AI workloads. Organizations should monitor ROCm development progress closely, as this hardware could become highly competitive once software support matures. For general computing applications, the system offers excellent performance and value, representing AMD's continued progress in processor design and integration.

The AMD AI Max+ 395 represents a glimpse into the future of integrated computing platforms, but early adopters should be prepared for software ecosystem growing pains. As AMD continues investing in ROCm development and the open-source community contributes solutions, this platform has the potential to become a compelling alternative to NVIDIA's ecosystem dominance.