TinyComputers.io (Posts about pine64)https://tinycomputers.io/categories/pine64.atom2024-03-28T02:59:50ZA.C. JokelaNikolaPine64 ROCKPro64 SATA Software RAID5https://tinycomputers.io/posts/pine64-rockpro64-sata-software-raid5.html2023-01-06T18:01:26-06:002023-01-06T18:01:26-06:00A.C. Jokela<p><img src="https://tinycomputers.io/images/rockpro64-nas-signal-2023-01-04-182040_003.jpeg.webp" style="zoom:33%; text-align: left; float: left; padding: 50px;" loading="lazy"></p>
<p>I love experimenting with all sorts of single board computers (SBCs) and systems on modules (SoMs) - like the Raspberry Pi CM4 or Pine64 SOQuartz. This extends even to needing to make a network-attached storage (NAS). One of the requirements is the ability to attached a bunch of disks to a tiny computer. The easiest way to accomplish this is using a PCIe SATA controller. To use this, you need a PCIe lane exposed. Luckily, there are a number of <a href="https://tinycomputers.io/posts/raspberry-pi-cm4-and-pin-compatible-modules.html">system on modules</a> with PCIe support. There is also Pine64 single board computers with an exposed PCIe lanes. These are the Quartz64 model A as well as the ROCKPro64. The former does not have the performance capabilities that the latter does, as such, we will be using the ROCKPro64.</p>
<p>My requirements for a NAS are bare-bones. I want <a href="https://en.wikipedia.org/wiki/Network_File_System">network file system</a> support; and secondarily, Windows' SMB support via the open source project <a href="https://www.samba.org/">Samba</a>. Samba is of secondary importance because the primary use case for this particular NAS is providing additional disk space to other SBCs that are running some flavor of Linux or BSD (like <a href="https://www.netbsd.org/">NetBSD</a>). </p>
<p>When the <a href="https://turingpi.com/turing-pi-v2-is-here/">Turing Pi 2</a> was being promoted on <a href="https://www.kickstarter.com/projects/turingpi/turing-pi-cluster-board">Kickstarter</a>, I had pledged to the project and then purchased a <a href="https://amzn.to/3X9edvw">2U shallow depth rack ITX server case</a>. The project has been moving along but is taking longer than I had anticipated. In the mean time, I decided to re-purpose the server case and use it for a simple NAS.</p>
<p>I purchased four <a href="https://amzn.to/3Ivf4T2">Seagate IronWolf 10TB drives</a>. These are spinning metal drives not fancy NVME drives. NVME is too cost prohibitive and would ultimately not be performative; e.g. the bottleneck would be the ROCKPro64.</p>
<p><img src="https://tinycomputers.io/images/jumpered-power-supply-IMG_0776.jpg.webp" style="zoom:25%; float: right; padding: 50px;" loading="lazy"></p>
<p>One of the four drives turned out to be a dud; there are only three in the above picture. The original goal was have 30TB of RAID5 -- 30TB of storage with 10TB for parity. But, because I did not want to spend much more on this project, I opted to return the broken drive and settle for 20TB of storage with 10TB of parity.</p>
<p>The setup is fairly simple. The 2U case, three 10TB drives, a ROCKPro64 4GB single board computer, and a <a href="https://amzn.to/3Ivf4T2">450W ATX power supply</a>. </p>
<p>Here's a pro-tip on using a power supply without a motherboard: while holding the power connector with the latching tab towards you, use a paper clip, or in my case, a piece of MIG welding wire, to short out the third and fourth connectors. This probably will void the warranty if something happens to the power supply.</p>
<p>Here is a bit of information on the <a href="https://github.com/ajokela/hwinfo/tree/main/single-board-computers/rockpro64">tech specs</a> of the ROCKPro64.</p>
<p>The drive setup is straight forward. Three drives, running in a RAID5 configuration producing a total of 20TB of usable storage. Setting up software RAID under Linux is simple. I used <a href="https://www.digitalocean.com/community/tutorials/how-to-create-raid-arrays-with-mdadm-on-ubuntu-22-04">this</a> as a foundation for setting up software RAID. We are using software RAID because hardware RAID controllers are expensive and have limited support with single board computers. The <a href="https://pipci.jeffgeerling.com/#sata-cards-and-storage">one that appears to work</a>, is about <a href="https://amzn.to/3ZlbP6E">$600</a>. It is not that I would not be against spending that much on a shiny new controller, it is that I do not feel there would be a noticeable benefit over a pure software solution. I will go into performance characteristics of the software RAID configuration later in this article.</p>
<h3>How To Create RAID5 Arrays with mdadm</h3>
<p><a href="https://en.wikipedia.org/wiki/Standard_RAID_levels#RAID_5">RAID5</a> has a requirement of at least three drives. As previously mentioned, this gives you <code>n - 1</code> drives of storage with one drive's worth of storage for parity. This parity is not stored on the single drive, it is stored across all of the drives but it is equal in total to the size of one drive. There is the assumption that all drives are of equal size.</p>
<p>Get a pretty list of all the available disk:</p>
<div class="code"><pre class="code literal-block">$<span class="w"> </span>lsblk<span class="w"> </span>-o<span class="w"> </span>NAME,SIZE,FSTYPE,TYPE,MOUNTPOINT
NAME<span class="w"> </span>SIZE<span class="w"> </span>FSTYPE<span class="w"> </span>TYPE<span class="w"> </span>MOUNTPOINT
sda<span class="w"> </span><span class="m">9</span>.1T<span class="w"> </span>disk<span class="w"> </span>
sdb<span class="w"> </span><span class="m">9</span>.1T<span class="w"> </span>disk<span class="w"> </span>
sdc<span class="w"> </span><span class="m">9</span>.1T<span class="w"> </span>disk<span class="w"> </span>
mtdblock0<span class="w"> </span>16M<span class="w"> </span>disk<span class="w"> </span>
mmcblk2<span class="w"> </span><span class="m">58</span>.2G<span class="w"> </span>disk<span class="w"> </span>
└─mmcblk2p1<span class="w"> </span><span class="m">57</span>.6G<span class="w"> </span>ext4<span class="w"> </span>part<span class="w"> </span>/
mmcblk2boot0<span class="w"> </span>4M<span class="w"> </span>disk<span class="w"> </span>
mmcblk2boot1<span class="w"> </span>4M<span class="w"> </span>disk<span class="w"> </span>
</pre></div>
<p>You will see we have three, "10TB" drives.</p>
<p>To create a RAID 5 array with the three 9.1T disks, pass them into the <code>mdadm --create</code> command. We will have to specify the device name you wish to create, the RAID level, and the number of devices. We will be naming the device <code>/dev/md0</code>, and include the disks that will build the array:</p>
<div class="code"><pre class="code literal-block">sudo<span class="w"> </span>mdadm<span class="w"> </span>--create<span class="w"> </span>--verbose<span class="w"> </span>/dev/md0<span class="w"> </span>--level<span class="o">=</span><span class="m">5</span><span class="w"> </span>--raid-devices<span class="o">=</span><span class="m">3</span><span class="w"> </span>/dev/sda<span class="w"> </span>/dev/sdb<span class="w"> </span>/dev/sdc
</pre></div>
<p>This will start to configure the array. <code>mdam</code> uses the recovery process to build the array. This process can and will take some time to complete, but the array can be used during this time. You can monitor the progress of the mirroring by checking the <code>/proc/mdstat</code> file:</p>
<div class="code"><pre class="code literal-block">$<span class="w"> </span>cat<span class="w"> </span>/proc/mdstat
Personalities<span class="w"> </span>:<span class="w"> </span><span class="o">[</span>raid6<span class="o">]</span><span class="w"> </span><span class="o">[</span>raid5<span class="o">]</span><span class="w"> </span><span class="o">[</span>raid4<span class="o">]</span><span class="w"> </span><span class="o">[</span>linear<span class="o">]</span><span class="w"> </span><span class="o">[</span>multipath<span class="o">]</span><span class="w"> </span><span class="o">[</span>raid0<span class="o">]</span><span class="w"> </span><span class="o">[</span>raid1<span class="o">]</span><span class="w"> </span><span class="o">[</span>raid10<span class="o">]</span>
md0<span class="w"> </span>:<span class="w"> </span>active<span class="w"> </span>raid5<span class="w"> </span>sda<span class="o">[</span><span class="m">0</span><span class="o">]</span><span class="w"> </span>sdc<span class="o">[</span><span class="m">3</span><span class="o">]</span><span class="w"> </span>sdb<span class="o">[</span><span class="m">1</span><span class="o">]</span>
<span class="w"> </span><span class="m">19532609536</span><span class="w"> </span>blocks<span class="w"> </span>super<span class="w"> </span><span class="m">1</span>.2<span class="w"> </span>level<span class="w"> </span><span class="m">5</span>,<span class="w"> </span>512k<span class="w"> </span>chunk,<span class="w"> </span>algorithm<span class="w"> </span><span class="m">2</span><span class="w"> </span><span class="o">[</span><span class="m">3</span>/3<span class="o">]</span><span class="w"> </span><span class="o">[</span>UUU<span class="o">]</span>
<span class="w"> </span>bitmap:<span class="w"> </span><span class="m">0</span>/73<span class="w"> </span>pages<span class="w"> </span><span class="o">[</span>0KB<span class="o">]</span>,<span class="w"> </span>65536KB<span class="w"> </span>chunk
unused<span class="w"> </span>devices:<span class="w"> </span><none>
</pre></div>
<p>For this 20TB to build, it took over 2000 minutes or about a day and half. Once the building is complete, your <code>/proc/mdstat</code> will look similar to the above.</p>
<p>Make a file system on our new array:</p>
<div class="code"><pre class="code literal-block">$<span class="w"> </span>sudo<span class="w"> </span>mkfs.ext4<span class="w"> </span>-F<span class="w"> </span>/dev/md0
</pre></div>
<p>Make a directory:</p>
<div class="code"><pre class="code literal-block">$<span class="w"> </span>sudo<span class="w"> </span>mkdir<span class="w"> </span>-p<span class="w"> </span>/mnt/md0
</pre></div>
<p>Mount the new array device</p>
<div class="code"><pre class="code literal-block">$<span class="w"> </span>sudo<span class="w"> </span>mount<span class="w"> </span>/dev/md0<span class="w"> </span>/mnt/md0
</pre></div>
<p>Check to see if the new array is available:</p>
<div class="code"><pre class="code literal-block">$<span class="w"> </span>df<span class="w"> </span>-h<span class="w"> </span>-x<span class="w"> </span>devtmpfs<span class="w"> </span>-x<span class="w"> </span>tmpfs
Filesystem<span class="w"> </span>Size<span class="w"> </span>Used<span class="w"> </span>Avail<span class="w"> </span>Use%<span class="w"> </span>Mounted<span class="w"> </span>on
/dev/mmcblk2p1<span class="w"> </span>57G<span class="w"> </span><span class="m">8</span>.7G<span class="w"> </span>48G<span class="w"> </span><span class="m">16</span>%<span class="w"> </span>/
/dev/md0<span class="w"> </span>19T<span class="w"> </span>80K<span class="w"> </span>18T<span class="w"> </span><span class="m">1</span>%<span class="w"> </span>/mnt/md0
</pre></div>
<p>Now, let's save our array's configuration:</p>
<div class="code"><pre class="code literal-block">$<span class="w"> </span>sudo<span class="w"> </span>mdadm<span class="w"> </span>--detail<span class="w"> </span>--scan<span class="w"> </span><span class="p">|</span><span class="w"> </span>sudo<span class="w"> </span>tee<span class="w"> </span>-a<span class="w"> </span>/etc/mdadm/mdadm.conf
</pre></div>
<p>Update <code>initramfs</code>so we can use our new array on boot:</p>
<div class="code"><pre class="code literal-block">$<span class="w"> </span>sudo<span class="w"> </span>update-initramfs<span class="w"> </span>-u
</pre></div>
<p>Add our device to <code>fstab</code>:</p>
<div class="code"><pre class="code literal-block">$<span class="w"> </span><span class="nb">echo</span><span class="w"> </span><span class="s1">'/dev/md0 /mnt/md0 ext4 defaults,nofail,discard 0 0'</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>sudo<span class="w"> </span>tee<span class="w"> </span>-a<span class="w"> </span>/etc/fstab
</pre></div>
<p>From some of the other documentation, there is a note that dumping the configuration to <code>/etc/mdadm/mdadm.conf</code> before the array is completely built, could result in the number of spare devices not being set correctly. This is what <code>mdadm.conf</code> looks like:</p>
<div class="code"><pre class="code literal-block">#<span class="w"> </span><span class="nv">mdadm</span>.<span class="nv">conf</span>
#
#<span class="w"> </span><span class="o">!</span><span class="nv">NB</span><span class="o">!</span><span class="w"> </span><span class="nv">Run</span><span class="w"> </span><span class="nv">update</span><span class="o">-</span><span class="nv">initramfs</span><span class="w"> </span><span class="o">-</span><span class="nv">u</span><span class="w"> </span><span class="nv">after</span><span class="w"> </span><span class="nv">updating</span><span class="w"> </span><span class="nv">this</span><span class="w"> </span><span class="nv">file</span>.
#<span class="w"> </span><span class="o">!</span><span class="nv">NB</span><span class="o">!</span><span class="w"> </span><span class="nv">This</span><span class="w"> </span><span class="nv">will</span><span class="w"> </span><span class="nv">ensure</span><span class="w"> </span><span class="nv">that</span><span class="w"> </span><span class="nv">initramfs</span><span class="w"> </span><span class="nv">has</span><span class="w"> </span><span class="nv">an</span><span class="w"> </span><span class="nv">uptodate</span><span class="w"> </span><span class="nv">copy</span>.
#
#<span class="w"> </span><span class="nv">Please</span><span class="w"> </span><span class="nv">refer</span><span class="w"> </span><span class="nv">to</span><span class="w"> </span><span class="nv">mdadm</span>.<span class="nv">conf</span><span class="ss">(</span><span class="mi">5</span><span class="ss">)</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="nv">information</span><span class="w"> </span><span class="nv">about</span><span class="w"> </span><span class="nv">this</span><span class="w"> </span><span class="nv">file</span>.
#
#<span class="w"> </span><span class="nv">by</span><span class="w"> </span><span class="nv">default</span><span class="w"> </span><span class="ss">(</span><span class="nv">built</span><span class="o">-</span><span class="nv">in</span><span class="ss">)</span>,<span class="w"> </span><span class="nv">scan</span><span class="w"> </span><span class="nv">all</span><span class="w"> </span><span class="nv">partitions</span><span class="w"> </span><span class="ss">(</span><span class="o">/</span><span class="nv">proc</span><span class="o">/</span><span class="nv">partitions</span><span class="ss">)</span><span class="w"> </span><span class="nv">and</span><span class="w"> </span><span class="nv">all</span>
#<span class="w"> </span><span class="nv">containers</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="nv">MD</span><span class="w"> </span><span class="nv">superblocks</span>.<span class="w"> </span><span class="nv">alternatively</span>,<span class="w"> </span><span class="nv">specify</span><span class="w"> </span><span class="nv">devices</span><span class="w"> </span><span class="nv">to</span><span class="w"> </span><span class="nv">scan</span>,<span class="w"> </span><span class="nv">using</span>
#<span class="w"> </span><span class="nv">wildcards</span><span class="w"> </span><span class="k">if</span><span class="w"> </span><span class="nv">desired</span>.
#<span class="nv">DEVICE</span><span class="w"> </span><span class="nv">partitions</span><span class="w"> </span><span class="nv">containers</span>
#<span class="w"> </span><span class="nv">automatically</span><span class="w"> </span><span class="nv">tag</span><span class="w"> </span><span class="nv">new</span><span class="w"> </span><span class="nv">arrays</span><span class="w"> </span><span class="nv">as</span><span class="w"> </span><span class="nv">belonging</span><span class="w"> </span><span class="nv">to</span><span class="w"> </span><span class="nv">the</span><span class="w"> </span><span class="nv">local</span><span class="w"> </span><span class="nv">system</span>
<span class="nv">HOMEHOST</span><span class="w"> </span><span class="o"><</span><span class="nv">system</span><span class="o">></span>
#<span class="w"> </span><span class="nv">instruct</span><span class="w"> </span><span class="nv">the</span><span class="w"> </span><span class="nv">monitoring</span><span class="w"> </span><span class="nv">daemon</span><span class="w"> </span><span class="nv">where</span><span class="w"> </span><span class="nv">to</span><span class="w"> </span><span class="k">send</span><span class="w"> </span><span class="nv">mail</span><span class="w"> </span><span class="nv">alerts</span>
<span class="nv">MAILADDR</span><span class="w"> </span><span class="nv">root</span>
#<span class="w"> </span><span class="nv">definitions</span><span class="w"> </span><span class="nv">of</span><span class="w"> </span><span class="nv">existing</span><span class="w"> </span><span class="nv">MD</span><span class="w"> </span><span class="nv">arrays</span>
#<span class="w"> </span><span class="nv">This</span><span class="w"> </span><span class="nv">configuration</span><span class="w"> </span><span class="nv">was</span><span class="w"> </span><span class="nv">auto</span><span class="o">-</span><span class="nv">generated</span><span class="w"> </span><span class="nv">on</span><span class="w"> </span><span class="nv">Thu</span>,<span class="w"> </span><span class="mi">22</span><span class="w"> </span><span class="nv">Dec</span><span class="w"> </span><span class="mi">2022</span><span class="w"> </span><span class="mi">16</span>:<span class="mi">46</span>:<span class="mi">51</span><span class="w"> </span><span class="o">-</span><span class="mi">0500</span><span class="w"> </span><span class="nv">by</span><span class="w"> </span><span class="nv">mkconf</span>
<span class="nv">ARRAY</span><span class="w"> </span><span class="o">/</span><span class="nv">dev</span><span class="o">/</span><span class="nv">md0</span><span class="w"> </span><span class="nv">metadata</span><span class="o">=</span><span class="mi">1</span>.<span class="mi">2</span><span class="w"> </span><span class="nv">spares</span><span class="o">=</span><span class="mi">1</span><span class="w"> </span><span class="nv">name</span><span class="o">=</span><span class="nv">k8s</span><span class="o">-</span><span class="nv">controlplane</span><span class="o">-</span><span class="mi">01</span>:<span class="mi">0</span><span class="w"> </span><span class="nv">UUID</span><span class="o">=</span><span class="mi">62</span><span class="nv">e3b0e3</span>:<span class="mi">3</span><span class="nv">cccb921</span>:<span class="mi">0</span><span class="nv">fc8e646</span>:<span class="nv">ac33bd0f</span>
</pre></div>
<p>Note <code>spares=1</code> is correct in the case of my set up.</p>
<p>By this point, if you have waited until the array has been successfully built - my 20TB array took about a day and half to complete.</p>
<h3>Performance</h3>
<p>We now have a shiny, new 20TB of available disk space, but what are the performance characteristics of a 20TB, Software RAID5?</p>
<p>We will be using <a href="https://www.iozone.org/">iozone</a>. Iozone has been around for decades; I first used it while in college and having a rack full of servers was all the rage -- before the creation of utility-scale cloud computing, as well as tiny computers.</p>
<p>Download iozone from <a href="https://tinycomputers.io/posts/iozone.org">iozone.org</a>. Extract and compile:</p>
<div class="code"><pre class="code literal-block">$<span class="w"> </span><span class="nb">cd</span><span class="w"> </span>iozone3_494/src/current
$<span class="w"> </span>make<span class="w"> </span>linux
</pre></div>
<p>You will end up with the executable <code>iozone</code> in the <code>src/current</code> directory.</p>
<p>Run <code>iozone</code>:</p>
<div class="code"><pre class="code literal-block">$<span class="w"> </span>sudo<span class="w"> </span>./iozone<span class="w"> </span>-a<span class="w"> </span>-b<span class="w"> </span>output.xls<span class="w"> </span>/mnt/md0
</pre></div>
<p>This will both output a lot of numbers as well as take a while to complete. When it is complete, you will have an Excel spreadsheet with the results in <code>output.xls</code>.</p>
<p>At this point, I am going to stop doing a step by step of what I have executing and running. Why, I am going to show some visualizations and analyses of our results. My steps will be roughly:</p>
<ol>
<li>Break out each block of metrics into its own csv file; for example, the first block found in <code>output.xls</code> would be saved in its own csv file; let's call it <code>writer.csv</code>; exclude the header <code>Writer Report</code>from the csv.
Repeat this step for all blocks of reports.</li>
</ol>
<pre>
Writer Report
4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384
64 703068 407457 653436 528618 566551
128 504196 306992 433727 962469 498114 757430
256 805021 475990 850282 594276 571198 582035 733527
512 660600 573916 439445 1319250 549959 645703 926116 591299
1024 1102176 1053512 610617 704230 902151 1326505 1011817 1161176 919928
2048 608964 1398145 1329751 822175 1140942 841094 1332432 1308682 1082427 1311879
4096 1066886 1304093 1168634 946820 1467135 881253 1360802 931521 1047309 1018923 1047054
8192 955344 1295186 1329052 1354035 1019915 1192806 1373082 1197294 1053501 866339 1116235 1368760
16384 1471798 1219970 2029709 1957942 2031269 1533341 1570127 1494752 1873291 1370365 1761324 1647601 1761143
32768 0 0 0 0 1948388 1734381 1389173 1315295 1848047 1916465 1944804 1646551 1632605
65536 0 0 0 0 1938246 1933611 1638071 1910004 1885763 1876212 1844374 1721776 1578535
131072 0 0 0 0 1969167 1962833 1921089 1757021 1644607 1780142 1869709 1566404 1356993
262144 0 0 0 0 2025197 2037129 2036955 1747487 1961757 1954913 1934085 1718841 1596327
524288 0 0 0 0 2041397 2080623 2087150 2049656 2007421 2005253 1930617 1876761 1813078
</pre>
<ol>
<li>I used a combination of using Excel (using LibreOffice Calc would work, too) and a <a href="https://jupyter.org/">Jupyter Notebook</a>. Excel was used to pull out each report's block and then save each to a CSV file; the <a href="https://tinycomputers.io/pages/rockpro64-armbian-raid5-iozone.html">Notebook</a> contains python code for reading in CSVs, and using <code>matplotlib</code> to produce visualizations.</li>
</ol>
<h3>Select Visualizations</h3>
<p><img alt="" src="https://tinycomputers.io/images/rockpro64-software-raid/Read-3D.png.webp"></p>
<p><img alt="" src="https://tinycomputers.io/images/rockpro64-software-raid/Read-heatmap.png.webp"></p>
<p>There actually is relatively consistent reads across the file sizes and transfer sizes.</p>
<p><img alt="" src="https://tinycomputers.io/images/rockpro64-software-raid/Write-3D.png.webp"></p>
<p><img alt="" src="https://tinycomputers.io/images/rockpro64-software-raid/Write-heatmap.png.webp"></p>
<script type="text/javascript">
amzn_assoc_placement = "adunit0";
amzn_assoc_tracking_id = "tinycompute05-20";
amzn_assoc_ad_mode = "search";
amzn_assoc_ad_type = "smart";
amzn_assoc_marketplace = "amazon";
amzn_assoc_region = "US";
amzn_assoc_title = "Goodies";
amzn_assoc_default_search_phrase = "seagate sata pcie";
amzn_assoc_default_category = "All";
amzn_assoc_linkid = "5fc86499de521ab9aacd2a8118914cfd";
amzn_assoc_rows = "4";
amzn_assoc_design = "text_links";
</script>
<script src="//z-na.amazon-adsystem.com/widgets/onejs?MarketPlace=US"></script>Raspberry Pi CM4 and Pin Compatible Moduleshttps://tinycomputers.io/posts/raspberry-pi-cm4-and-pin-compatible-modules.html2022-12-22T10:30:00-06:002022-12-22T10:30:00-06:00A.C. Jokela<div style="padding-bottom: 50px; padding-top: 50px;">
<p><span style="font-weight: bold;">UPDATE: 2023/02/04:</span> <a href="https://tinycomputers.io/posts/bigtreetech-cb1-review.html">Review of BIGTREETECH CB1 - RPi CM4 Pin Compatible Module</a>
</p>
</div>
<p>Hardware, in our modern era, does not exist in a vacuum; it requires software to function and be useful. One of main benefits of living within the Raspberry Pi ecosystem is you get up-to-date software that is maintained by a large network of source code contributors. Just for the <a href="https://github.com/raspberrypi/linux/graphs/contributors">linux kernel</a> used in Raspberry Pi OS, there have been over 5,000 people contributing to the project. That's hundreds of thousands of lines of code added, removed and modified. Raspberry Pi is successful because of its ecosystem. It is so large, it is self-sustaining. The Raspberry Pi Compute Module 4 (<em>announcement of the <a href="https://www.seeedstudio.com/blog/2020/10/19/new-raspberry-pi-compute-module-4-cm4-released-from-25-and-up/">"CM4"</a></em>) was introduced about two years ago. <a href="https://datasheets.raspberrypi.com/cm4/cm4-datasheet.pdf">Official Raspberry Pi CM4 Datasheet</a>. It is a followup to the wildly successful <a href="https://rpilocator.com/?cat=PI4" target="_blank">Raspberry Pi 4b</a>. The CM4 is a different form factor from the 4b. Unlike the 4b, it requires a <a href="https://amzn.to/3WpGv4u" target="_blank">carrier or IO board</a> to be useful. The good news is it is compatible with a <a href="https://pipci.jeffgeerling.com/boards_cm">bewildering array of carrier and IO boards</a>.</p>
<h3>Raspberry Pi CM4</h3>
<div style="width: 100%; text-align: center; padding-bottom: 1em;">
<img src="https://tinycomputers.io/images/rpi-cm4-1gb-signal-2022-12-19-162656.png.webp" style="width:45%; text-align:center; float:center; padding: 2px;" loading="lazy"><br>
Raspberry Pi Compute Module 4, 1GB memory
</div>
<div></div>
<p>There are IO boards that give you the same form factor as the RPi 4b, there are also IO boards that turn your CM4 into a KVM for a server management, there are boards with two ethernet ports -- allowing for the creation of a simple router. There also boards that expose the CM4's PCIe bus. This opens up the possibilities for using peripherals like addition network adapters or SATA controllers. More on that later.</p>
<p>Since the CM4's release, there have been a few pin compatible modules developed by other firms. By <em>pin compatible</em>, I mean that these other modules can correctly be attached via <a href="https://www.digikey.com/en/maker/projects/creating-a-raspberry-pi-compute-module-4-cm4-carrier-board-in-kicad/7812da347e5e409aa28d59ea2aaea490">Hirose mating connectors</a> to the IO boards.</p>
<p>One of the primary benefits of using a genuine Raspberry Pi CM4, as I mentioned in the first paragraph, is the ecosystem. The CM4 uses the same operating system as the 4b. This allows for nearly all the same software to be usable across the RPi family of single board computers. This sheds light on one of the most commonly brought up issues with non-Raspberry Pi single board computers: the software ecosystem just is not as robust as Raspberry Pi. This is not limited to the alternatives to the CM4. There are an array of alternatives to the RPi family, like the boards made by <a href="https://www.pine64.com/">Pine64</a>, or <a href="https://libre.computer/products/aml-s905x-cc/">Libre</a>, or <a href="https://www.hardkernel.com/">Hardkernel's</a> Odroid series. These all cannot run the official Raspberry Pi OS.</p>
<p>Jeff Geerling does a fantastic job of <a href="https://www.jeffgeerling.com/blog/2020/raspberry-pi-compute-module-4-review">reviewing the RPi CM4</a>. I am not going to give a complete, indepth review; Jeff has already done that.</p>
<p>Core Features:</p>
<ul>
<li>Optional eMMC, zero to 32GB</li>
<li>Optional Wireless (WiFi and Bluetooth)</li>
<li>Variety of memory sizes, 1GB to 8GB</li>
</ul>
<p>If by some chance you stumbled onto this post and you need assistance in getting Raspberry Pi OS running on a CM4 unit, check out <a href="https://www.jeffgeerling.com/blog/2020/how-flash-raspberry-pi-os-compute-module-4-emmc-usbboot">this</a>. I'm not going to go into details here; it is <a href="https://www.jeffgeerling.com/blog/2020/how-flash-raspberry-pi-os-compute-module-4-emmc-usbboot">a solved problem</a>.</p>
<p>Many of the alternatives to Raspberry Pi OS have a very similar feel and shallow curve for learning and setting up, but they are not 100% the same. Take for example, the the multi-board Linux distribution <a href="https://www.armbian.com/">Armbian</a>. Armbian supports over 160 different single board computers. If you have a well established board, there is a good chance there's an Armbian build for it. Armbian is very similar to RPi OS; they are both derivatives of <a href="https://www.debian.org/">Debian</a>, both can use standard Ubuntu and Debian packages, both have a similar method of writing a disk image to an SD card and booting the OS. There is no guarantee, however, that all software designed for the Raspberry Pi OS will run under Armbian. Particularly when dealing with third party shields and GPIO boards as well as things that I tend to ignore like video encoding/decoding and sound.</p>
<p>The common quip as of late goes something like this: <em>because of the shortage of Raspberry Pi computers, some people have turned to alternatives.</em> This might be the case for some, but this is not going to be my justification for using or testing out the three alternatives that will be present throughout the rest of this article.</p>
<p><strong>All Raspberry Pi single board computers and modules are in tight supply for the retail and hobbists markets. Check out <a href="https://rpilocator.com/">Raspberry Pi Locator</a> for places that might have supply. If you are willing to pay a significant premium, <a href="https://ebay.us/WCHlgs">eBay has quite a few available</a>.</strong></p>
<p>With the RPi CM4 having been covered extensively - like <a href="https://www.jeffgeerling.com/blog/2020/raspberry-pi-compute-module-4-review">Jeff Geerling's Review</a>; instead, I'll be looking at the remaining three modules.
<br><br></p>
<h4>Performance Metrics</h4>
<table style="width: 100%; text-align: center; border: thin dotted grey; padding: 2px;">
<tr style="text-align: center; border-bottom: thin dotted grey; margin: 2px;">
<th colspan="3" style="text-align: left; font-weight: bold; margin-left: 2px; padding-left: 10px;">
<a href="https://www.geekbench.com/" target="_blank">Geekbench Metrics</a>
</th>
</tr>
<tr>
<th>
Module
</th>
<th>
Single CPU Metrics
</th>
<th>
Multi-CPU Metrics
</th>
</tr>
<tr style="background-color: #F5F5F5;">
<td>
<a href="https://browser.geekbench.com/v5/cpu/19380000" target="_blank">Raspberry Pi CM4</a>
</td>
<td>228</td><td>644</td>
</tr>
<tr>
<td>
<a href="https://browser.geekbench.com/v5/cpu/19379584" target="_blank">Radxa CM3</a>
</td>
<td>163</td><td>508</td>
</tr>
<tr style="background-color: #F5F5F5;">
<td>
<a href="https://browser.geekbench.com/v5/cpu/19400478" target="_blank">Pine64 SOQuartz</a>
</td>
<td>156</td><td>491</td>
</tr>
<tr style="text-align: center;">
<td>
<a href="https://browser.geekbench.com/v5/cpu/19254559" target="_blank">Banana Pi CM4</a>
</td>
<td>295</td><td>1087</td>
</tr>
</table>
<div style="height: 3em;"></div>
<table style="width: 100%; text-align: center; border: thin dotted grey; padding: 2px;">
<tr style="text-align: center; border-bottom: thin dotted grey; margin: 2px;">
<th colspan="5" style="text-align: left; font-weight: bold; margin-left: 2px; padding-left: 10px;">
Features Comparison
</th>
</tr>
<tr style="border-bottom: thin dotted grey; text-align: center;">
<th></th>
<th><a href="https://www.raspberrypi.com/documentation/computers/compute-module.html" target="_blank">Raspberry Pi CM4</a></th>
<th><a href="https://wiki.radxa.com/Rock3/CM/CM3" target="_blank">Radxa CM3</a></th>
<th><a href="https://wiki.pine64.org/wiki/SOQuartz" target="_blank">Pine64 SOQuartz</a></th>
<th><a href="https://wiki.banana-pi.org/BPI-CM4_Computer_module_and_development_Kit" target="_blank">Banana Pi CM</a></th>
</tr>
<tr style="background-color: #F5F5F5;">
<td></td>
<td><a href="https://datasheets.raspberrypi.com/cm4/cm4-product-brief.pdf" target="_blank">Specifications</a></td>
<td><a href="https://dl.radxa.com/cm3/docs/radxa-cm3-product-brief_Revision_1.3_g0ecb072.pdf" target="_blank">Specifications</a></td>
<td><a href="https://files.pine64.org/doc/quartz64/SOQuartz_SOM_schematic_v1.1_20210816.pdf" target="_blank">Specifications</a></td>
<td><a href="https://wiki.banana-pi.org/BPI-CM4_Computer_module_and_development_Kit#Features" target="_blank">Specifications</a></td>
</tr>
<tr style="border-bottom: thin dotted grey;">
<td style="border-right: thin dotted grey;">Core</td>
<td>Broadcom BCM2711, Quad core Cortex-A72 (ARM v8) 64-bit SoC @ 1.5GHz</td>
<td>Rockchip RK3566, Quad core Cortex-A55 (ARM v8) 64-bit SoC @ 2.0GHz</td>
<td>Rockchip RK3566, Quad core Cortex-A55 (ARM v8) 64-bit SoC @ 1.8GHz and Embedded 32-bit RISC-V CPU</td>
<td>Amlogic A311D Quad core ARM Cortex-A73 and dual core ARM Cortex-A53 CPU</td>
</tr>
<tr style="border-bottom: thin dotted grey; background-color: #F5F5F5;">
<td style="border-right: thin dotted grey;">NPU</td>
<td>-</td>
<td>0.8T NPU</td>
<td>0.8 TOPS Neural Network Acceleration Engine</td>
<td>5.0 TOPS</td>
</tr>
<tr style="border-bottom: thin dotted grey;">
<td style="border-right: thin dotted grey;">GPU</td>
<td>-</td>
<td>Mali G52 GPU</td>
<td>Mali-G52 2EE Bifrost GPU</td>
<td>Arm Mali-G52 MP4 (6EE) GPU</td>
</tr>
<tr style="border-bottom: thin dotted grey; background-color: #F5F5F5;">
<td style="border-right: thin dotted grey;">Memory</td>
<td>1GB, 2GB, 4GB or 8GB LPDDR4</td>
<td>1GB, 2GB, 4GB or 8GB LPDDR4</td>
<td>2GB, 4GB, 8GB LPDDR4</td>
<td>4GB LPDDR4</td>
</tr>
<tr style="border-bottom: thin dotted grey;">
<td style="border-right: thin dotted grey;">eMMC</td>
<td>On module - 0GB to 32GB</td>
<td>On module - 0GB to 128GB</td>
<td>External - 16GB to 128GB</td>
<td>On module - 16GB to 128G)</td>
</tr>
<tr style="border-bottom: thin dotted grey; background-color: #F5F5F5;">
<td style="border-right: thin dotted grey;">Network</td>
<td>1Gbit Ethernet - Option for WiFi5 with Bluetooth 5.0</td>
<td>1Gbit Ethernet - Option for WiFi5 with Bluetooth 5.0</td>
<td>1Gbit Ethernet - WiFi 802.11 b/g/n/ac with Bluetooth 5.0</td>
<td>1Gbit Ethernet</td>
</tr>
<tr style="border-bottom: thin dotted grey;">
<td style="border-right: thin dotted grey;">PCIe</td>
<td>1-lane</td>
<td>1-lane</td>
<td>1-lane</td>
<td>1-lane</td>
</tr>
<tr style="border-bottom: thin dotted grey; background-color: #F5F5F5;">
<td style="border-right: thin dotted grey;">HDMI</td>
<td>2x HDMI</td>
<td>1x HDMI</td>
<td>1x HDMI</td>
<td>1x HDMI</td>
</tr>
<tr style="border-bottom: thin dotted grey;">
<td style="border-right: thin dotted grey;">GPIO</td>
<td>28 pin</td>
<td><a href="https://wiki.radxa.com/Rock3/CM3/IO/GPIO" target="_blank">40 pin</a></td>
<td>28 pin</td>
<td>26 pin</td>
</tr>
<tr style="border-bottom: thin dotted grey; background-color: #F5F5F5;">
<td style="border-right: thin dotted grey;">Extras</td>
<td>-</td>
<td>-</td>
<td>-</td>
<td>SATA ports, one shared with USB 3, one shared with PCIe; Audio Codec</td>
</tr>
<tr style="border-bottom: thin dotted grey;">
<td style="border-right: thin dotted grey;">Geekbench Score - Single CPU</td>
<td>228</td>
<td>163</td>
<td>156</td>
<td>295</td>
</tr>
<tr style="border-bottom: thin dotted grey; background-color: #F5F5F5;">
<td style="border-right: thin dotted grey;">Geekbench Score - Multi CPU</td>
<td>644</td>
<td>508</td>
<td>491</td>
<td>1087</td>
</tr>
<tr style="border-bottom: thin dotted grey;">
<td style="border-right: thin dotted grey;">Price of Tested*</td>
<td>$65</td>
<td>$69</td>
<td>$49</td>
<td>$105</td>
</tr>
<tr style="border-bottom: thin dotted grey; background-color: #F5F5F5;">
<td style="border-right: thin dotted grey;">Power Consumption</td>
<td>7 watts</td>
<td>N/A</td>
<td>2 watts</td>
<td>N/A</td>
</tr>
</table>
<div>
* Prices exclude shipping
</div>
<div style="height: 3em;"></div>
<h3>Pine64 SOQuartz</h3>
<div style="width: 100%; text-align: center; padding-bottom: 1em;">
<img src="https://tinycomputers.io/images/signal-2022-11-30-202052_004-SOQuartz-module.png.webp" style="width:45%; text-align:center; float:center; padding: 2px;" loading="lazy"><br>
Pine64 SOQuartz Module, 4GB memory
</div>
<p>For whatever reason, I really like <a href="https://wiki.pine64.org/wiki/SOQuartz">Pine64's SOQuartz module</a>. It is by far the least performant of the four compute modules I have tried. It has a wonky antenna and needs a far from mainstream variety of Linux to be useful. There are two Linux distributions available: <a href="https://dietpi.com/">DietPI</a> and <a href="https://github.com/Plebian-Linux/quartz64-images">Plebian Linux</a>. I settled upon using Plebian. I would have gone with DietPi but my initial use case of making a two ethernet router using a <a href="https://amzn.to/3PIkU5h">Waveshare Dual Gigabit Ethernet Base Board Designed for Raspberry Pi Compute Module 4</a>, but I was unable to get both ethernet ports working. Plebian was simpler. For those interested in trying Plebian, you can download recent disk images by going to <a href="https://github.com/Plebian-Linux/quartz64-images/actions">Plebian Linux's Github Actions</a>, and select one of the recent "Build Quartz64 Images"; at the bottom there will be zipped disk image Artifacts to download for the various flavors of Quartz64. Plebian is a bit rough around the edges. It is derived from Debian Testing (currently codenamed <em>bookworm</em>) and runs a release candidate Linux Kernel. Its developer, <a href="https://github.com/CounterPillow">CounterPillow</a>, also states that "<strong><em>This is a work-in-progress project. Things don't work yet. Please do not flash these images yet unless you are trying to help develop this pipeline.</em></strong>" The interactions with the system feel similar to that of <a href="https://www.netbsd.org/">NetBSD</a> from the early-2010s. It is not to say it is not a modern flavor of Linux, it is simply lacking some of the usual expectations. You want your network interfaces to be named <code>eth0</code>? How about no. Interfaces have not been aliased, if you can get WiFi drivers working, you will end up with a device named something like <code>wlxe84e069541e6</code> instead of <code>wlan0</code>. Given that it is running a testing branch of Debian, things like docker and the like will likely not work without some significant wrangling. </p>
<p>Why do I like this compute module? I like Pine64's products. I like the community that has grown up around the products. In the course of trying to get an operating system up and running, I had numerous questions that I asked on <a href="https://discord.gg/pine64">Pine64's Discord Server</a>. Everyone was extremely helpful and despite my own feelings that some of my questions were simplistic, no one expressed that sentiment. There were no massive egos to speak of.</p>
<p>Core Features:</p>
<ul>
<li>Variety of memory options: 2gb to 8gb</li>
<li>external eMMC module support: 8gb to 128gb</li>
<li>Wifi</li>
</ul>
<p>Getting Plebian running on a SOQuartz module is straightforward; write the appropriate image to an eMMC module, attach the eMMC onto the SOQuartz and place it into a carrier or IO board. You should get working HDMI, one ethernet port, along with USB working. A quick run down of the steps are as follows:</p>
<ol>
<li>
<p>Get a USB to eMMC adapter; Pine64 has <a href="https://pine64.com/product/usb-adapter-for-emmc-module/">one available</a>; you could also try <a href="https://www.ebay.com/itm/284891437420?hash=item4254da596c:g:kY0AAOSwllNiyutf&amdata=enc%3AAQAHAAAA4HwIWh9DOlbgf8Pj6hjQy1odilvD6hb%2FcoV2KHnlBbof6or4iiyXiycys5fN2xKETBm8nKX2KuMRzW5CigAY4jZuzsSE7aAsgJYIrQhX2BIxA4YXkSIApnWqLJy1TrKrL%2F0PBa%2BcWe3Y3l3VR1XFMtifa0q8QE3BTHxTDLwSNlHPe%2Fep49NfMEHEUXc9Hzy7ki8D%2BGtvAVyKT%2BI%2BrlkWRN7GUrseWkw27jHgAF2OudN32Nk8Yx7l1lRwixmPrBRK89zwfieZ3AaOjXY79MitKHi8CyfsL5RwpBi%2BNiiRRtTC%7Ctkp%3ABFBMht3M26Vh">eBay</a>; you may need to get a micro SD to USB adapter, too.</p>
</li>
<li>
<p>Get an eMMC module. Pine64 has <a href="https://pine64.com/product/16gb-emmc-module/">a few available</a></p>
</li>
<li>
<p>Obvious step: connect your eMMC to your USB to eMMC adapter and then connect that to your desktop/laptop/etc.</p>
</li>
<li>
<p>Download a SOQuartz Plebian Linux disk image from <a href="https://github.com/Plebian-Linux/quartz64-images/actions">Plebian Linux's Github Actions</a></p>
</li>
<li>
<p>Download<code>SOQuartz CM4 IO Board Image</code></p>
</li>
<li>Unzip the contents</li>
<li>
<p>You will end up with a file called <code>plebian-debian-bookworm-soquartz-cm4.img.xz</code>;</p>
</li>
<li>
<p>Write to your eMMC module. You could use something <a href="https://www.balena.io/etcher/">balena Etcher</a> or, if you're command-line-comfortable, use <code>dd</code></p>
</li>
<li>
<p>balena Etcher will take care of decompressing <code>plebian-debian-bookworm-soquartz-cm4.img.xz</code></p>
</li>
<li>
<p>using <code>dd</code>, you can do something like this:</p>
<p><code>bash
sudo xzcat plebian-debian-bookworm-soquartz-cm4.img.xz | sudo dd of=/dev/mmcblk1 status=progress</code></p>
<p>where <code>/dev/mmcblk1</code> is the correct device for your USB to eMMC adapter.</p>
<p><img alt="" src="https://tinycomputers.io/images/2022-12-19%2022-28-59.png.webp"></p>
</li>
<li>
<p>Attach your eMMC module to your SOQuartz module and attach the module to an IO or carrier board.</p>
</li>
<li>
<p>Attach peripherals and apply power. You'll eventually get presented with a prompt to set the password for the user <code>pleb</code></p>
</li>
</ol>
<p>If would be more cost effective if you were to buy a SOQuartz module, an USB to eMMC adapter and an eMMC module all at once; for orders being shipped to the United States, it is roughly a $9 flat rate.</p>
<p>Finally, if you really feel like going for an alternative to Linux, <a href="https://www.netbsd.org/">NetBSD</a> will also work on the SOQuartz, but it is more complicated. You will need to download a <a href="https://nycdn.netbsd.org/pub/arm/#HEAD">Generic 64bit</a> image from under the NetBSD-daily HEAD tab. This will need to be written to an eMMC module. Next, you will need to write the appropriate <a href="https://github.com/jaredmcneill/quartz64_uefi/releases">UEFI image</a> to an SD card from Jared McNeill's port of <a href="https://github.com/tianocore/edk2-platforms">Tianocore</a> to the Quartz64 family. UEFI and the disk image cannot exist on the same media. </p>
<p><strong>Pine64 sells <a href="https://pine64.com/product/soquartz-4gb-compute-module-w/">SOQuartz modules</a> directly from their site. The modules I have purchased and used are the 4gb models. They are about $50 excluding shipping.</strong></p>
<h3>Radxa CM3</h3>
<div style="width: 100%; text-align: center; padding-bottom: 1em;">
<img src="https://tinycomputers.io/images/RADXA-CM3-IMG_0733.png.webp" style="width:45%; text-align:center; float:center; padding: 2px;" loading="lazy"><br>
Radxa CM3, 4GB memory, without heatsinks
</div>
<p>As far as performance goes, the <a href="https://shop.allnetchina.cn/products/rock3-computing-module?variant=39486827069542">Radxa CM3</a> is just above Pine64's SOQuartz module but below Raspberry Pi CM4. Radxa is better known for its <a href="https://wiki.radxa.com/Rock3">Rock3</a> and <a href="https://wiki.radxa.com/Rock5">Rock5</a> series of single board computers; available from <a href="https://shop.allnetchina.cn/collections/rock5-model-b">ALLNET.China</a> and <a href="https://ebay.us/CQIZjw">eBay</a>. The CM3 is in the Rock3 series of boards and modules. The series features Rockchip RK3566/RK3568 processors, the RK3566 also is used in Pine64's Quartz64 and SOQuartz boards. Even though the module will function without issue on a carrier or IO board designed for Raspberry Pi CM4, the <a href="https://shop.allnetchina.cn/products/radxa-cm3-io-board?_pos=1&_sid=77ee539b1&_ss=r&variant=39531550867558">CM3 IO board</a> by Radxa exposes two SATA ports in addition to the PCIe 1x lane. The CM3 has an <a href="https://wiki.radxa.com/Rock3/Debian"><em>official</em> Debian</a> and <a href="https://wiki.radxa.com/Rock3/Ubuntu">Ubuntu</a> distributions, but like all the other compute modules, these are artisanally crafted specifically for the CM3. That means, you can not take an actual-official Debian or Ubuntu disk image for <a href="https://wiki.debian.org/Arm64Port">Arm64</a> and have it just have it work. Radxa does, however, maintain an up-to-date Github build pipeline for producing both <a href="https://github.com/radxa/debos-radxa/releases">Debian and Ubuntu images</a> for the CM3. Like the operating system's need to be different, so is the eMMC - it is not flashed in a typical manner - like what you would expect from a Raspberry Pi CM4 or even the Pine64 SOQuartz. In order to install Linux on the onboard eMMC, you need to use tools provided by Rockchip. The <a href="https://wiki.radxa.com/Rock3/installusb-install-radxa-cm3-io#Step_2:_Download_RK356X_Loader">Radxa Wiki page for CM3</a> is a good place to start. The CM3 and its installation process are about as far from Raspberry Pi CM4 territory as you will deal with for the modules presented in this article. The following instructions are available from <a href="https://wiki.radxa.om/">wiki.radxa.com</a> but they are found across a disparate set of pages; some describing the rockchip tools with references to disk images but with no clear and convenient place to download the files. This is an attempt to streamline the process. Let's get at it!</p>
<p>Core Features:</p>
<ul>
<li>On-module eMMC of 16 to 128GB</li>
<li>Two SATA (when using the appropriate IO board)</li>
</ul>
<p>The Rockchip tools are available on Windows as well as macOS/Linux. Downloading, compiling and running the macOS/Linux tool is straightforward; Windows involves a set of drivers and an executable tool.</p>
<h5>Linux/macOS</h5>
<ul>
<li>Install necessary USB and <code>autoconf</code> packages</li>
</ul>
<p><code>sudo apt-get install libudev-dev libusb-1.0-0-dev dh-autoreconf pkg-config libusb-1.0</code></p>
<ul>
<li>Clone the <a href="https://github.com/rockchip-linux/rkdeveloptool">github</a> repository</li>
</ul>
<div class="code"><pre class="code literal-block"><span class="n">git</span><span class="w"> </span><span class="n">clone</span><span class="w"> </span><span class="n">https</span><span class="p">:</span><span class="o">//</span><span class="n">github</span><span class="o">.</span><span class="n">com</span><span class="o">/</span><span class="n">rockchip</span><span class="o">-</span><span class="n">linux</span><span class="o">/</span><span class="n">rkdeveloptool</span>
<span class="n">cd</span><span class="w"> </span><span class="n">rkdeveloptool</span>
<span class="n">autoreconf</span><span class="w"> </span><span class="o">-</span><span class="n">i</span>
<span class="o">./</span><span class="n">configure</span>
<span class="n">make</span>
<span class="n">sudo</span><span class="w"> </span><span class="n">make</span><span class="w"> </span><span class="n">install</span>
</pre></div>
<ul>
<li>Run <code>rkdeveloptool --help</code>to verify it is installed</li>
</ul>
<h5>Windows</h5>
<ol>
<li>Download <a href="https://dl.radxa.com/tools/windows/RKDevTool_Release_v2.86.zip">RKDevTool</a></li>
<li>Download <a href="https://dl.radxa.com/tools/windows/DriverAssitant_v5.0.zip">RKDriverAssistant</a></li>
<li>Unzip and execute RKDriverAssistant (<code>DriverInstall.exe</code>)</li>
<li>Unzip RKDevTool</li>
<li>Before executing the tool, you will want to change the language to English; change <code>Chinese.ini</code> to <code>Englist.ini</code></li>
</ol>
<p><img alt="" src="https://tinycomputers.io/images/rkdevtools-set-english-config.ini.png.webp"></p>
<p>There is an assumption of using a <a href="https://amzn.to/3PIyaqA">Raspberry Pi CM4 IO Board</a>.</p>
<h5>Boot into maskrom mode</h5>
<ol>
<li>Unplug the board and remove any SD card</li>
<li>Plug a micro USB to USB Type-A cable into the micro USB port on the IO board. The other end of the cable gets plugged into your desktop or laptop. My laptop only has USB-C, so I had to use an <a href="https://amzn.to/3v6EJK6">adapter</a></li>
<li>On the CM3, there is a very tiny golden button; while pressing this, plug the power back in on the IO board</li>
<li>After a few seconds, you can stop pressing the button</li>
<li>Check for a USB device</li>
<li>Linux/macOS should show <code>Bus 001 Device 112: ID 2207:350a Fuzhou Rockchip Electronics Company</code></li>
<li>Windows, you will need to run <code>RKDevTool</code>; the status at the bottom of the application should read <code>maskrom mode</code></li>
</ol>
<div style="width: 100%; text-align: center; padding-bottom: 1em;">
<img src="https://tinycomputers.io/images/radxa-cm3.png.webp" style="width:45%; text-align:center; float:center; padding: 2px;" loading="lazy"><br>
maskrom button
</div>
<h5>Flashing/Writing a Disk Image</h5>
<p>You will need to download two files:</p>
<ol>
<li><a href="https://dl.radxa.com/rock3/images/loader/rk356x_spl_loader_ddr1056_v1.06.110.bin">rk356x_spl_loader_ddr1056_v1.06.110.bin</a></li>
<li>A Radax CM3 disk image from <a href="https://wiki.radxa.com/Rock3/downloads">https://wiki.radxa.com/Rock3/downloads</a> or <a href="https://github.com/radxa-build/radxa-cm3-io/releases/latest">https://github.com/radxa-build/radxa-cm3-io/releases/latest</a> or this <a href="https://ajokela.github.io/aws-s3-bucket-browser/index.html?bucket=https://s3.us-east-1.amazonaws.com/cdn.tinycomputers.io/index.html#radxa-rock3/">mirror</a></li>
</ol>
<p>We will be using <a href="https://github.com/radxa-build/radxa-cm3-io/releases/download/20221101-0118/radxa-cm3-io-ubuntu-focal-server-arm64-20221101-0254-gpt.img.xz">radxa-cm3-io-ubuntu-focal-server-arm64-20221101-0254-gpt.img.xz</a>; it is advisable to follow <a href="https://github.com/radxa-build/radxa-cm3-io/releases/latest">this</a> and download a more recent disk image.</p>
<h5>Linux Flashing</h5>
<div class="code"><pre class="code literal-block"><span class="n">rkdeveloptool</span><span class="w"> </span><span class="n">ld</span>
</pre></div>
<p><code>DevNo=1 Vid=0x2207,Pid=0x350a,LocationID=104 Maskrom</code></p>
<div class="code"><pre class="code literal-block"><span class="n">rkdeveloptool</span><span class="w"> </span><span class="n">db</span><span class="w"> </span><span class="n">rk356x_spl_loader_ddr1056_v1</span><span class="o">.</span><span class="mf">06.110</span><span class="o">.</span><span class="n">bin</span>
<span class="n">rkdeveloptool</span><span class="w"> </span><span class="n">wl</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="n">radxa</span><span class="o">-</span><span class="n">cm3</span><span class="o">-</span><span class="n">io</span><span class="o">-</span><span class="n">ubuntu</span><span class="o">-</span><span class="n">focal</span><span class="o">-</span><span class="n">server</span><span class="o">-</span><span class="n">arm64</span><span class="o">-</span><span class="mi">20221101</span><span class="o">-</span><span class="mi">0254</span><span class="o">-</span><span class="n">gpt</span><span class="o">.</span><span class="n">img</span><span class="o">.</span><span class="n">xz</span>
</pre></div>
<p>Reboot CM3</p>
<div class="code"><pre class="code literal-block"><span class="n">rkdeveloptool</span><span class="w"> </span><span class="n">rd</span>
</pre></div>
<h5>Windows Flashing</h5>
<p><img alt="" src="https://tinycomputers.io/images/RKDevTool-complete-english.png.webp"></p>
<p>You will need to specify a <code>loader</code> as well as an <code>image</code>. In the table on the left side of the screenshot, click in right-most the rectangle of the first row. This should bring up a file dialog box. Navigate to where you downloaded <code>rk356x_spl_loader_ddr1056_v1.06.110.bin</code>. Likewise for the second row (<code>image</code>), navigate to where you downloaded <code>radxa-cm3-io-ubuntu-focal-server-arm64-20221101-0254-gpt.img.xz</code></p>
<p>Click <code>Run</code></p>
<p>This operation will take several minutes; be patient.</p>
<p>The CM3 should automatically boot and bring you to a login prompt. <strong>The default user is <code>rock</code> with a password of <code>rock</code></strong></p>
<iframe width="100%" height="480" src="https://www.youtube-nocookie.com/embed/49zG4EZiJKo" title="Radxa CM3 Linux Booting" frameborder="0" loading="lazy" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
<p><strong>The Radxa CM3 can be purchased from <a href="https://shop.allnetchina.cn/products/rock3-computing-module?variant=39486827069542">ALLNET.China</a> for about $70 excluding shipping.</strong></p>
<h3>Banana Pi CM4</h3>
<div style="width: 100%; text-align: center; padding-bottom: 1em;">
<img src="https://tinycomputers.io/images/BPi-CM4-IMG_0754.png" style="width:45%; text-align:center; float:center; padding: 2px;" loading="lazy"><br>
Banana Pi CM4, 4GB memory
</div>
<p>Looking at the Geekbench table (above), you will notice at the Banana Pi CM4 seriously outperforms the other three modules I have tested. It is also the most expensive module - including shipping - it was about $120. This was not an inflated Raspberry Pi price, this is directly from <a href="http://www.sinovoip.com.cn/eindex.asp">Sinovoip</a>, the company behind the Banana Pi family of single board computers. But, before you start searching for where you can buy one, as of the time of this writing, I purchased Sinovoip's last module that they had allocated to developers and testers; and they have not started to commercially produce any, yet.</p>
<p><img src="https://tinycomputers.io/images/sinovoip-banana-pi-correspondence.png.webp" style="zoom:45%;" loading="lazy"></p>
<p>Core Features:</p>
<ul>
<li>4 x ARM Cortex-A73 CPU cores</li>
<li>
<p>2 x ARM Cortex-A53 CPU cores</p>
</li>
<li>
<p>4GB of memory</p>
</li>
</ul>
<p>Like the Radxa CM3, operating system software is very limited. For very detail instructions on install an operating system, in this case Android, check out <a href="https://wiki.banana-pi.org/Getting_Started_with_CM4">https://wiki.banana-pi.org/Getting_Started_with_CM4</a>.</p>
<p>Installing and boot Linux is fairly straight forward. You can either boot from an SD card, or you can choose to boot from the on-board eMMC module. That said, nonetheless, you will need an SD card.</p>
<p>Head over to <a href="https://wiki.banana-pi.org/Banana_Pi_BPI-M2S#Linux">https://wiki.banana-pi.org/Banana_Pi_BPI-M2S#Linux</a>, and you find a similar table of distributions images:</p>
<h5>Distributions</h5>
<h6>Ubuntu</h6>
<ul>
<li>2022-06-20-ubuntu-20.04-mate-desktop-bpi-m2s-aarch64-sd-emmc.img.zip
Baidu Cloud: https://pan.baidu.com/s/1kRukI-H-xliNqIqVacXWRw?pwd=8888 (pincode:8888)
Google drive: https://drive.google.com/file/d/1P2YQUwdrREdiwidr8YtCvOdMmwLPerVu/view</li>
</ul>
<p>S3 Mirror: https://s3.us-east-1.amazonaws.com/cdn.tinycomputers.io/banana-pi-m2s-cm4-linux/2022-06-20-ubuntu-20.04-mate-desktop-bpi-m2s-aarch64-sd-emmc.img.zip
MD5:2945f225eadba1b350cd49f47817c0cd</p>
<ul>
<li>2022-06-20-ubuntu-20.04-server-bpi-m2s-aarch64-sd-emmc.img.zip
Baidu Cloud:https://pan.baidu.com/s/1UoYR0k9YH9SE_A-MpqZ2fg?pwd=8888 (pincode: 8888)
Google Drive:https://drive.google.com/file/d/1y0DUVDhLyhw_C7p6SD2q1EjOZLEV_c_w/view
S3 Mirror: https://s3.us-east-1.amazonaws.com/cdn.tinycomputers.io/banana-pi-m2s-cm4-linux/2022-06-20-ubuntu-20.04-server-bpi-m2s-aarch64-sd-emmc.img.zip
MD5:9b17a00cbc17c46e414a906e659e7ca2</li>
</ul>
<h6>Debian</h6>
<ul>
<li>2022-06-20-debian-10-buster-bpi-m2s-aarch64-sd-emmc.img.zip
Baidu Cloud: https://pan.baidu.com/s/1TTsdyy5I7HLWS_Tptg7r2w?pwd=8888 (pincode: 8888)
Google Drive:https://drive.google.com/file/d/116ZydpggYpZ1WoSyVsc4QuchdIa3vGyI/view</li>
</ul>
<p>S3 Mirror: https://s3.us-east-1.amazonaws.com/cdn.tinycomputers.io/banana-pi-m2s-cm4-linux/2022-06-20-debian-10-buster-bpi-m2s-aarch64-sd-emmc.img.zip
MD5:9d39558ad37e5da47d7d144c8afec45e</p>
<h5>Flashing/Writing Images</h5>
<p>Let's assume we are using <code>2022-06-20-debian-10-buster-bpi-m2s-aarch64-sd-emmc.img.zip</code>; the handiest thing to start out with is making a bootable sd card. On your laptop or desktop computer, and assuming you are using a flavor Linux, issue the following at a command line:</p>
<div class="code"><pre class="code literal-block"><span class="nv">unzip</span><span class="w"> </span><span class="mi">2022</span><span class="o">-</span><span class="mi">06</span><span class="o">-</span><span class="mi">20</span><span class="o">-</span><span class="nv">debian</span><span class="o">-</span><span class="mi">10</span><span class="o">-</span><span class="nv">buster</span><span class="o">-</span><span class="nv">bpi</span><span class="o">-</span><span class="nv">m2s</span><span class="o">-</span><span class="nv">aarch64</span><span class="o">-</span><span class="nv">sd</span><span class="o">-</span><span class="nv">emmc</span>.<span class="nv">img</span>.<span class="nv">zip</span>
<span class="nv">dd</span><span class="w"> </span><span class="k">if</span><span class="o">=</span><span class="mi">2022</span><span class="o">-</span><span class="mi">06</span><span class="o">-</span><span class="mi">20</span><span class="o">-</span><span class="nv">debian</span><span class="o">-</span><span class="mi">10</span><span class="o">-</span><span class="nv">buster</span><span class="o">-</span><span class="nv">bpi</span><span class="o">-</span><span class="nv">m2s</span><span class="o">-</span><span class="nv">aarch64</span><span class="o">-</span><span class="nv">sd</span><span class="o">-</span><span class="nv">emmc</span>.<span class="nv">img</span><span class="w"> </span><span class="nv">of</span><span class="o">=/</span><span class="nv">dev</span><span class="o">/</span><span class="nv">sda0</span>
</pre></div>
<p>Change <code>sda0</code> to the appropriate device.</p>
<p>Instead the sd card into the IO board, and apply power to the board. That's it for booting from an SD card. In order to boot from eMMC, you will need to follow the above steps, but instead of downloading and writing the image from your laptop or desktop, you will be using the BPI CM4 instead. Download and unzip the image file:</p>
<div class="code"><pre class="code literal-block"><span class="nv">unzip</span><span class="w"> </span><span class="mi">2022</span><span class="o">-</span><span class="mi">06</span><span class="o">-</span><span class="mi">20</span><span class="o">-</span><span class="nv">debian</span><span class="o">-</span><span class="mi">10</span><span class="o">-</span><span class="nv">buster</span><span class="o">-</span><span class="nv">bpi</span><span class="o">-</span><span class="nv">m2s</span><span class="o">-</span><span class="nv">aarch64</span><span class="o">-</span><span class="nv">sd</span><span class="o">-</span><span class="nv">emmc</span>.<span class="nv">img</span>.<span class="nv">zip</span>
<span class="nv">dd</span><span class="w"> </span><span class="k">if</span><span class="o">=</span><span class="mi">2022</span><span class="o">-</span><span class="mi">06</span><span class="o">-</span><span class="mi">20</span><span class="o">-</span><span class="nv">debian</span><span class="o">-</span><span class="mi">10</span><span class="o">-</span><span class="nv">buster</span><span class="o">-</span><span class="nv">bpi</span><span class="o">-</span><span class="nv">m2s</span><span class="o">-</span><span class="nv">aarch64</span><span class="o">-</span><span class="nv">sd</span><span class="o">-</span><span class="nv">emmc</span>.<span class="nv">img</span><span class="w"> </span><span class="nv">of</span><span class="o">=/</span><span class="nv">dev</span><span class="o">/</span><span class="nv">mmcblk0</span>
</pre></div>
<p>Now, power down the IO board, and remove the SD card. Apply power once more, and you should be booting up from eMMC.</p>
<p>As a side note, when I first booted my CM4, it began an unattended system update and that took a while to complete. It will be best if you let it finish this before doing any serious usage. Just use <code>top</code> to check on the running processes.</p>
<h5>Other bits of information:</h5>
<ul>
<li>
<p><a href="https://forum.banana-pi.org/t/banana-pi-bpi-cm4-computer-module-with-amlogic-a311d/13390">Forum posting and associated threads</a> for discussion of BPI CM4 and its release.</p>
</li>
<li>
<p><a href="https://github.com/BPI-SINOVOIP/BPI-M2S-bsp">Linux Board Support Package (BSP)</a></p>
</li>
</ul>
<p><strong>As of this writing, the Banana Pi CM4 is currently unavailable for general purchase.</strong></p>
<h4>Final Thoughts</h4>
<p>If you are needing to operate in a familiar environment, you will want to go with the Raspberry Pi CM4. As of this writing, you will pay a premium - a 100% markup or more. You can get high priced CM4s from <a href="https://ebay.us/XCG14x">eBay</a> or <a href="https://amzn.to/3jlXZk3">Amazon</a>. If you need performance, and are not needing to use crazy shields and hats, you will want to go with the Banana Pi CM4, but the catch is, it has not been released yet. It is hands down the most robust compute module. If you are looking to use bleed-edge Linux and want a bit of a challenge, the Pine64 SOQuartz module is for you. And that leaves the Radxa CM3. If you willing to use the Rockchip tools to flash the eMMC module, and you are not concerned with software compatibility for shields and hats and you want similar performance to an RPi CM4, the Radxa might be a good choice.</p>
<script type="text/javascript">
amzn_assoc_placement = "adunit0";
amzn_assoc_tracking_id = "tinycompute05-20";
amzn_assoc_ad_mode = "search";
amzn_assoc_ad_type = "smart";
amzn_assoc_marketplace = "amazon";
amzn_assoc_region = "US";
amzn_assoc_title = "Affiliate Links";
amzn_assoc_default_search_phrase = "raspberry pi compute module";
amzn_assoc_default_category = "All";
amzn_assoc_linkid = "5fc86499de521ab9aacd2a8118914cfd";
amzn_assoc_rows = "4";
amzn_assoc_design = "text_links";
</script>
<script src="//z-na.amazon-adsystem.com/widgets/onejs?MarketPlace=US"></script>Pine64 Quartz64 A - Running Dockerhttps://tinycomputers.io/posts/pine64-quartz64-a-running-docker.html2022-12-12T17:40:49-06:002022-12-12T17:40:49-06:00A.C. Jokela<h3>Getting Started</h3>
<p>Getting <a href="https://www.docker.com/">docker</a> installed and running on a Pine64 Quartz64 A is relatively straightforward once you have found a disk image that has a few necessary things. We have a number of artisan-crafted operating systems. Most are Debian/Ubuntu feeling; there is <a href="https://dietpi.com/">DietPi</a> which has a very nice look and feel - minimalist creation. There is also <a href="https://github.com/Plebian-Linux/quartz64-images">Plebian Linux</a>, which is maintained by <a href="https://github.com/CounterPillow">CounterPillow</a>. It is very bleeding edge for its development stage. It feels less like a Linux system and more like a quirky <a href="https://www.netbsd.org">NetBSD</a> system. But, will not be using either of those; we will be using <a href="https://armbian.org">Armbian</a>. There are community maintained disk images but there is one problem: as of the time this post has been written, Dec 12, 2022, the images available from Armbian on Github do not have functioning ethernet. There is a paragraph or so on Pine64's <a href="https://wiki.pine64.org/wiki/Quartz64#No_Ethernet_Connectivity">wiki</a> about network connectivity, but I am not certain if the issue I am experiencing is related to that or if it is just a red herring.</p>
<p>Even though DietPi and Plebian have that familiar Debian/Ubuntu smell, each has its own issue that makes us drive toward Armbian. </p>
<p>First, DietPi. I love the minimalist feel but attempting to run <code>sudo apt install docker-ce</code>results in <code>dpkg</code> failing for some unclear reason. I really wanted something that should easily follow instructions that are found just about anywhere on internet for installing Docker on a Debian/Ubuntu system -- I did not want to take time trying to figure what was causing this error.</p>
<p>Second, Plebian. Plebian feels like NetBSD from early 2000s. It's rough around the edges and according its maintainer, should not be used in any capacity other than trying to help develop the distribution. The main issue that I found is it is based upon <a href="https://www.debian.org/doc/manuals/debian-faq/ftparchives#testing">Debian bookworm</a>, also known as <em>testing</em>. Docker does not have readily available Debian packages for bookworm. Bookworm is too new. I could have taken the time to figure out how to handcraft an install, but that is not what I am looking to do.</p>
<p>If you dig deep enough into the world of niche internet forums related to Pine64, Quartz64 or single board computers, you may find your way to <a href="https://forum.armbian.com/topic/23054-quartz64-model-a-installation-help/">Quartz64 Model A installation help</a> --> brings you to <a href="https://forum.pine64.org/showthread.php?tid=16312">this forum post</a> on Armbian's main forum site. Scroll down, and you find another link, this time to use <a href="https://users.armbian.com/balbes150/quartz64/">balbes150's Armbian's file storage</a> (<a href="https://ajokela.github.io/aws-s3-bucket-browser/index.html?bucket=https://s3.us-east-1.amazonaws.com/cdn.tinycomputers.io/index.html#balbes150/">mirror</a>).</p>
<p>I selected a <a href="https://users.armbian.com/balbes150/quartz64/Armbian_22.11.0-trunk_Quartz64a_jammy_edge_6.1.0.img.xz">Jammy distribution image</a>. Why? Because Docker has packages readily available for Jammy.</p>
<h3>Write a Disk Image</h3>
<p>There are literally thousands of examples of how to write a disk image to an SD card or eMMC module; everyone's setup maybe a different from others. If you are GUI-inclined, I would recommend <a href="https://www.balena.io/etcher/">balena Etcher</a>. One of the only issues I can see if it will not run on Arm-based computers, it is only available for x86|64.</p>
<p>If you are more command line inclined, you can use the program <code>dd</code>. <a href="https://www.linuxandubuntu.com/home/how-to-burn-iso-image-to-dvd-and-usb-using-dd">Here is one</a> that is fairly indepth look at using <code>dd</code>. If you are impatient and know what the name is of your device, something like the following:</p>
<div class="code"><pre class="code literal-block">xzcat Armbian-Quartz64-jammy-edge.img.xz | dd of=/dev/mmcblk1 status=progress
</pre></div>
<p>This assumes many things, but as a skilled and sophisticated Linux/UNIX user, you should not have any issues writing an image to a media type.</p>
<h3><img alt="dd-writing-image-to-media" src="https://tinycomputers.io/images/dd-writing-image-to-media.png.webp"></h3>
<h3>Installing Docker</h3>
<p>The following is a summarization of <a href="https://docs.docker.com/engine/install/ubuntu/">Install Docker Engine on Ubuntu</a>.</p>
<h4>Cleanup Old Versions</h4>
<p>Previous versions of <code>docker</code> were called a few things; let's remove the cruft.</p>
<div class="code"><pre class="code literal-block">$<span class="w"> </span>sudo<span class="w"> </span>apt-get<span class="w"> </span>remove<span class="w"> </span>docker<span class="w"> </span>docker-engine<span class="w"> </span>docker.io<span class="w"> </span>containerd<span class="w"> </span>runc
</pre></div>
<p>With those things removed, we will move onto setting a few things up to use Docker's package repository.</p>
<div class="code"><pre class="code literal-block"><span class="w"> </span>$<span class="w"> </span>sudo<span class="w"> </span>apt-get<span class="w"> </span>update
<span class="w"> </span>$<span class="w"> </span>sudo<span class="w"> </span>apt-get<span class="w"> </span>install<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>ca-certificates<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>curl<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>gnupg<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>lsb-release
</pre></div>
<p>Add Docker's GPG key (this makes it so you are less likely to get a compromised docker patch from Docker):</p>
<div class="code"><pre class="code literal-block"><span class="w"> </span>$<span class="w"> </span>sudo<span class="w"> </span>mkdir<span class="w"> </span>-p<span class="w"> </span>/etc/apt/keyrings
<span class="w"> </span>$<span class="w"> </span>curl<span class="w"> </span>-fsSL<span class="w"> </span>https://download.docker.com/linux/ubuntu/gpg<span class="w"> </span><span class="p">|</span><span class="w"> </span>sudo<span class="w"> </span>gpg<span class="w"> </span>--dearmor<span class="w"> </span>-o<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>/etc/apt/keyrings/docker.gpg
</pre></div>
<p>Setup the repository:</p>
<div class="code"><pre class="code literal-block"><span class="nb">echo</span><span class="w"> </span><span class="se">\</span>
<span class="w"> </span><span class="s2">"deb [arch=</span><span class="k">$(</span>dpkg<span class="w"> </span>--print-architecture<span class="k">)</span><span class="s2"> signed-by=/etc/apt/keyrings/docker.gpg] \ </span>
<span class="s2"> https://download.docker.com/linux/ubuntu \</span>
<span class="s2"> </span><span class="k">$(</span>lsb_release<span class="w"> </span>-cs<span class="k">)</span><span class="s2"> stable"</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>sudo<span class="w"> </span>tee<span class="w"> </span>/etc/apt/sources.list.d/docker.list<span class="w"> </span>><span class="w"> </span>/dev/null
</pre></div>
<p>This settings things up for us to be able to pull down officials packages from Docker. Let's refresh our available packages and install Docker.</p>
<div class="code"><pre class="code literal-block">$<span class="w"> </span>sudo<span class="w"> </span>apt-get<span class="w"> </span>update
$<span class="w"> </span>sudo<span class="w"> </span>apt-get<span class="w"> </span>install<span class="w"> </span>docker-ce<span class="w"> </span>docker-ce-cli<span class="w"> </span>containerd.io<span class="w"> </span>docker-compose-plugin
</pre></div>
<p>Docker should be running; let's test it.</p>
<div class="code"><pre class="code literal-block">$<span class="w"> </span>sudo<span class="w"> </span>docker<span class="w"> </span>run<span class="w"> </span>hello-world
</pre></div>
<p><img alt="success!" src="https://tinycomputers.io/images/pine64-quartz64-A-docker-2022-12-12%2017-28-15.png.webp"></p>