For this build, as the title suggests, we will be using a Creality Ender 3 Pro as our base. I have written before about the Ender 3 Pro printers (here and here) that I have, but here is a bit of information about why I like this particular printer. The Ender 3 Pro features a sturdy frame that is made of aluminum extrusions. The frame is easy to assemble and disassemble, which we will be doing the latter, but that will be for another post.

The BIGTREETECH E3EZ Manta Mainboard is a 32-bit control board designed for use in 3D printers. It features an ARM Cortex-M4 CPU with a clock speed of 120 MHz, offering higher processing power and more precise control than 8-bit or even slower clock speed 32-bit boards. My other Ender 3 Pro inspired printers are running the 32-bit Creality 3D Printer Ender 3 Silent Motherboard V4.2.7. It is a solid motherboard, and I have had no real issues with it. The Silent Motherboard V4.2.7 that I have been running, as I have mentioned in a couple recent posts (here and here), are using a custom-configured Marlin v2.0.x firmware. Out of the box, the Manta E3EZ runs Smoothieware firmware, but, we won't be using that, nor will we be using Marlin, we will, instead, be using Klipper.

One of the key features of the E3EZ Manta board is its use of EZ2209 stepper motor drivers. These drivers offer advanced features such as stealthChop2 for silent operation, spreadCycle for dynamic current control, and stallGuard4 for stall detection. This allows for smoother and more precise movement of the printer's axes, resulting, in theory, in higher quality prints. The Silent Motherboard V4.2.7 uses TMC2225 stepper motor drivers; these are much quieter than the HR4988 used on the 8-bit motherboard that originally shipped with a stock Ender 3 Pro.

In a previous post on BIGTREETECH's CB1 compute module, I mentioned BIGTREETECH's Manta E3EZ board as being a great combination for 3D printing. Even though I have a CB1 from the previous review, felt strongly that it would be a good choice, we will, instead, be using a SOQuartz compute module. Here is a quick run down on the SOQuartz. The SOQuartz module from Pine64 is a powerful single-board computer designed for embedded systems and IoT applications. It is based on the Rockchip RK3566 SoC, which features a quad-core Arm Cortex-A55 processor running at up to 1.8GHz, along with a Mali-G52 2EE graphics processor. The module being used for this project comes with 4GB of LPDDR4 RAM and and will be paired with 16GB of external eMMC storage. It also features a wide range of connectivity options, including Gigabit Ethernet, Wi-Fi 5, Bluetooth 5.0, and support for up to two displays with resolutions of up to 4K@60Hz via HDMI and DisplayPort. Other features of the SOQuartz module include support for up to four USB 3.0 ports, a 40-pin GPIO header, and a dedicated AI accelerator for machine learning applications. With its high-performance specifications and versatile connectivity options, the SOQuartz module is a promising option for a wide range of embedded and IoT applications, like using it with a BIGTREETECH Manta E3EZ. The E3EZ will support just about any Raspberry Pi CM4 form factor. Why the SOQuartz? I have a strange adoration for things-Pine64. I like their boards and compute modules, I also love their Pinecil soldering iron (which happens to be powered by a RISCV processor). I have two ROCKPro64 single board computers running as network file storage on my home network; one even has four 10TB drives running in a (software)RAID5 configuration.

Let's get down to brass tacks and look more holistically at this project. We have already discussed the use of an Ender 3 Pro as our starting point, and using a BTT Manta E3EZ for control + Klipper; what else is going to be used? The following a list of parts, printers, primer and paint for this project.

Some of this was unnecessary. I did not need to buy two Ender 3 Pros, but the first one I bought was missing frame components and the second one's listing on eBay had photos of the actual contents. The second one will also give me an ample supply of spare parts for the other Ender 3 Pros I have in service. Other items that would be optional are the gloves, primer and paint. The intent is to use a caustic solution (like lye or Draino) to etch away at the anodizing on the aluminum. A coat of etching primer and then an off white paint job. So, what is the total of the above list? I'll just say that the total has crossed over into four-digit territory.

That's it for now. Look for parts two and three of this project build.

Polycarbonate plastics were first discovered by Dr. Hermann Schnell and Dr. Daniel Fox in 1953 at the General Electric Company. They were trying to develop a new type of material that could be used for electric insulators, but instead, stumbled upon a transparent and highly durable plastic. This new material was named "Lexan" by GE, and it quickly gained popularity due to its superior performance compared to other plastics at the time.

Polycarbonate is a type of thermoplastic polymer that is commonly used in a variety of applications due to its durability, transparency, and heat resistance. The chemical composition of polycarbonate is characterized by a repeating unit known as a carbonate group, which consists of three atoms: one carbon atom (C), one oxygen atom (O), and one additional oxygen atom that is double-bonded to the carbon atom (O=C-O). The chemical formula for polycarbonate can be represented as follows:

(-O-C(=O)-C6H4-C(=O)-O-)n

where "n" represents the number of repeating units in the polymer chain, and C6H4 refers to a phenylene group, which is a benzene ring (C6H5) with one hydrogen atom replaced by a carbon atom. The phenylene groups alternate with the carbonate groups in the polymer chain, resulting in a linear and highly branched structure. This molecular arrangement contributes to the unique properties of polycarbonate, such as its high impact strength, optical clarity, and resistance to UV radiation.

Polycarbonate has a relatively high glass transition temperature. The glass transition temperature is the temperature at which a polymer transitions from a glassy, rigid state to a rubbery, more flexible state, called Tg. Below the Tg, polycarbonate is in a glassy state and has a high modulus, meaning it is stiff and brittle. Above the Tg, polycarbonate transitions to a rubbery state and its modulus decreases, making it more flexible and capable of flowing or deforming under stress.

The Tg of polycarbonate typically ranges from about 145°C to 155°C. When polycarbonate is heated above its Tg, it becomes softer and more pliable, allowing it to flow and take on the shape of the mold or container it is placed in. This property makes polycarbonate suitable for a wide range of applications, including injection molding, thermoforming, and extrusion, where it can be melted, shaped, and cooled.

The Tg of polycarbonate also impacts its thermal stability. Polycarbonate has good thermal stability below its Tg, meaning it can withstand relatively high temperatures without significant degradation. However, above its Tg, polycarbonate can undergo molecular mobility and may be prone to degradation or reduced mechanical properties if exposed to prolonged elevated temperatures.

In the years that followed the discovery, polycarbonate plastics found widespread use in various industries. One of the early milestones was the development of bulletproof glass in the 1960s, which utilized the high impact resistance of polycarbonate to provide enhanced protection. Polycarbonate was also used to make eyeglass lenses, providing a lightweight and shatter-resistant alternative to glass lenses. Over the years, advancements in processing techniques and material formulations have led to further improvements in polycarbonate properties and expanded its applications in automotive parts, medical devices, and consumer goods, among others. The Lockheed Martin F-22 Raptor's canopy is constructed of specialty polycarbonate.

Enter, fused filament fabrication (FDM), or what most call, 3D printing. Using pure polycarbonate (whatever that actually means) as a 3D printing material can be extremely frustrating when using a desktop or kit 3D printer. You will need to be able to achieve temperatures that most retail printers are not capable of out of the box. You will need an all metal hotend that can handle temperatures at or above 300°C. Likewise, your build surface will need to be able to achieve temperatures exceeding 100°C.

If you are one of the few repeat readers of this site, you will know that I am a fan of Creality's Ender 3 Pro model of 3D printers. The Ender 3 Pro has been on the market for years, has a loyal following, and has a near endless array of aftermarket upgrades. Check out Scott Yu-Jan and his awesome Ender 3 Pro modifications. In previous posts (here, here and here), we were printing variants of nylon with fiber reinforced filaments, Like polycarbonate, nylon requires higher temperatures than normal PLA filaments. You can use the same aftermarket upgrades for our hotend, a Creality Sprite Pro. This has been a solid upgrade to my Ender 3 Pro printers; I would highly recommend it, even if you are only printing PLA. I have tried other direct drive Ender 3 Pro upgrades, and the Sprite Pro has shortest piece of internal bowden tube. The ease of loading filament is also a huge plus. There is a lower priced Creality Sprite that is not rated for high temperatures; you might be able to get by with this for printing nylon-based filaments, it would definitely work extremely well for printing with PLA filaments.

The other significant upgrade is the build surface. I upgraded to a Gulf Robotics' silicone encased all aluminum, three point build plate. This is an upgrade that might not be necessary, as the stock build plate as reach temperatures of 100°C, but, anecdotally, the heating is not as even as the Gulf Robotics' build plate. It heats up quicker, has more evenly distributed heat, and should be able to tolerate higher temperatures for longer periods of time compared to the stock build plate. As an addition to the upgraded build plate, I also decided to insulate the underside. I used Befenybay lightweight insulation. Silicone springs were also used. As an aside, I am planning on building and upgrading an Ender 3 Pro and tracking the build time as well as costs associated with making a high temperature, advanced materials printer. This will include setting Klipper firmware. Why Klipper instead of Marlin? You will have to come back to read why.

I will call this out in its own paragraph: you will need to update your Marlin firmware. This article will only get you so far. You will need to modify a few other settings.

Download High Temperature Marlin Firmware (April 16, 2023)

Source Code for Firmware

The first two temperature changes just might be too extreme, but I picked subjectively high values because I was running into temperature overshoot faults. Initially setting the hotend temperature to 320°C with an overshoot value of 20°C, when your gcode temperature was set to 300°C and with the variability of heating element, this routinely resulted in those temperature faults. By, increasing the hotend temperature and narrowing the overshoot value, this seemed to allow for higher temperatures without risking a temperature fault.

The last option, disabling runout prevention was necessary because I ended up destroying a Sprite Pro. How? Somehow, the printing stopped, and the hotend carriage got parked in one place. Every 10 seconds or so, a small amount of filament was extruded. Let this run over night, and you end up with a blob of reinforced polycarbonate that encases the nozzle, silicone sock, part of the cooling fan nozzle, as well as part of the heat sink.

In attempting to dislodge the blob of hardened polycarbonate, the two screws that hold the heat block (where the nozzle screws in) to the heat sink snapped. The whole Sprite Pro unit might be salvageable, and I do have a Sprite that can be used for parts, but I will need to expose enough of the screws to use a needle nosed plier to turn them.

Filament

Let's get down to the brass tacks on using polycarbonate filament. 3DXTech has an easy to print carbon fiber reinforced polycarbonate (or here). All of the reinforced filaments are relatively expensive compared to non-reinforced PLA or ABS filaments. You can start with easy to print polycarbonate filaments to save a little bit of money before taking the jump into reinforced filaments. Using non-reinforced filaments would also allow for the use of a 0.4mm nozzle, which should result in finer details coming through better than with a 0.6mm nozzle.

You will need an enclosure, but it is not necessary to have a heated enclosure. I have found the Gulf Robotics' upgraded build plate heater does a great job of increasing the enclosure's ambient temperature without needing another heating source.

Another tip for successful printing is to use a raft instead of a brim or skirt. Using a raft results in as much build surface contact as possible. Polycarbonate does not stick very well to most surfaces - it will gladly stick to itself. I used an adhesive - a Nano Polymer Adhesive (or here). It is not cheap. But, putting a few layers of adhesive onto the glass build surface, along with a temperature of 115°C and a raft, I have had little issues with prints becoming dislodged. PEI build plate surfaces are also said to have great adhesion properties for polycarbonate filaments.

Good luck and leave a comment if you print, successfully or unsuccessfully, with reinforced polycarbonate filaments.

Introduction:

Pure nylon may not always provide the required strength for certain applications. This is where glass reinforced nylon, carbon fiber reinforced nylon, and kevlar reinforced nylon comes in; it offers improved strength and durability while retaining the advantages of nylon. As we have already covered carbon fiber and Kevlar reinforced nylon, we will explore the discovery of glass reinforced nylon, its technical properties, and its uses in 3D printing.

The discovery of glass fiber reinforced nylon is attributed to several researchers and engineers who were working on improving the mechanical properties of nylon in the 1960s. One of the most notable contributors was Dr. Herman Mark, a polymer scientist who worked at the Polytechnic Institute of New York University. In the early 1960s, Herman and his team discovered that the addition of glass fibers to nylon significantly improved its strength and stiffness. This led to the development of glass fiber reinforced nylon, which was initially used in the automotive industry for components such as engine covers and air intake manifolds. Since then, glass fiber reinforced nylon has found its way into many different applications, including the 3D printing realm.

Glass reinforced nylon is a composite material made of nylon and glass fibers (pretty obvious, isn't it?). The glass fibers are usually added in the form of short fibers or long continuous strands, which are mixed with the nylon during the manufacturing process. The addition of glass fibers to nylon improves its mechanical properties, making it stronger, stiffer, and more resistant to wear and tear. The amount of glass fibers added to nylon can vary, with higher percentages resulting in higher strength and stiffness but lower ductility. Also, because glass is generally colorless, nylon + glass fiber is available in various colors compared to nylon + carbon fiber's only color of black.

The mechanical properties of glass reinforced nylon make it an ideal material for 3D printing applications. Glass reinforced nylon has excellent strength and stiffness. It also has good chemical resistance, making it resistant to solvents and chemicals, and can withstand high temperatures, making it suitable for applications such as automotive components and electronic enclosures.

One of the biggest challenges of 3D printing with nylon and glass fiber filament is getting the print settings just right. The material requires specific settings to achieve a successful print, and these settings can vary depending on the specific brand and type of filament being used. If the settings are not correct, the print may not adhere properly to the bed, causing warping or detachment during printing. Additionally, nylon with glass fiber filament is more prone to stringing and oozing during printing, which can leave unsightly and difficult-to-remove strands of filament on the printed object.

Another challenge is maintaining a consistent temperature throughout the printing process. Nylon and glass fiber filament require high temperatures to melt and print correctly, but if the temperature is too high or too low, the filament can become brittle and break during printing. Maintaining the right temperature can be difficult, especially for larger or more complex prints, as the filament may cool down or heat up unevenly in different parts of the object.

Post-processing can also be a challenge with nylon and glass fiber filament prints. Because of the material's strength and durability, it can be difficult to sand or smooth the surface of the printed object without damaging it. Additionally, the material is more difficult to paint or coat, which can limit the options for finishing the final product. Also, being chemically resistant to solvents and other chemicals also means it is difficult to vapor smooth objects made with the composites (or even just pure nylon alone).

With all of the background information on nylon with glass fiber out of the way, let's discuss the details of getting functional and great looking printed components.

I found that the Cura settings used for nylon + carbon fiber work very well. The only tweak I made was increasing the temperature of the print bed by 5° and this resulted in better adhesion. The other adjustment to the process that I was making sure the print bed was cleaned each time I printed - soap and hot water - as well as putting on a fresh thin layer of glue stick on the bed. Glue stick seemed to work better than the more expensive Magigoo for nylon. Since I already covered the Cura settings in the carbon fiber post, I won't duplicate them here.

Intro

What even is carbon fiber? Carbon fiber is a lightweight and extremely strong material that is widely used in a variety of industries due to its exceptional properties. It was invented in the mid-20th century by a British scientist named Roger Bacon, who discovered that carbon fibers could be produced by heating rayon fibers at high temperatures in the absence of oxygen. The resulting carbon fibers were found to be much stronger and more durable than other materials at the time.

Carbon fiber is made by weaving together thin strands of carbon atoms, which are then impregnated with a resin or polymer to create a composite material. The process of creating carbon fiber typically involves several steps, including spinning, stabilization, carbonization, and finishing. In the spinning stage, precursor fibers are created by heating a polymer or other material until it becomes a liquid, which is then extruded through small holes to create long, thin filaments. These filaments are then woven together and placed in an oven to be stabilized, which involves heating the fibers to a high temperature to remove any residual moisture and strengthen them. The stabilized fibers are then carbonized, which involves heating them to an even higher temperature in the absence of oxygen to convert them into pure carbon. Finally, the carbon fibers are coated with a resin or other material to give them additional strength and durability.

The last sentence of the paragraph, "Finally, the carbon fibers are coated with a resin or other material to give them additional strength and durability.", is where we will pick up on the story. To make nylon filament with embedded carbon fiber, and make it useful in a fused filament fabrication (e.g. 3D printing) application, the carbon fibers cannot be stringy strands, they have to be chopped. These chopped fibers are then mixed with nylon - either Nylon 6 or Nylon 12 (each have their merits and properties) - to produce a usable and useful filament.

What's Needed

Now with the introduction out of the way, let's get into what you will need to successfully print with nylon + carbon fiber filament.

I am using what started out as a Creality Ender 3 Pro but, since purchasing it, I have thrown a whole lot of upgrades at it. If you are not wanting to drop the amount of money into your Ender, there is three upgrades needed for printing Nylon + carbon fiber (this also applies to all fiber embedded nylon filaments).

First, you will need to have an all metal hotend. Micro-Swiss makes a drop-in all metal hotend that, as of this writing, is about $65. If you have a little more money to spend, I would recommend Creality Sprite Extruder Pro Upgrade. For $45 more, you can get a direct drive all metal hotend that can survive high temperatures. I like the Creality Sprite extruder because the shortness between the drive gears and the top of the hotend.

Second, you will need update the Marlin Firmware on your Ender. Previously, I wrote a post on what modifications I have made to the firmware that is running on my Ender 3 Pros.

Third, you will need a tougher nozzle. I use a 6mm ruby tipped nozzle. I use a 6mm nozzle because I am concerned with fibers getting jammed up in the nozzle if the extrusion hole is too narrow. I have read other articles on people who have successfully printed with a 4mm nozzle; if you are feeling subjectively adventurous, try out a 4mm nozzle and report back with a comment on this article.

For around $90, you can get the necessary upgrades to successfully print with fibrous nylon on your Ender. My personal preference is for a few other upgrades that make printing in general easier. I would highly recommend getting a BL Touch or CR Touch for auto-leveling; and a second Z-axis drive - it just makes printing a lot more stable.

On the slicer front, I use UltiMaker's Cura. There are a number of settings I changed:

PK       ! T��c9  9  .   creality_ender3pro_nylon_+_fiber_-_100%_infill[general]
version = 4
name = Nylon + Fiber - 100% Infill
definition = creality_base

[metadata]
type = quality_changes
quality_type = super
setting_version = 21

[values]
adhesion_type = brim
layer_height = 0.25
layer_height_0 = 0.25
material_bed_temperature = 80
support_enable = True
support_type = buildplate

PK       ! �z^*�  �  7   creality_base_extruder_0_#2_nylon_+_fiber_-_100%_infill[general]
version = 4
name = Nylon + Fiber - 100% Infill
definition = creality_base

[metadata]
type = quality_changes
quality_type = super
intent_category = default
position = 0
setting_version = 21

[values]
alternate_extra_perimeter = True
cool_fan_speed = 50
infill_line_distance = 1.5
infill_pattern = trihexagon
infill_randomize_start_location = True
infill_sparse_density = 100
ironing_enabled = True
ironing_monotonic = True
line_width = 0.55
material_alternate_walls = True
material_print_temperature = 270.0
retract_at_layer_change = True
skin_monotonic = True
speed_print = 55.0
support_angle = 35
support_brim_line_count = 32
support_brim_width = 8
support_pattern = triangles
top_bottom_pattern = zigzag
wall_thickness = 1.75
zig_zaggify_infill = True

PK       ! T��c9  9  .           �    creality_ender3pro_nylon_+_fiber_-_100%_infillPK       ! �z^*�  �  7           ��  creality_base_extruder_0_#2_nylon_+_fiber_-_100%_infillPK      �   �    

You can also download the configuration export file.

Things to note...

1. Layer height should be no less than 0.22mm
2. Reduce the fan speed to help with prevention of warping
3. To provide (in theory) better layer adhesion, reduce the line width slightly
4. And of course, make sure to up your hotend temperature and bed temperature. I have the hotend at 270° and the bed temperature at 80°.

Final Thoughts

I am using nylon + carbon fiber filament from Amazon. I would also recommend 3DXTech's nylon + carbon fiber filament.

As for what will print well and what will not: sharp details are not going to be created, printing small models (something smaller than a 3DBenchy) is not going to turn out well, using Octolapse will result into somewhat stringier results, use a very clean bed with a fill layer of glue stick applied, and, of course, have a very level bed.

Post a comment if you have tried printing nylon + carbon fiber on an Ender printer.

I am not new to 3D printing, but I am by far, not an expert. I have only been creating objects for the last couple of years. Up to this point, I have only been using filament made from polylactic acid, or more commonly know by its initials: PLA. It is probably the most common filament used by hobby-printers. It comes in a million different colors (ok, I haven't actually counted the colors), dozens of finishes - matte, gloss, metallic, satin sheen and many more. It is relatively easy to print, does not require an all metal hotend, use of a standard 0.4mm brass nozzle is the norm, does not require high temperatures and it's relatively inexpensive.

Printing with Nylon with embedded Kevlar is none of those things. Before we get to the filament, let's take a look at Kevlar and nylon and a bit of their histories and properties.

Kevlar is a high-performance synthetic material that is known for its exceptional strength and durability. It was developed by Stephanie Kwolek at the DuPont company in 1965. It is a popular material with a wide range of applications, including body armor, tires, aerospace, fire fighting equipment and sporting equipment. It is a type of aramid fiber, which is a class of synthetic polymer materials that are characterized by their high strength and heat resistance. Aramid fibers are composed of long chains of molecules that are held together by strong chemical bonds, which give them their exceptional strength and durability. Kevlar, in particular, is known for its high tensile strength - five times greater than steel, and its high temperature resistance properties - having a melting point of 600° C.

The nylon used in Nylon + Kevlar filament is called PA6, or Nylon 6, and as a filament material for 3D printing, it is popular due to its ability to produce high-quality, precise prints with a smooth surface finish. One of the most significant properties of PA6 nylon filament is its high tensile strength, making it ideal for producing functional, light-load-bearing parts. Post-printing characteristics of PA6 nylon filament include its low water absorption and this property ensures that the printed parts maintain their dimensional stability even in humid environments, reducing the risk of warping or deformation. One of the primary issues with PA6 nylon filament (e.g. pre-printing) is its tendency to absorb moisture from the air. Moisture can degrade the filament's quality, leading to decreased strength and brittleness, which can affect the performance of printed parts. Additionally, PA6 nylon filament has a relatively low melting point compared to other high-performance materials such as ABS or polycarbonate, making it easier to print on a wider range of 3D printers.

Filament

The filament is dark grey in color with a rough texture. With nylon being hygroscopic, and particularly PA6 nylon, we need to deal with moisture in the filament. I am using an Ivation Countertop Dehydrator Drying Machine and set to highest temperature, 158° F or 70° C. I left the filament in the dehydrator for about sixteen hours.

Printing

The print quality of objects made from Nylon + Kevlar filament is questionable at best. I use UltiMaker Cura for model slicing and, unfortunately, Cura does not have default settings for Nylon with a 0.6 mm nozzle. It took many iterations of adjusting parameters in Cura to arrive upon something that was close an ok quality print. Here is the configuration profile that I used. The configuration is a bit of a mess; the material is PLA and the nozzle size is 0.8 mm; in the available profiles, you should get a profile named "Nylon Kevlar"

I am using a 0.6 mm ruby tipped nozzle; if you use a vanilla brass nozzle, the kevlar fibers will quickly chew into the filament hole making it be no longer a perfectly round circle.

I had to modify the Marlin firmware on my heavily modified Ender 3 Pro to allow the hotend temperature to get up to at least 270° C and bed up to at least 80° C. I attempted printing at 255° C and the filament jammed in the hotend. I settled upon using 270° C for the hotend and 80° C for the bed.

I printed eight Benchys (there are nine in the photo; one is a PLA print for comparison), each with different Cura settings. It was a trial and error of adjusting single variables and then printing a Benchy. The quality of a print, compared to a perfectly tuned Ender 3 Pro, that uses PLA is a stark difference. I was unable to get anything that remotely appeared to be a smooth surface. All surfaces have a rough, sandpaper-like feel. YouTuber 3DP Iceland made a brief video about Nylon + Kevlar, and his results were similar to mine: rough surfaces, and very stringy results.

The first round tests involved tuning temperatures. As I mentioned, 270° C was settled on for the hotend, and 80° C for the bed. There were fewer stringing at that temperature for the hotend. Second and third rounds involved adjusting retraction of filament on moves; this too reduced stringiness. The rest of the tunings were layer height, flow, extruder movement speed, and so on.

One of the other settings that I found was just about a must use: a raft instead of a brim or skirt. I used Magigoo for better adhesion. For longer (never successful) prints, using a raft proved to not work either. The edges of the raft curled up from the bed; using a wider, tighter brim might be more helpful.

Printing with this filament is very frustrating at times. Good bed adhesion is critical. A clean, wide enough nozzle is very important. Correctly calibrated nozzle height and leveled bed is important.

This is probably now one of my least favorite materials to print with;

Just about the only thing original and stock on my two Creality Ender 3 Pro 3D printers are the extruded aluminum frames and the control interface with its infinite-turn control knob. Everything else has been replaced; mainboard, extruder hot end and direct filament drive, Z-axis upgrade with additional stepper motor, auto bed leveling and, of course, the firmware and the addition of printer management software, Octo Print via a Raspberry Pi 4b. Oh, and a web camera. The incredibly cluttered photo to left is one of my two heavily upgraded Ender 3 Pro printers.

If you are new to the 3D printer scene, and in particular the world of upgrades and modifications to kit-printers, let's step back and have an brief overview. I won't get into the super-weedy-details because that has likely been covered ad nauseam.

The gist of 3D printing is, you have filament; it can be made of a whole host of materials; everything from nylon with carbon fiber embedded in it, to the more mundane, polylactic acid or more commonly called PLA. This filament is softened enough to flow by way of the hot end and is pushed out of a precision nozzle. This hot end is most often mounted on a series X and Z-axis rails. A heated bed is mounted on the Y-axis. All the movement is made possible by the use of stepper motors. The motors, the hotend and bed temperatures are all controlled by a mainboard.

Upgrades

The upgraded mainboard has a STM32 F103 RET6 microcontroller. The upgrade gives you a 32 bit processor versus the original 8 bit -- this allows for more complicated firmware installs. The board also has improved, silent stepper motor controllers. In order to fully take advantage of this motherboard and accessories like the CR Touch or BL Touch, you will need configure and recompile the Marlin Firmware. We get to that later in this post.

The upgrades listed above are what I eventually arrived upon. There was a Micro Swiss Direct Drive Extruder.

As you can see, a base model Ender 3 Pro costs $236.00, but throw in an armful of higher end upgrades (for the retail market), and you suddenly have a setup that has cost nearly $900.00. Yikes! Are all of these upgrades necessary? I would have to say, No. The Creality Direct Drive extruder is well worth the money - never again deal with bowden tubes. The other two must upgrades are the mainboard and adding a CR Touch or BL Touch auto-leveling sensor. Runners up is the dual Z-axis; it really stabilizes the frame.

Firmware

In order to take advantage of a CR Touch or BL Touch, you will need to configure the firmware to use it. The probe-to-offset also needs to be changed when using the Sprite Direct Drive as the nozzle is a slight different location than the stock nozzle. I won't go into all the details of, but you can compare Configuration_og.h (the original) and Configuration.h as well as Configuration_adv_og.h and Configuration_adv.h. The changes range from enabling CR Touch/BL Touch and enabling a comprehensive bed leveling system, to adjusting the position of the nozzle and enabling thermal safety features.

git clone https://github.com/ajokela/ender3pro_marlin-2.0.x.git

Open Visual Studio Code, and Open Folder. Navigate to where you cloned the repository to and open it.

If you are wanting configuration and compile your own firmware, checkout Marlin and Platform.io. It will get your started. Once Platform.io is installed, you can clone the repo and open it in Visual Code.

Here are the things that were changed in Configuration.h and Configuration_adv.h

Configuration.h
#define STRING_CONFIG_H_AUTHOR "(Alex, Ender-3 Pro)"	Who made the changes.
#define CUSTOM_MACHINE_NAME "Ender-3 Pro 4.2.7 - fw v2.0.9.3 - 2023-02-23"	I like to put the date and version numbers in firmware so it is easy to identify a what and a when
#define HEATER_0_MAXTEMP 315	You will want to be careful with this setting; it is the temperature of the hotend in celsius; Needed higher than default for printing nylon and PET-G. Because of HOTEND_OVERSHOOT, maximum temperature will always be MAXTEMP - HOTEND_OVERSHOOT DO NOT SET AT THIS IF YOU HAVE A STOCK HOTEND
#define HOTEND_OVERSHOOT 20 #define BED_OVERSHOOT 15	(°C) Forbid temperatures over MAXTEMP - OVERSHOOT for hotend and (°C) Forbid temperatures over MAXTEMP - OVERSHOOT for bed
#define S_CURVE_ACCELERATION	Smoother curve motions
//#define Z_MIN_PROBE_USES_Z_MIN_ENDSTOP_PIN	Comment out because we will be using a CR-Touch or BL-Touch
#define USE_PROBE_FOR_Z_HOMING	Force the use of the probe for Z-axis homing
#define BLTOUCH	Enable BL Touch/CR Touch
#define NOZZLE_TO_PROBE_OFFSET { -10.0, -10.0, 0 }	Move the offset for the Sprite Direct Drive hotend
#define PROBING_MARGIN 15	A little more buffer around the perimeter
#define MULTIPLE_PROBING 2 #define EXTRA_PROBING 1	Add extra probings to eliminate outliers
#define PREHEAT_BEFORE_PROBING #if ENABLED(PREHEAT_BEFORE_PROBING) #define PROBING_NOZZLE_TEMP 200 // (°C) Only applies to E0 at this time #define PROBING_BED_TEMP 60 #endif	Require minimum nozzle and/or bed temperature for probing; bump temperature to match pre-probing temperature
#define Y_BED_SIZE 210 #define Z_MAX_POS X_BED_SIZE	Adjust bed size; I ran into problems where the extruder would overshoot the bed.
#define AUTO_BED_LEVELING_UBL	Unified Bed Leveling. A comprehensive bed leveling system combining the features and benefits of other systems. UBL also includes integrated Mesh Generation, Mesh Validation and Mesh Editing systems.
#define ENABLE_LEVELING_AFTER_G28	Always enable leveling immediately after G28.
#define G26_MESH_VALIDATION	Enable the G26 Mesh Validation Pattern tool.
#define GRID_MAX_POINTS_X 6 #define UBL_HILBERT_CURVE #define UBL_MESH_WIZARD	Use Hilbert distribution for less travel when probing multiple points. Run several commands in a row to get a complete mesh.
#define LCD_BED_LEVELING	Add a bed leveling sub-menu for ABL or MBL.
#define Z_SAFE_HOMING	Moves the Z probe (or nozzle) to a defined XY point before Z homing.
#define PREHEAT_1_TEMP_HOTEND 200 #define PREHEAT_1_TEMP_BED 60	Bump up the preheat temperatures of hotend and bed

Configuration_adv.h
#define THERMAL_PROTECTION_PERIOD 120 // Seconds #define THERMAL_PROTECTION_HYSTERESIS 10 // Degrees Celsius	False positives with Thermal Runaway
#define EXTRUDER_RUNOUT_PREVENT #if ENABLED(EXTRUDER_RUNOUT_PREVENT) #define EXTRUDER_RUNOUT_MINTEMP 195 #define EXTRUDER_RUNOUT_SECONDS 30 #define EXTRUDER_RUNOUT_SPEED 1500 // (mm/min) #define EXTRUDER_RUNOUT_EXTRUDE 5 // (mm) #endif	Extruder runout prevention. If the machine is idle and the temperature over MINTEMP then extrude some filament every couple of SECONDS.
#define HOTEND_IDLE_TIMEOUT #if ENABLED(HOTEND_IDLE_TIMEOUT) #define HOTEND_IDLE_TIMEOUT_SEC (10*60) // (seconds) Time without extruder movement to trigger protection #define HOTEND_IDLE_MIN_TRIGGER 195 // (°C) Minimum temperature to enable hotend protection #define HOTEND_IDLE_NOZZLE_TARGET 0 // (°C) Safe temperature for the nozzle after timeout #define HOTEND_IDLE_BED_TARGET 0 // (°C) Safe temperature for the bed after timeout #endif	Hotend Idle Timeout and Prevent filament in the nozzle from charring and causing a critical jam.
#define PROBE_OFFSET_WIZARD	Add Probe Z Offset calibration to the Z Probe Offsets menu
#define PROBE_OFFSET_WIZARD_START_Z -4.0	Enable to init the Probe Z-Offset when starting the Wizard. Use a height slightly above the estimated nozzle-to-probe Z offset.
#define PROBE_OFFSET_WIZARD_XY_POS { X_CENTER, Y_CENTER }	Set a convenient position to do the calibration (probing point and nozzle/bed-distance).
#define LCD_SET_PROGRESS_MANUALLY	Add an 'M73' G-code to set the current percentage
#define USE_M73_REMAINING_TIME #define ROTATE_PROGRESS_DISPLAY	Use remaining time from M73 command instead of estimation; and Display (P)rogress, (E)lapsed, and (R)emaining time
#define LCD_PROGRESS_BAR	Show a progress bar on HD44780 LCDs for SD printing
#define BINARY_FILE_TRANSFER	Add an optimized binary file transfer mode, initiated with 'M28 B1'
#define BABYSTEP_DISPLAY_TOTAL #define BABYSTEP_ZPROBE_OFFSET #if ENABLED(BABYSTEP_ZPROBE_OFFSET) #define BABYSTEP_ZPROBE_GFX_OVERLAY #endif	Display total babysteps since last G28 Combine M851 Z and Babystepping Enable graphical overlay on Z-offset editor
#define HOST_ACTION_COMMANDS #if ENABLED(HOST_ACTION_COMMANDS) #define HOST_PAUSE_M76 #define HOST_PROMPT_SUPPORT #if ENABLED(HOST_PROMPT_SUPPORT) #define HOST_STATUS_NOTIFICATIONS #endif #define HOST_START_MENU_ITEM #define HOST_SHUTDOWN_MENU_ITEM #endif	Tell the host to pause in response to M76 Initiate host prompts to get user feedback Send some status messages to the host as notifications Add a menu item that tells the host to start Add a menu item that tells the host to shut down

Even with all of this add-ons and modifications, the printer remains finicky. It is constantly needing adjustments which is expected to an extent when you are dealing with moving material and high heat.

Does it print well? It depends. It depends upon the nozzle wear, the flexibility and moisture content of the filament, and the type of the filament. These are all variables that any 3d printer would encounter. I just don't know how big of a deal these would be to another printer. I have also two Creality CR-6 SE printers, and they worked well until they did not. Maybe someday I will get a higher-end printer and be able to do more comparisons.

Download most recent compiled firmware (v2.0.9.3)

A commenter on the previous review of Raspberry Pi CM4 and pin compatible modules brought to my attention that there exists a fifth module: BIGTREETECH CB1.

My hot take on this system on a module is it is underwhelming. The two call outs are the memory size - 1 gigabyte - and the ethernet - 100 megabits only. The other four modules previously tested all had 4 gigabytes of memory and all had 1 gigabit ethernet.

If you are thinking, what could this comparatively underwhelming module be used for? First, let's take a look at BIGTREETECH. If you have been into the 3D printer kit scene, you might be familiar with the manufacturer. BIGTREETECH is known for its 3D printer mainboards and other 3D printing related electronics. The CB1 could be easily dropped in in-place for a Raspberry Pi for your Creality Ender 3 Pro or other printer kit. You will need a carrier board for it, but it will work.

OctoPrint or Klipper will run just fine on this module. You will most certainly not need 1Gbit ethernet for printing when most 3D printers print fractions of a millimeter per minute; transmission of gcode will not max out the bandwidth. Likewise for needing more memory; OctoPrint or Klipper will certainly be more responsive with more memory, but 1GB will work just fine.

One thing that this mostly underwhelming module has going for itself is HDMI. It is capable of pumping out 60 fps 4k video. If you are looking for a module that can do this, pick the CB1. For only $40, it is a bargain compared to the RPi CM4 and compatible modules.

Disk Images for the CB1

Information and instructions on WiFi setup

For some of the CM4 pin compatible modules, like the Radxa CM3, an eMMC flash writing utility that I was only able to get working on MS Windows was needed. The CB1 is straightforward in comparison. Simply download an image (link above), and use balenaEtcher or Raspberry Pi Imager or dd to write the image to a micro SD card. The image I ultimately used comes with Linux kernel v5.16.1. Like so many Linux distributions for Arm systems, this kernel is BSP, or Board Specific Package. It is a fork from mainline Linux and it is specifically for the CB1 and its associated Arm processor. Given that this is a niche module, and short of a lot of demand for it, the kernel will likely drift as mainline Linux progresses, eventually becoming outdated. But for now, it is a contemporary, relatively new kernel by comparison; put in constrast with semi-official distribution kernel for the Banana Pi CM4, which comes with v4.9.x, was released in December of 2016.

If you stumbled upon this post by way of some 3D printer-related search, and you are just wanting to write an image to a micro sd card and get on with printing awesome stuff on your printer...here is a video with instructions.

If you do not need much computing or memory, you are mostly interested in a simple 3D printer manager or a barebones HDMI streamer, the CB1, for its price, is pretty good. There even is a drop-in replacement for Ender 3 mainboards, the BIGTREETECH Manta E3EZ V1.0 Mainboard 32 Bit Silent Control Board. This gives you OctoPrint or Klipper, for print management, plus Marlin Firmware, for printer control and gcode execution, all-in-one board for about $65. This is a great deal give the much griped about availability of Raspberry Pi modules and boards, and secondary market prices, for the small order and maker crowds.

Finally, Polycube compiles on runs successfully on this module, I will eventually include it in a network routing comparison of Raspberry Pi CM4 pin compatible modules.

With my eventual goal of testing out network and router capabilities of four compute modules that are pin compatible with the Raspberry Pi CM4, I have been doing setup work. My last few postings (here, here and here) on getting Polycube, a drop-in replacement for iptables and a number of other utilities that uses eBPF instead of the usual netfilter-based mechanisms. The objective is to test out netfilterand ebpf routing on the four modules (giving me a collection of eight test sets).

I have Polycube compiled and appearing to function on the Raspberry Pi CM4, the Pine64 SOQuartz module, and the code compiled and runnable on the Radxa CM3. There is one problem with running Polycube on the CM3: the SYSCALL for eBPF was not compiled into the kernel. Even though the code successfully compiled to an executable binary, the necessary kernel hooks are not present. The solution: compile a new kernel and create a new disk image.

If you are a person who is interested in tiny computers of various flavors, you will have noticed that there are an abundance of different distributions out on the internet. An example, for the Pine64 Quartz64 model A, there are at least three different variant distributions - Plebian Linux, DietPI, and balbes150's Armbian. They all have one thing in common, they all use Debian packages and are in one sense or another, a derivative of Debian and the Debian ecosystem. If you have used Ubuntu, you have used a distribution that leverages Debian architecture and infrastructure.

The available distributions for Radxa CM3 also use Debian ecosystem components; everything from being able to utilize other arm64 packages, to using the build infrastructure for bundling up things into a handy disk image that can be burned/written to media.

Many single board computer distributions are what is called a "board support package", or BSP for short. A BSP includes low level boot programs (a first stage bootloader, prebuilt binaries and Arm Trustzone Firmware) a boot program (a second stage bootloader , like u-boot or Tianocore EFI), an operating system and the compatible drivers for that are specific to the board. The BSP is a unique bundling of software that is specific to a given board or family of boards. Often times, the Linux kernel that is included with a given BSP has been modified and new drivers have been added. The kernel is essentially a fork and no longer tracks the "main branch" of Linux kernel development and any upstream changes in the main branch maybe difficult or impossible to incorporate. The kernel is, therefore, a snapshot in time that all too often fades into obscurity because of lack of attention from the developers or a broader community (if a community exists).

Despite not having the following and community backing like that of Raspberry Pi, Radxa does have well maintained series of BSP distributions. Many do have their kernels pegged to a specific version within the Linux kernel repository, but much of the userland software is not usually tied to specific features found in specific versions -- unless the software is something like Polycube.

Radxa does a great job of providing build frameworks for both configuring and compiling a new kernel, as well as downloading packages and building a disk image. Let's get started.

The following information is based on this.

1. As a pregame note, I made a virtual machine using VirtualBox. Specifically, Debian 11 for build the new kernel in order to prevent any unnecessary contaminations of packages, dependencies or the like on my laptop. The building of the distribution image uses Docker and will not pose any issues.

2. Clone the Github repository rockchip-bsp and specifically the stable-4.19-rock3 branch. The pull in any submodules.

   git clone -b stable-4.19-rock3 https://github.com/radxa/rockchip-bsp.git
   cd rockchip-bsp
   git submodule init
   git submodule update

   The stable-4.19-rock3 branch has support for the following boards:

   • ROCK 3A
   • ROCK 3B
   • Radxa CM3 IO
   • Radxa E23
   • Radxa E25
   • Radxa CM3 RASPCM4IO

   Cloning the repository and checking out the stable-4.19-rock3 branch produces the following directories:

   • build: Some script files and configuration files for building u-boot, kernel and rootfs.
   • kernel: kernel source code, current version is 4.19.193..
   • rkbin: Prebuilt Rockchip binaries, include first stage loader and Arm TrustZone Firmware.
   • rootfs: Bootstrap a Debian based rootfs, support architecture armhf and arm64, supports Debian Jessie, Stretch and Buster.
   • u-boot: u-boot as the second stage bootloader

   There are a few things to note. First, our kernel is version 4.19.193. Polycube requires at minimum v4.15. With v4.19, we are covered. Second, this repository/project contains scripts to bootstrap and build a disk image. We will not be using this functionality. The supported Debian distributions are too old. We have been using at least Debian bullseye for all of our Polycube testing.

3. Install a Linaro toolchain. This is used for compiling code on an x86/amd64 and producing arm64 binaries.
   
   

   wget https://releases.linaro.org/components/toolchain/binaries/7.3-2018.05/aarch64-linux-gnu/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu.tar.xz
   sudo tar xvf gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu.tar.xz  -C /usr/local/

   Linaro has driven open source software development on Arm since 2010, providing the tools, Linux kernel quality and security needed for a solid foundation to innovate on. Linaro works with member companies and the open source community to maintain the Arm software ecosystem and enable new markets on Arm architecture.

4. In your user's .bashrc file, append the following line:

   export PATH="/usr/local/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin:$PATH"

   Then source .bashrc to update your PATH variable.

   source ~/.bashrc

   Verify that the Linaro GCC toolchain is visable from your PATH

   which aarch64-linux-gnu-gcc
   /usr/local/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc

5. Install a few packages:
   

   sudo apt-get install gcc-aarch64-linux-gnu \
                 device-tree-compiler libncurses5 libncurses5-dev \
                 build-essential libssl-dev mtools bc python dosfstools

6. Build u-boot for Radxa CM3 and specifically for use with a Raspberry Pi CM4 carrier/io board.
   

   ./build/mk-uboot.sh rk3566-radxa-cm3-raspcm4io

   There should be files in out/u-boot

   ls -l out/u-boot
   total 2132
   -rw-rw-r-- 1 alex alex  299008 Feb  1 22:43 idbloader.img
   -rw-rw-r-- 1 alex alex  453056 Feb  1 22:43 rk356x_spl_loader_ddr1056_v1.10.111.bin
   -rw-rw-r-- 1 alex alex 1426944 Feb  1 22:43 u-boot.itb

7. Configure a new kernel. If you have ever cloned the Linux source code repository or unarchived a tar-file of the source and then configured kernel and then compiled it, the following step will be familiar. The build process has been remarkably similar for better part of twenty-five years. I had not configured and compiled a kernel from source in a very long time; the kernel configuration process was remarkably familiar.

   cd kernel
   export ARCH=arm64
   export CROSS_COMPILE=aarch64-linux-gnu-
   make rockchip_linux_defconfig

   There will be a file named .config, you can either edit this by hand (if you have an idea of what you are doing and need to do) or you can use a handy menu-driven interface. Either way, for my specific needs of enabling eBPF, I simply opened .config in an editor, and searched for references to BPF.

   If you want to try the menu-driven method, execute the following:

   make menuconfig

   Save your new configuration (run this whether you editted by hand or used menuconfig)

   make savedefconfig
   cp defconfig arch/arm64/configs/rockchip_linux_defconfig
   cd ..

   
   

8. Build a kernel

   ./build/mk-kernel.sh rk3566-radxa-cm3-raspcm4io

   This will kick off the compilation of the kernel; obviously, depending upon your build machine, it might take a while.

   You will likely be presented with some configuration questions:

   

   Give that I am not entirely versed in things-kernel, I answered y to all of the questions. Leave a comment below if you have some insight into the questions that are presented during the build process.

9. Pack up your new kernel and associated headers into Debian package files (e.g. .deb). The parameters for pack-kernel.sh are: 1) the name of the kernel configuration file (from step #7); 2) ebpf is a release value, it should be something useful.

   ./build/pack-kernel.sh -d rockchip_linux_defconfig -r ebpf

   This will compile the kernel, again, but this appears to be necessary because this steps does not configure the appropriate chip and board as in the previous step.
   

   ls out/packages/
   linux-4.19.193-ebpf-rockchip-g67a0c0ce87a0_4.19.193-ebpf-rockchip_arm64.changes
   linux-headers-4.19.193-ebpf-rockchip-g67a0c0ce87a0_4.19.193-ebpf-rockchip_arm64.deb
   linux-image-4.19.193-ebpf-rockchip-g67a0c0ce87a0_4.19.193-ebpf-rockchip_arm64.deb
   linux-image-4.19.193-ebpf-rockchip-g67a0c0ce87a0-dbg_4.19.193-ebpf-rockchip_arm64.deb
   linux-libc-dev_4.19.193-ebpf-rockchip_arm64.deb

   These Debian packages will be needed when we build a Debian bullseye distribution.

10. You will also need to copy rk3566-radxa-cm3-rpi-cm4-io.dtb from out/kernel directory; this device table is needed when writing a new disk image to the CM3.

    If you do want to assemble an older distribution (Debian buster or stretch), you can follow steps for Make rootfs image found here. I have a pre-built Debian buster with Desktop disk image available here

11. Change directories to place outside of the rockchip-bsp directory, and now, clone the Radxa rbuild tool

    git clone https://github.com/radxa-repo/rbuild.git

    You will need docker and associated software packages. Installing these tools should be straightforward and there are dozens if not hundreds of howtos widely available to assist you. If you do not have docker command line tools installed and you looking for a quick guide, follow these instructions before proceding.

12. Make a directory for your kernel packages; copy kernel packages

    cd rbuild

    Make a directory for the kernel packages; I will be using docker outside of the virtual machine that I used to build the kernel packages. You are free to use the VM for building the bullseye disk image, I ran into issues and decided to use my laptop to directly use docker. I used scp to copy the kernel packages from the VM into a directory named kernel that is in the rbuild directory containing the cloned repo.

13. Run rbuild to magically assemble a disk image for you; this will take a while, best to grab some coffee, or lunch, or just go home for the day. There is also a strong chance of having network timeouts while downloading necessary files. I ended up having at least five times where a package download failed and killed the whole build process. On a my Dell XPS Developer Edition laptop, in a VirtualBox VM, the process took over eight hours. It should be noted that even if there is a timeout, by specifying the -r parameter to rbuild, this is caching the necessary Debian packages.

    ./rbuild -r -k kernel/linux-image-4.19.193-ebpf-rockchip-g67a0c0ce87a0_4.19.193-ebpf-rockchip_arm64.deb radxa-cm3-rpi-cm4-io cli

    

    ls -l
    total 1434692
    -rw-rw-r-- 1 alex alex       3322 Feb  1 22:17 action.yaml
    drwxrwxr-x 6 alex alex       4096 Feb  2 09:38 common
    drwxrwxr-x 2 alex alex       4096 Feb  1 22:17 configs
    drwxrwxr-x 2 alex alex       4096 Feb  1 22:38 kernel
    -rw-r--r-- 1 alex alex 6442450944 Feb  2 11:48 radxa-cm3-rpi-cm4-io_debian_bullseye_cli.img
    -rw-rw-r-- 1 alex alex        175 Feb  2 11:48 radxa-cm3-rpi-cm4-io_debian_bullseye_cli.img.sha512
    -rwxrwxr-x 1 alex alex      18869 Feb  1 22:17 rbuild
    -rw-rw-r-- 1 alex alex       1542 Feb  1 22:17 README.md

14. And there we have it. radxa-cm3-rpi-cm4-io_debian_bullseye_cli.img is your new disk image, complete with a custom compiled kernel with eBPF enabled. We can compress the disk image with xz to make it more manageable.

    xz -z -v radxa-cm3-rpi-cm4-io_debian_bullseye_cli.img
    radxa-cm3-rpi-cm4-io_debian_bullseye_cli.img (1/1)
      3.0 %     5,938.2 KiB / 185.7 MiB = 0.031    10 MiB/s       0:18   9 min 50 s

15. You can download the kernel and disk image that was built during the writing of this post: https://cdn.tinycomputers.io/radxa-rock3/debian-buster-linux-4.19.193-2a-eBPF-rockchip-rk3566-radxa-cm3-rpicm4io.img.xz

    The Device Table file built during the writing of this post: https://cdn.tinycomputers.io/radxa-rock3/linux-image-4.19.193-ebpf-rockchip-g67a0c0ce87a0_4.19.193-ebpf-rockchip_arm64.dtb

16. Instructions on writing the disk image to eMMC on the Radxa CM3, you can follow the instructions on my previous post, Raspberry Pi CM4 and Pin Compatible Modules

More Information on Radxa's build scripts, rbuild documentation and its github repo

1. Adding backports and stretch package locations to /etc/apt/source.list
   

   deb https://deb.debian.org/debian bullseye-backports main contrib non-free
   deb https://deb.debian.org/debian/ stretch-backports main contrib non-free
   deb https://deb.debian.org/debian/ stretch main contrib non-free

2. Update local cache
   

   sudo apt update

3. Install packages
   

   sudo apt-get -y install git build-essential cmake bison flex \
          libelf-dev libllvm9 llvm-9-dev libclang-9-dev libpcap-dev \
          libnl-route-3-dev libnl-genl-3-dev uuid-dev pkg-config \
          autoconf libtool m4 automake libssl-dev kmod jq bash-completion  \
          gnupg2 golang-go-1.19 tmux bc libfl-dev libpcre2-dev libpcre3-dev

4. Add go to your $PATH in .bashrc; this needs to be done for root user, as well.
   
   

   export PATH=/usr/lib/go-1.19/bin:$PATH

5. Verify go is in $PATH
   

   go version
   go version go1.19.4 linux/arm64

6. Install pistache - needed for the RESTful control daemon, polycubed
   

   git clone https://github.com/oktal/pistache.git
   cd pistache
   # known working version of pistache
   git checkout 117db02eda9d63935193ad98be813987f6c32b33
   git submodule update --init
   mkdir -p build && cd build
   cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release -DPISTACHE_USE_SSL=ON ..
   make -j $(getconf _NPROCESSORS_ONLN)
   sudo make install

7. Install libtins
   

   cd
   git clone --branch v3.5 https://github.com/mfontanini/libtins.git
   cd libtins
   mkdir -p build && cd build
   cmake -DLIBTINS_ENABLE_CXX11=1 \
    -DLIBTINS_BUILD_EXAMPLES=OFF -DLIBTINS_BUILD_TESTS=OFF \
    -DLIBTINS_ENABLE_DOT11=OFF -DLIBTINS_ENABLE_PCAP=OFF \
    -DLIBTINS_ENABLE_WPA2=OFF -DLIBTINS_ENABLE_WPA2_CALLBACKS=OFF ..
   make -j $(getconf _NPROCESSORS_ONLN)
   sudo make install
   sudo ldconfig

8. Install libyang
   

   cd
   git clone https://github.com/CESNET/libyang.git
   cd libyang
   git checkout libyang1
   mkdir build; cd build
   cmake ..
   make
   sudo make install

9. Clone polycube repository that contains the necessary changes to config.cpp
   

   cd
   git clone https://github.com/ajokela/polycube
   cd polycube
   git submodule update --init --recursive

10. Build prometheus-cpp

    cd src/libs/prometheus-cpp
    mkdir build; cd build
    cmake .. -DBUILD_SHARED_LIBS=ON
    make -j4
    sudo make install

11. Configure polycube

    cd; cd polycube
    mkdir build; cd build
    cmake .. -DBUILD_SHARED_LIBS=ON \
               -DENABLE_PCN_IPTABLES=ON     \
               -DENABLE_SERVICE_BRIDGE=ON   \
               -DENABLE_SERVICE_DDOSMITIGATOR=OFF \
               -DENABLE_SERVICE_FIREWALL=ON   \
               -DENABLE_SERVICE_HELLOWORLD=OFF     \
               -DENABLE_SERVICE_IPTABLES=ON     \
               -DENABLE_SERVICE_K8SFILTER=OFF     \
               -DENABLE_SERVICE_K8SWITCH=OFF     \
               -DENABLE_SERVICE_LBDSR=OFF     \
               -DENABLE_SERVICE_LBRP=OFF     \
               -DENABLE_SERVICE_NAT=ON     \
               -DENABLE_SERVICE_PBFORWARDER=OFF     \
               -DENABLE_SERVICE_ROUTER=ON     \
               -DENABLE_SERVICE_SIMPLEBRIDGE=ON     \
               -DENABLE_SERVICE_SIMPLEFORWARDER=ON     \
               -DENABLE_SERVICE_TRANSPARENTHELLOWORLD=OFF \
               -DENABLE_SERVICE_SYNFLOOD=OFF     \
               -DENABLE_SERVICE_PACKETCAPTURE=OFF     \
               -DENABLE_SERVICE_K8SDISPATCHER=OFF
    

12. Build polycube (this will take a while; you might want to use tmux)

    tmux
    make -j4

    To detach from the tmux terminal, press CTL+b, d
    
    

    To reattached, execute:

    tmux attach -t 0

    
    
    

    Grab a coffee and go stare at your phone for a while.

13. If all goes well, you should see the following:

    [100%] Building CXX object src/polycubed/src/CMakeFiles/polycubed.dir/load_services.cpp.o
    [100%] Building CXX object src/polycubed/src/CMakeFiles/polycubed.dir/version.cpp.o
    [100%] Linking CXX executable polycubed
    [100%] Built target polycubed

14. Try to execute polycubed; we should get some sort of error(s)

    sudo src/polycubed/src/polycubed
    [2023-01-27 13:57:22.022] [polycubed] [info] loading configuration from /etc/polycube/polycubed.conf
    [2023-01-27 13:57:22.023] [polycubed] [warning] default configfile (/etc/polycube/polycubed.conf) not found, creating a new with default parameters
    terminate called after throwing an instance of 'spdlog::spdlog_ex'
      what():  Failed opening file /var/log/polycube/polycubed.log for writing: No such file or directory
    Aborted
    

15. This is progress and we can handle this by making a directory.

    sudo mkdir /var/log/polycube

16. Run polycubed again, and you should run into kernel header files not being foundation

    sudo src/polycubed/src/polycubed 
    [2023-01-27 14:01:21.048] [polycubed] [info] loading configuration from /etc/polycube/polycubed.conf
    [2023-01-27 14:01:21.051] [polycubed] [info] configuration parameters:
    [2023-01-27 14:01:21.051] [polycubed] [info]  loglevel: info
    [2023-01-27 14:01:21.051] [polycubed] [info]  daemon: false
    [2023-01-27 14:01:21.052] [polycubed] [info]  pidfile: /var/run/polycube.pid
    [2023-01-27 14:01:21.052] [polycubed] [info]  port: 9000
    [2023-01-27 14:01:21.052] [polycubed] [info]  addr: localhost
    [2023-01-27 14:01:21.052] [polycubed] [info]  logfile: /var/log/polycube/polycubed.log
    [2023-01-27 14:01:21.052] [polycubed] [info]  cubes-dump-file: /etc/polycube/cubes.yaml
    [2023-01-27 14:01:21.052] [polycubed] [info]  cubes-dump-clean-init: false
    [2023-01-27 14:01:21.052] [polycubed] [info]  cubes-dump-enable: false
    [2023-01-27 14:01:21.052] [polycubed] [info] polycubed starting...
    [2023-01-27 14:01:21.052] [polycubed] [info] version v0.9.0+ [git: (branch/commit): master/75da2773]
    modprobe: FATAL: Module kheaders not found in directory /lib/modules/5.15.61-v8+
    Unable to find kernel headers. Try rebuilding kernel with CONFIG_IKHEADERS=m (module) or installing the kernel development package for your running kernel version.
    chdir(/lib/modules/5.15.61-v8+/build): No such file or directory
    [2023-01-27 14:01:21.092] [polycubed] [error] error creating patch panel: Unable to initialize BPF program
    [2023-01-27 14:01:21.093] [polycubed] [critical] Error starting polycube: Error creating patch panel

17. We need to get the Raspberry Pi Linux kernel source.

    cd /usr/src
    sudo git clone --depth=1 https://github.com/raspberrypi/linux

    We need to see what kernel version the Raspberry Pi

    uname -a
    Linux polycube-network 5.15.61-v8+ #1579 SMP PREEMPT Fri Aug 26 11:16:44 BST 2022 aarch64 GNU/Linux

    We are using 5.15.61-v8+. We will need to checkout the correct branch of the kernel. First move linux to linux-upstream-5.15.61-v8+

    sudo mv linux linux-upstream-5.15.89-v8+
    cd linux-upstream-5.15.89-v8+

    Now, checkout the correct branch. It takes a format like rpi-5.15.y which corresponds to version 5.15.89

    sudo git checkout rpi-5.15.y

18. Make a symlink from within /lib/modules to our source directory

    cd /lib/modules/5.15.61-v8+
    sudo ln -s /usr/src/linux-upstream-5.15.89-v8+ build

19. Build a new kernel to auto-generate the necessary header files.

    cd /usr/src/linux-upstream-5.15.89-v8+
    sudo make ARCH=arm64 bcm2711_defconfig
    sudo make -j4

    Grab another cup of coffee and stare at your phone for a while; this will take some time to complete. Doing this in situ will be slower than cross-compiling on a faster laptop or desktop, but the point of these instructions is not to productionize the process, it is to show how to make polycube compile and run on Arm-based systems, specifically, Raspberry Pi 4b or CM4 systems.

    It might be unnecessary to completely build recompile a kernel; maybe experiment a bit with it. 19. Installing and running. Make sure go is available in your PATH for root user; it is needed to compile polycubectl.

    sudo su -
    cd ~pi/polycube/build
    make install

    make install should finished a message of Installation completed successfully. Now we can run polycubed and it will find all the associated shared libraries for the functionality we will be investigating in the next post.

    sudo polycubed

    You should get output that looks like this:

    [2023-01-27 17:13:46.791] [polycubed] [info] loading configuration from /etc/polycube/polycubed.conf
    [2023-01-27 17:13:46.793] [polycubed] [info] configuration parameters:
    [2023-01-27 17:13:46.793] [polycubed] [info]  loglevel: info
    [2023-01-27 17:13:46.793] [polycubed] [info]  daemon: false
    [2023-01-27 17:13:46.793] [polycubed] [info]  pidfile: /var/run/polycube.pid
    [2023-01-27 17:13:46.793] [polycubed] [info]  port: 9000
    [2023-01-27 17:13:46.793] [polycubed] [info]  addr: localhost
    [2023-01-27 17:13:46.793] [polycubed] [info]  logfile: /var/log/polycube/polycubed.log
    [2023-01-27 17:13:46.794] [polycubed] [info]  cubes-dump-file: /etc/polycube/cubes.yaml
    [2023-01-27 17:13:46.794] [polycubed] [info]  cubes-dump-clean-init: false
    [2023-01-27 17:13:46.794] [polycubed] [info]  cubes-dump-enable: false
    [2023-01-27 17:13:46.794] [polycubed] [info] polycubed starting...
    [2023-01-27 17:13:46.794] [polycubed] [info] version v0.9.0+ [git: (branch/commit): master/75da2773]
    prog tag mismatch 3e70ec38a5f6710 1
    WARNING: cannot get prog tag, ignore saving source with program tag
    prog tag mismatch 1e2ac42799daebd8 1
    WARNING: cannot get prog tag, ignore saving source with program tag
    [2023-01-27 17:14:03.905] [polycubed] [info] rest server listening on '127.0.0.1:9000'
    [2023-01-27 17:14:03.906] [polycubed] [info] rest server starting ...
    [2023-01-27 17:14:04.010] [polycubed] [info] service bridge loaded using libpcn-bridge.so
    [2023-01-27 17:14:04.050] [polycubed] [info] service firewall loaded using libpcn-firewall.so
    [2023-01-27 17:14:04.149] [polycubed] [info] service nat loaded using libpcn-nat.so
    [2023-01-27 17:14:04.277] [polycubed] [info] service router loaded using libpcn-router.so
    [2023-01-27 17:14:04.340] [polycubed] [info] service simplebridge loaded using libpcn-simplebridge.so
    [2023-01-27 17:14:04.370] [polycubed] [info] service simpleforwarder loaded using libpcn-simpleforwarder.so
    [2023-01-27 17:14:04.413] [polycubed] [info] service iptables loaded using libpcn-iptables.so
    [2023-01-27 17:14:04.553] [polycubed] [info] service dynmon loaded using libpcn-dynmon.so
    [2023-01-27 17:14:04.554] [polycubed] [info] loading metrics from yang files
    

After emailing three of the committers to the original Polycube project, and receiving short replies from each of that basically said, polycube was never tested on an arm-based system will likely not work without significant efforts as well as, I believe the [polycube] project is no longer active, I wanted to follow through and test the former statement and really see how much effort would it take to get a compiled binary of polycubed running on an Arm-based system.

With my previous Work In Progress, I appeared to be able to successfully build and compile an executable, but when run, the program did nothing but consume 100% of one core of the Raspberry Pi's processes.

What does this mean? A hung process, consuming 100% of one core; that feels to me like it is getting stuck in a loop without having an exit/break condition met. I started by doing what any ham-handed developer would do: I started at main() in polycubed.cpp and started to put std::cerr << "Code gets to this spot #1" << std:endl; into the code.

I narrowed this initial issue of the process hang to the following:

try {

    if (!config.load(argc, argv)) {
        exit(EXIT_SUCCESS);
    }

    std::cerr << "Configs loaded..." << std::endl;

} catch (const std::exception &e) {

    // The problem of the error in loading the config file may be due to
    // polycubed executed as normal user
    if (getuid())
        logger->critical("polycubed should be executed with root privileges");

    logger->critical("Error loading config: {}", e.what());
    exit(EXIT_FAILURE);
}

Both of the cerr statements that I added were never getting called. This narrowed down the issue to config.load(argc, argv).

Looking at config.cpp and specifically at the method, load(int argc, char *argv[]), you will find the following:

bool Config::load(int argc, char *argv[]) {
  logger = spdlog::get("polycubed");

  int option_index = 0;
  char ch;

  // do a first pass looking for "configfile", "-h", "-v"
  while ((ch = getopt_long(argc, argv, "l:p:a:dhv", options, &option_index)) !=
         -1) {
    switch (ch) {
    case 'v':
      show_version();
      return false;
    case 'h':
      show_usage(argv[0]);
      return false;
    case 4:
      configfile = optarg;
      break;
    }
  }

  load_from_file(configfile);
  load_from_cli(argc, argv);
  check();

  if (cubes_dump_clean_init) {
    std::ofstream output(cubes_dump_file);
    if (output.is_open()) {
      output << "{}";
      output.close();
    }
  }

  return true;
}

Through some amateur debugging statements, I determined that while (( ch = getopt_long..) != -1) was never ceasing. The while loop never exited. Why would this statement work flawlessly on Intel amd64-based systems and not on Arm64 systems? I am still stumped as why it would matter. However, implementing the while look as the following got me slightly further in the start-up process:

  while(true) {
    const auto ch = getopt_long(argc, argv, "l:p:a:dhv", options, &option_index);

    switch (ch) {
    case 'v':
      show_version();
      return false;
    case 'h':
      show_usage(argv[0]);
      return false;
    case 4:
      configfile = optarg;
      break;
    }

    if(-1 == ch) {
      break;
    }
  }

Maybe someone with more systems experience and C++ knowledge might have an idea as to why these two blocks of code behave differently when run on different architectures.

Anyway, being able to get a little farther into the start-up process was a sign I should keep looking into the issue. Using my Bush-league skills of debugging (e.g. liberal use of std::cerr), I determined that things were getting bound up on:

load_from_cli(argc, argv);

A look at that method reveals another, similar, while statement:

void Config::load_from_cli(int argc, char *argv[]) {
  int option_index = 0;
  char ch;
  optind = 0;
  while ((ch = getopt_long(argc, argv, "l:p:a:dhv", options, &option_index)) !=
         -1) {
    switch (ch) {
    case 'l':
      setLogLevel(optarg);
      break;
    case 'p':
      setServerPort(optarg);
      break;
    case 'd':
      setDaemon(optarg ? std::string(optarg) : "true");
      break;
    case 'a':
      setServerIP(optarg);
      break;
    case 'c':
      setCertPath(optarg);
      break;
    case 'k':
      setKeyPath(optarg);
      break;
    case '?':
      throw std::runtime_error("Missing argument, see stderr");
    case 1:
      setLogFile(optarg);
      break;
    case 2:
      setPidFile(optarg);
      break;
    case 5:
      setCACertPath(optarg);
      break;
    case 6:
      setCertWhitelistPath(optarg);
      break;
    case 7:
      setCertBlacklistPath(optarg);
      break;
    case 8:
      setCubesDumpFile(optarg);
      break;
    case 9:
      setCubesDumpCleanInit();
      break;
    case 10:
      //setCubesNoDump();
      setCubesDumpEnabled();
      break;
    }
  }
}

Again, I determined that while (( ch = getopt_long..) != -1) was never breaking from the while loop. Changing it to:

  while(true) {

    const auto ch = getopt_long(argc, argv, "l:p:a:dhv", options, &option_index);

    ...

    if(-1 == ch) {
      break;
    }

  }

This did the trick, as it had done with the previous while loop. I was able to execute polycubed but ran into a new error:

[2023-01-26 15:25:19.131] [polycubed] [info] configuration parameters:
[2023-01-26 15:25:19.131] [polycubed] [info]  loglevel: info
[2023-01-26 15:25:19.131] [polycubed] [info]  daemon: false
[2023-01-26 15:25:19.131] [polycubed] [info]  pidfile: /var/run/polycube.pid
[2023-01-26 15:25:19.131] [polycubed] [info]  port: 9000
[2023-01-26 15:25:19.131] [polycubed] [info]  addr: localhost
[2023-01-26 15:25:19.131] [polycubed] [info]  logfile: /var/log/polycube/polycubed.log
[2023-01-26 15:25:19.131] [polycubed] [info]  cubes-dump-file: /etc/polycube/cubes.yaml
[2023-01-26 15:25:19.132] [polycubed] [info]  cubes-dump-clean-init: false
[2023-01-26 15:25:19.132] [polycubed] [info]  cubes-dump-enable: false
[2023-01-26 15:25:19.132] [polycubed] [info] polycubed starting...
[2023-01-26 15:25:19.132] [polycubed] [info] version v0.9.0
modprobe: FATAL: Module kheaders not found in directory /lib/modules/5.15.84-v8+
Unable to find kernel headers. Try rebuilding kernel with CONFIG_IKHEADERS=m (module)
chdir(/lib/modules/5.15.84-v8+/build): No such file or directory
[2023-01-26 15:25:19.180] [polycubed] [error] error creating patch panel: Unable to initialize BPF program
[2023-01-26 15:25:19.188] [polycubed] [critical] Error starting polycube: Error creating patch panel

Next, I grabbed the linux kernel source from Raspberry Pi's github and setup a symlink for polycubed to find kernel headers:

git clone --depth=1 https://github.com/raspberrypi/linux.git
mv linux linux-upstream-5.15.89-v8+
sudo ln -s /usr/src/linux-upstream-5.15.89-v8+ /lib/modules/5.15.89-v8+/build
sudo ~/polycube/build/src/polycubed/src/polycubed

This results in:

[2023-01-26 15:40:19.035] [polycubed] [info] configuration parameters:
[2023-01-26 15:40:19.035] [polycubed] [info]  loglevel: trace
[2023-01-26 15:40:19.035] [polycubed] [info]  daemon: false
[2023-01-26 15:40:19.036] [polycubed] [info]  pidfile: /var/run/polycube.pid
[2023-01-26 15:40:19.036] [polycubed] [info]  port: 9000
[2023-01-26 15:40:19.036] [polycubed] [info]  addr: localhost
[2023-01-26 15:40:19.036] [polycubed] [info]  logfile: /var/log/polycube/polycubed.log
[2023-01-26 15:40:19.036] [polycubed] [info]  cubes-dump-file: /etc/polycube/cubes.yaml
[2023-01-26 15:40:19.036] [polycubed] [info]  cubes-dump-clean-init: false
[2023-01-26 15:40:19.036] [polycubed] [info]  cubes-dump-enable: false
[2023-01-26 15:40:19.036] [polycubed] [info] polycubed starting...
[2023-01-26 15:40:19.036] [polycubed] [info] version v0.9.0
bpf: Failed to load program: Invalid argument
jump out of range from insn 9 to 37
processed 0 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0

[2023-01-26 15:40:46.751] [polycubed] [error] cannot load ctrl_rx: Failed to load controller_module_rx: -1
[2023-01-26 15:40:46.800] [polycubed] [critical] Error starting polycube: cannot load controller_module_rx

It is entirely possible that I am including the wrong version of bcc;

BPF Compiler Collection (BCC)

BCC is a toolkit for creating efficient kernel tracing and manipulation programs, and includes several useful tools and examples. It makes use of extended BPF (Berkeley Packet Filters), formally known as eBPF, a new feature that was first added to Linux 3.15. Much of what BCC uses requires Linux 4.1 and above.

I decided to step back, and grab a clean copy of polycubed from github.

pi@raspberrypi:~/polycube $ git submodule update --init --recursive
pi@raspberrypi:~/polycube/build $ cmake ..  -DENABLE_PCN_IPTABLES=ON \
                                            -DENABLE_SERVICE_BRIDGE=ON \    
                                            -DENABLE_SERVICE_DDOSMITIGATOR=OFF \     
                                            -DENABLE_SERVICE_FIREWALL=ON    \
                                            -DENABLE_SERVICE_HELLOWORLD=OFF   \
                                            -DENABLE_SERVICE_IPTABLES=ON    \
                                            -DENABLE_SERVICE_K8SFILTER=OFF    \
                                            -DENABLE_SERVICE_K8SWITCH=OFF    \
                                            -DENABLE_SERVICE_LBDSR=OFF    \
                                            -DENABLE_SERVICE_LBRP=OFF  \
                                            -DENABLE_SERVICE_NAT=ON   \
                                            -DENABLE_SERVICE_PBFORWARDER=ON   \
                                            -DENABLE_SERVICE_ROUTER=ON    \
                                            -DENABLE_SERVICE_SIMPLEBRIDGE=ON    \
                                            -DENABLE_SERVICE_SIMPLEFORWARDER=ON    \
                                            -DENABLE_SERVICE_TRANSPARENTHELLOWORLD=OFF   \
                                            -DENABLE_SERVICE_SYNFLOOD=OFF   \
                                            -DENABLE_SERVICE_PACKETCAPTURE=OFF     -DENABLE_SERVICE_K8SDISPATCHER=OFF
-- The C compiler identification is GNU 10.2.1
-- The CXX compiler identification is GNU 10.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Version is v0.9.0+ [git: (branch/commit): master/a143e3c0-dirty]
-- Latest recognized Git tag is v0.9.0
-- Git HEAD is a143e3c0325400dad7b9ff3406848f5a953ed3d1
-- Revision is 0.9.0-a143e3c0
-- Performing Test HAVE_NO_PIE_FLAG
-- Performing Test HAVE_NO_PIE_FLAG - Success
-- Performing Test HAVE_REALLOCARRAY_SUPPORT
-- Performing Test HAVE_REALLOCARRAY_SUPPORT - Success
-- Found LLVM: /usr/lib/llvm-9/include 9.0.1 (Use LLVM_ROOT envronment variable for another version of LLVM)
-- Found BISON: /usr/bin/bison (found version "3.7.5")
-- Found FLEX: /usr/bin/flex (found version "2.6.4")
-- Found LibElf: /usr/lib/aarch64-linux-gnu/libelf.so  
-- Performing Test ELF_GETSHDRSTRNDX
-- Performing Test ELF_GETSHDRSTRNDX - Success
-- Could NOT find LibDebuginfod (missing: LIBDEBUGINFOD_LIBRARIES LIBDEBUGINFOD_INCLUDE_DIRS)
-- Using static-libstdc++
-- Could NOT find LuaJIT (missing: LUAJIT_LIBRARIES LUAJIT_INCLUDE_DIR)
-- jsoncons v0.142.0
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY - Success
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY - Success
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Success
-- The following OPTIONAL packages have been found:

 * BISON
 * FLEX
 * Threads

-- The following REQUIRED packages have been found:

 * LibYANG
 * LLVM
 * LibElf

-- The following OPTIONAL packages have not been found:

 * LibDebuginfod
 * LuaJIT

-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.2")
-- Found OpenSSL: /usr/lib/aarch64-linux-gnu/libcrypto.so (found version "1.1.1n")  
-- Checking for module 'libnl-3.0'
--   Found libnl-3.0, version 3.4.0
-- Checking for module 'libnl-genl-3.0'
--   Found libnl-genl-3.0, version 3.4.0
-- Checking for module 'libnl-route-3.0'
--   Found libnl-route-3.0, version 3.4.0
-- Checking for module 'libtins'
--   Found libtins, version 3.5
-- Found nlohmann_json: /home/pi/polycube/cmake/nlohmann_json/Findnlohmann_json.cmake (Required is at least version "3.5.0")
-- Checking for module 'systemd'
--   Found systemd, version 247
-- systemd services install dir: /lib/systemd/system
-- Configuring done
-- Generating done
-- Build files have been written to: /home/pi/polycube/build

cd ../src/libs/prometheus-cpp
mkdir build; cd build
cmake .. -DBUILD_SHARED_LIBS=ON
make
sudo make install

I made changes to config.cpp to deal with our issue with getopt_long and the while loop. The changes are in my polycube clone.

I also did not have to add any of the #include lines that I had added during my first attempt on a SOQuartz module.

sudo src/polycubed/src/polycubed
[2023-01-26 20:58:06.453] [polycubed] [info] loading configuration from /etc/polycube/polycubed.conf
[2023-01-26 20:58:06.456] [polycubed] [info] configuration parameters:
[2023-01-26 20:58:06.456] [polycubed] [info]  loglevel: info
[2023-01-26 20:58:06.456] [polycubed] [info]  daemon: false
[2023-01-26 20:58:06.456] [polycubed] [info]  pidfile: /var/run/polycube.pid
[2023-01-26 20:58:06.456] [polycubed] [info]  port: 9000
[2023-01-26 20:58:06.456] [polycubed] [info]  addr: localhost
[2023-01-26 20:58:06.456] [polycubed] [info]  logfile: /var/log/polycube/polycubed.log
[2023-01-26 20:58:06.456] [polycubed] [info]  cubes-dump-file: /etc/polycube/cubes.yaml
[2023-01-26 20:58:06.456] [polycubed] [info]  cubes-dump-clean-init: false
[2023-01-26 20:58:06.457] [polycubed] [info]  cubes-dump-enable: false
[2023-01-26 20:58:06.457] [polycubed] [info] polycubed starting...
[2023-01-26 20:58:06.457] [polycubed] [info] version v0.9.0+ [git: (branch/commit): master/a143e3c0-dirty]
prog tag mismatch 3e70ec38a5f6710 1
WARNING: cannot get prog tag, ignore saving source with program tag
prog tag mismatch 1e2ac42799daebd8 1
WARNING: cannot get prog tag, ignore saving source with program tag
[2023-01-26 20:58:23.636] [polycubed] [info] rest server listening on '127.0.0.1:9000'
[2023-01-26 20:58:23.637] [polycubed] [info] rest server starting ...
[2023-01-26 20:58:23.740] [polycubed] [info] service bridge loaded using libpcn-bridge.so
[2023-01-26 20:58:23.779] [polycubed] [info] service firewall loaded using libpcn-firewall.so
[2023-01-26 20:58:23.882] [polycubed] [info] service nat loaded using libpcn-nat.so
[2023-01-26 20:58:24.012] [polycubed] [info] service pbforwarder loaded using libpcn-pbforwarder.so
[2023-01-26 20:58:24.145] [polycubed] [info] service router loaded using libpcn-router.so
[2023-01-26 20:58:24.210] [polycubed] [info] service simplebridge loaded using libpcn-simplebridge.so
[2023-01-26 20:58:24.239] [polycubed] [info] service simpleforwarder loaded using libpcn-simpleforwarder.so
[2023-01-26 20:58:24.282] [polycubed] [info] service iptables loaded using libpcn-iptables.so
[2023-01-26 20:58:24.412] [polycubed] [info] service dynmon loaded using libpcn-dynmon.so
[2023-01-26 20:58:24.412] [polycubed] [info] loading metrics from yang files

The daemon successfully runs. I do, however, need to capture the work I did in getting the linux kernel source headers in place for the daemon to find to compile the eBPF code into byte code.

1. Clone the Linux repository from Raspberry Pi, https://github.com/raspberrypi/linux, into /usr/src on the Raspberry Pi
2. In /lib/modules/5.15.84-v8+/ make a symlink named build and point it to /usr/src/linux-upstream-5.15.89-v8+

That will be it for the Work In Progress posts on polycube; I could attempt to recreate the steps taken, but I feel my notes across three posts should be enough. It is also isn't like polycube deployments are in hot demand. There is a strong likely hood that I am the first and only person who has run it on Arm-based hardware. The next post on polycube will be actually using it and in particular, the drop in replacement for iptables; that is what I am most interested in.