Polycube - Complete Installation on Raspberry Pi CM4

A.C. Jokela

2023-01-27

Adding backports and stretch package locations to /etc/apt/source.list

deb https://deb.debian.org/debian bullseye-backports main contrib non-free
deb https://deb.debian.org/debian/ stretch-backports main contrib non-free
deb https://deb.debian.org/debian/ stretch main contrib non-free

Update local cache
```
sudo apt update
```

Install packages

sudo apt-get -y install git build-essential cmake bison flex \
       libelf-dev libllvm9 llvm-9-dev libclang-9-dev libpcap-dev \
       libnl-route-3-dev libnl-genl-3-dev uuid-dev pkg-config \
       autoconf libtool m4 automake libssl-dev kmod jq bash-completion  \
       gnupg2 golang-go-1.19 tmux bc libfl-dev libpcre2-dev libpcre3-dev

Add go to your $PATH in .bashrc; this needs to be done for root user, as well.
```
export PATH=/usr/lib/go-1.19/bin:$PATH
```

Verify go is in $PATH

go version
go version go1.19.4 linux/arm64

Install pistache - needed for the RESTful control daemon, polycubed

git clone https://github.com/oktal/pistache.git
cd pistache
# known working version of pistache
git checkout 117db02eda9d63935193ad98be813987f6c32b33
git submodule update --init
mkdir -p build && cd build
cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release -DPISTACHE_USE_SSL=ON ..
make -j $(getconf _NPROCESSORS_ONLN)
sudo make install

Install libtins

cd
git clone --branch v3.5 https://github.com/mfontanini/libtins.git
cd libtins
mkdir -p build && cd build
cmake -DLIBTINS_ENABLE_CXX11=1 \
 -DLIBTINS_BUILD_EXAMPLES=OFF -DLIBTINS_BUILD_TESTS=OFF \
 -DLIBTINS_ENABLE_DOT11=OFF -DLIBTINS_ENABLE_PCAP=OFF \
 -DLIBTINS_ENABLE_WPA2=OFF -DLIBTINS_ENABLE_WPA2_CALLBACKS=OFF ..
make -j $(getconf _NPROCESSORS_ONLN)
sudo make install
sudo ldconfig

Install libyang

cd
git clone https://github.com/CESNET/libyang.git
cd libyang
git checkout libyang1
mkdir build; cd build
cmake ..
make
sudo make install

Clone polycube repository that contains the necessary changes to config.cpp

cd
git clone https://github.com/ajokela/polycube
cd polycube
git submodule update --init --recursive

Build prometheus-cpp

cd src/libs/prometheus-cpp
mkdir build; cd build
cmake .. -DBUILD_SHARED_LIBS=ON
make -j4
sudo make install

Configure polycube

cd; cd polycube
mkdir build; cd build
cmake .. -DBUILD_SHARED_LIBS=ON \
           -DENABLE_PCN_IPTABLES=ON     \
           -DENABLE_SERVICE_BRIDGE=ON   \
           -DENABLE_SERVICE_DDOSMITIGATOR=OFF \
           -DENABLE_SERVICE_FIREWALL=ON   \
           -DENABLE_SERVICE_HELLOWORLD=OFF     \
           -DENABLE_SERVICE_IPTABLES=ON     \
           -DENABLE_SERVICE_K8SFILTER=OFF     \
           -DENABLE_SERVICE_K8SWITCH=OFF     \
           -DENABLE_SERVICE_LBDSR=OFF     \
           -DENABLE_SERVICE_LBRP=OFF     \
           -DENABLE_SERVICE_NAT=ON     \
           -DENABLE_SERVICE_PBFORWARDER=OFF     \
           -DENABLE_SERVICE_ROUTER=ON     \
           -DENABLE_SERVICE_SIMPLEBRIDGE=ON     \
           -DENABLE_SERVICE_SIMPLEFORWARDER=ON     \
           -DENABLE_SERVICE_TRANSPARENTHELLOWORLD=OFF \
           -DENABLE_SERVICE_SYNFLOOD=OFF     \
           -DENABLE_SERVICE_PACKETCAPTURE=OFF     \
           -DENABLE_SERVICE_K8SDISPATCHER=OFF

Build polycube (this will take a while; you might want to use tmux)
```
tmux
make -j4
```
To detach from the tmux terminal, press CTL+b, d

To reattached, execute:
```
tmux attach -t 0
```
Grab a coffee and go stare at your phone for a while.

If all goes well, you should see the following:

[100%] Building CXX object src/polycubed/src/CMakeFiles/polycubed.dir/load_services.cpp.o
[100%] Building CXX object src/polycubed/src/CMakeFiles/polycubed.dir/version.cpp.o
[100%] Linking CXX executable polycubed
[100%] Built target polycubed

Try to execute polycubed; we should get some sort of error(s)

sudo src/polycubed/src/polycubed
[2023-01-27 13:57:22.022] [polycubed] [info] loading configuration from /etc/polycube/polycubed.conf
[2023-01-27 13:57:22.023] [polycubed] [warning] default configfile (/etc/polycube/polycubed.conf) not found, creating a new with default parameters
terminate called after throwing an instance of 'spdlog::spdlog_ex'
  what():  Failed opening file /var/log/polycube/polycubed.log for writing: No such file or directory
Aborted

This is progress and we can handle this by making a directory.
```
sudo mkdir /var/log/polycube
```

Run polycubed again, and you should run into kernel header files not being foundation

sudo src/polycubed/src/polycubed 
[2023-01-27 14:01:21.048] [polycubed] [info] loading configuration from /etc/polycube/polycubed.conf
[2023-01-27 14:01:21.051] [polycubed] [info] configuration parameters:
[2023-01-27 14:01:21.051] [polycubed] [info]  loglevel: info
[2023-01-27 14:01:21.051] [polycubed] [info]  daemon: false
[2023-01-27 14:01:21.052] [polycubed] [info]  pidfile: /var/run/polycube.pid
[2023-01-27 14:01:21.052] [polycubed] [info]  port: 9000
[2023-01-27 14:01:21.052] [polycubed] [info]  addr: localhost
[2023-01-27 14:01:21.052] [polycubed] [info]  logfile: /var/log/polycube/polycubed.log
[2023-01-27 14:01:21.052] [polycubed] [info]  cubes-dump-file: /etc/polycube/cubes.yaml
[2023-01-27 14:01:21.052] [polycubed] [info]  cubes-dump-clean-init: false
[2023-01-27 14:01:21.052] [polycubed] [info]  cubes-dump-enable: false
[2023-01-27 14:01:21.052] [polycubed] [info] polycubed starting...
[2023-01-27 14:01:21.052] [polycubed] [info] version v0.9.0+ [git: (branch/commit): master/75da2773]
modprobe: FATAL: Module kheaders not found in directory /lib/modules/5.15.61-v8+
Unable to find kernel headers. Try rebuilding kernel with CONFIG_IKHEADERS=m (module) or installing the kernel development package for your running kernel version.
chdir(/lib/modules/5.15.61-v8+/build): No such file or directory
[2023-01-27 14:01:21.092] [polycubed] [error] error creating patch panel: Unable to initialize BPF program
[2023-01-27 14:01:21.093] [polycubed] [critical] Error starting polycube: Error creating patch panel

We need to get the Raspberry Pi Linux kernel source.
```
cd /usr/src
sudo git clone --depth=1 https://github.com/raspberrypi/linux
```
We need to see what kernel version the Raspberry Pi
```
uname -a
Linux polycube-network 5.15.61-v8+ #1579 SMP PREEMPT Fri Aug 26 11:16:44 BST 2022 aarch64 GNU/Linux
```
We are using 5.15.61-v8+. We will need to checkout the correct branch of the kernel. First move linux to linux-upstream-5.15.61-v8+
```
sudo mv linux linux-upstream-5.15.89-v8+
cd linux-upstream-5.15.89-v8+
```
Now, checkout the correct branch. It takes a format like rpi-5.15.y which corresponds to version 5.15.89
```
sudo git checkout rpi-5.15.y
```

Make a symlink from within /lib/modules to our source directory

cd /lib/modules/5.15.61-v8+
sudo ln -s /usr/src/linux-upstream-5.15.89-v8+ build

Build a new kernel to auto-generate the necessary header files.

cd /usr/src/linux-upstream-5.15.89-v8+
sudo make ARCH=arm64 bcm2711_defconfig
sudo make -j4

Grab another cup of coffee and stare at your phone for a while; this will take some time to complete. Doing this in situ will be slower than cross-compiling on a faster laptop or desktop, but the point of these instructions is not to productionize the process, it is to show how to make polycube compile and run on Arm-based systems, specifically, Raspberry Pi 4b or CM4 systems.

It might be unnecessary to completely build recompile a kernel; maybe experiment a bit with it. 19. Installing and running. Make sure go is available in your PATH for root user; it is needed to compile polycubectl.

sudo su -
cd ~pi/polycube/build
make install

make install should finished a message of Installation completed successfully. Now we can run polycubed and it will find all the associated shared libraries for the functionality we will be investigating in the next post.

sudo polycubed

You should get output that looks like this:

[2023-01-27 17:13:46.791] [polycubed] [info] loading configuration from /etc/polycube/polycubed.conf
[2023-01-27 17:13:46.793] [polycubed] [info] configuration parameters:
[2023-01-27 17:13:46.793] [polycubed] [info]  loglevel: info
[2023-01-27 17:13:46.793] [polycubed] [info]  daemon: false
[2023-01-27 17:13:46.793] [polycubed] [info]  pidfile: /var/run/polycube.pid
[2023-01-27 17:13:46.793] [polycubed] [info]  port: 9000
[2023-01-27 17:13:46.793] [polycubed] [info]  addr: localhost
[2023-01-27 17:13:46.793] [polycubed] [info]  logfile: /var/log/polycube/polycubed.log
[2023-01-27 17:13:46.794] [polycubed] [info]  cubes-dump-file: /etc/polycube/cubes.yaml
[2023-01-27 17:13:46.794] [polycubed] [info]  cubes-dump-clean-init: false
[2023-01-27 17:13:46.794] [polycubed] [info]  cubes-dump-enable: false
[2023-01-27 17:13:46.794] [polycubed] [info] polycubed starting...
[2023-01-27 17:13:46.794] [polycubed] [info] version v0.9.0+ [git: (branch/commit): master/75da2773]
prog tag mismatch 3e70ec38a5f6710 1
WARNING: cannot get prog tag, ignore saving source with program tag
prog tag mismatch 1e2ac42799daebd8 1
WARNING: cannot get prog tag, ignore saving source with program tag
[2023-01-27 17:14:03.905] [polycubed] [info] rest server listening on '127.0.0.1:9000'
[2023-01-27 17:14:03.906] [polycubed] [info] rest server starting ...
[2023-01-27 17:14:04.010] [polycubed] [info] service bridge loaded using libpcn-bridge.so
[2023-01-27 17:14:04.050] [polycubed] [info] service firewall loaded using libpcn-firewall.so
[2023-01-27 17:14:04.149] [polycubed] [info] service nat loaded using libpcn-nat.so
[2023-01-27 17:14:04.277] [polycubed] [info] service router loaded using libpcn-router.so
[2023-01-27 17:14:04.340] [polycubed] [info] service simplebridge loaded using libpcn-simplebridge.so
[2023-01-27 17:14:04.370] [polycubed] [info] service simpleforwarder loaded using libpcn-simpleforwarder.so
[2023-01-27 17:14:04.413] [polycubed] [info] service iptables loaded using libpcn-iptables.so
[2023-01-27 17:14:04.553] [polycubed] [info] service dynmon loaded using libpcn-dynmon.so
[2023-01-27 17:14:04.554] [polycubed] [info] loading metrics from yang files