1. Adding backports and stretch package locations to /etc/apt/source.list

    deb https://deb.debian.org/debian bullseye-backports main contrib non-free
    deb https://deb.debian.org/debian/ stretch-backports main contrib non-free
    deb https://deb.debian.org/debian/ stretch main contrib non-free
  2. Update local cache

    sudo apt update
  3. Install packages

    sudo apt-get -y install git build-essential cmake bison flex \
           libelf-dev libllvm9 llvm-9-dev libclang-9-dev libpcap-dev \
           libnl-route-3-dev libnl-genl-3-dev uuid-dev pkg-config \
           autoconf libtool m4 automake libssl-dev kmod jq bash-completion  \
           gnupg2 golang-go-1.19 tmux bc libfl-dev libpcre2-dev libpcre3-dev
  4. Add go to your $PATH in .bashrc; this needs to be done for root user, as well.

    export PATH=/usr/lib/go-1.19/bin:$PATH
  5. Verify go is in $PATH

    go version
    go version go1.19.4 linux/arm64
  6. Install pistache - needed for the RESTful control daemon, polycubed

    git clone https://github.com/oktal/pistache.git
    cd pistache
    # known working version of pistache
    git checkout 117db02eda9d63935193ad98be813987f6c32b33
    git submodule update --init
    mkdir -p build && cd build
    cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release -DPISTACHE_USE_SSL=ON ..
    make -j $(getconf _NPROCESSORS_ONLN)
    sudo make install
  7. Install libtins

    git clone --branch v3.5 https://github.com/mfontanini/libtins.git
    cd libtins
    mkdir -p build && cd build
    cmake -DLIBTINS_ENABLE_CXX11=1 \
    make -j $(getconf _NPROCESSORS_ONLN)
    sudo make install
    sudo ldconfig
  8. Install libyang

    git clone https://github.com/CESNET/libyang.git
    cd libyang
    git checkout libyang1
    mkdir build; cd build
    cmake ..
    sudo make install
  9. Clone polycube repository that contains the necessary changes to config.cpp

    git clone https://github.com/ajokela/polycube
    cd polycube
    git submodule update --init --recursive
  10. Build prometheus-cpp

    cd src/libs/prometheus-cpp
    mkdir build; cd build
    make -j4
    sudo make install
  11. Configure polycube
    cd; cd polycube
    mkdir build; cd build
    cmake .. -DBUILD_SHARED_LIBS=ON \
               -DENABLE_PCN_IPTABLES=ON     \
               -DENABLE_SERVICE_BRIDGE=ON   \
               -DENABLE_SERVICE_IPTABLES=ON     \
               -DENABLE_SERVICE_K8SFILTER=OFF     \
               -DENABLE_SERVICE_K8SWITCH=OFF     \
               -DENABLE_SERVICE_LBDSR=OFF     \
               -DENABLE_SERVICE_LBRP=OFF     \
               -DENABLE_SERVICE_NAT=ON     \
               -DENABLE_SERVICE_ROUTER=ON     \
               -DENABLE_SERVICE_SYNFLOOD=OFF     \
  12. Build polycube (this will take a while; you might want to use tmux)

    make -j4
    To detach from the tmux terminal, press CTL+b, d

    To reattached, execute:

    tmux attach -t 0

    Grab a coffee and go stare at your phone for a while.

  13. If all goes well, you should see the following:

    [100%] Building CXX object src/polycubed/src/CMakeFiles/polycubed.dir/load_services.cpp.o
    [100%] Building CXX object src/polycubed/src/CMakeFiles/polycubed.dir/version.cpp.o
    [100%] Linking CXX executable polycubed
    [100%] Built target polycubed
  14. Try to execute polycubed; we should get some sort of error(s)
    sudo src/polycubed/src/polycubed
    [2023-01-27 13:57:22.022] [polycubed] [info] loading configuration from /etc/polycube/polycubed.conf
    [2023-01-27 13:57:22.023] [polycubed] [warning] default configfile (/etc/polycube/polycubed.conf) not found, creating a new with default parameters
    terminate called after throwing an instance of 'spdlog::spdlog_ex'
      what():  Failed opening file /var/log/polycube/polycubed.log for writing: No such file or directory
  15. This is progress and we can handle this by making a directory.
    sudo mkdir /var/log/polycube
  16. Run polycubed again, and you should run into kernel header files not being foundation
    sudo src/polycubed/src/polycubed 
    [2023-01-27 14:01:21.048] [polycubed] [info] loading configuration from /etc/polycube/polycubed.conf
    [2023-01-27 14:01:21.051] [polycubed] [info] configuration parameters:
    [2023-01-27 14:01:21.051] [polycubed] [info]  loglevel: info
    [2023-01-27 14:01:21.051] [polycubed] [info]  daemon: false
    [2023-01-27 14:01:21.052] [polycubed] [info]  pidfile: /var/run/polycube.pid
    [2023-01-27 14:01:21.052] [polycubed] [info]  port: 9000
    [2023-01-27 14:01:21.052] [polycubed] [info]  addr: localhost
    [2023-01-27 14:01:21.052] [polycubed] [info]  logfile: /var/log/polycube/polycubed.log
    [2023-01-27 14:01:21.052] [polycubed] [info]  cubes-dump-file: /etc/polycube/cubes.yaml
    [2023-01-27 14:01:21.052] [polycubed] [info]  cubes-dump-clean-init: false
    [2023-01-27 14:01:21.052] [polycubed] [info]  cubes-dump-enable: false
    [2023-01-27 14:01:21.052] [polycubed] [info] polycubed starting...
    [2023-01-27 14:01:21.052] [polycubed] [info] version v0.9.0+ [git: (branch/commit): master/75da2773]
    modprobe: FATAL: Module kheaders not found in directory /lib/modules/5.15.61-v8+
    Unable to find kernel headers. Try rebuilding kernel with CONFIG_IKHEADERS=m (module) or installing the kernel development package for your running kernel version.
    chdir(/lib/modules/5.15.61-v8+/build): No such file or directory
    [2023-01-27 14:01:21.092] [polycubed] [error] error creating patch panel: Unable to initialize BPF program
    [2023-01-27 14:01:21.093] [polycubed] [critical] Error starting polycube: Error creating patch panel
  17. We need to get the Raspberry Pi Linux kernel source.
    cd /usr/src
    sudo git clone --depth=1 https://github.com/raspberrypi/linux
    We need to see what kernel version the Raspberry Pi
    uname -a
    Linux polycube-network 5.15.61-v8+ #1579 SMP PREEMPT Fri Aug 26 11:16:44 BST 2022 aarch64 GNU/Linux
    We are using 5.15.61-v8+. We will need to checkout the correct branch of the kernel. First move linux to linux-upstream-5.15.61-v8+
    sudo mv linux linux-upstream-5.15.89-v8+
    cd linux-upstream-5.15.89-v8+
    Now, checkout the correct branch. It takes a format like rpi-5.15.y which corresponds to version 5.15.89
    sudo git checkout rpi-5.15.y
  18. Make a symlink from within /lib/modules to our source directory
    cd /lib/modules/5.15.61-v8+
    sudo ln -s /usr/src/linux-upstream-5.15.89-v8+ build
  19. Build a new kernel to auto-generate the necessary header files.

    cd /usr/src/linux-upstream-5.15.89-v8+
    sudo make ARCH=arm64 bcm2711_defconfig
    sudo make -j4
    Grab another cup of coffee and stare at your phone for a while; this will take some time to complete. Doing this in situ will be slower than cross-compiling on a faster laptop or desktop, but the point of these instructions is not to productionize the process, it is to show how to make polycube compile and run on Arm-based systems, specifically, Raspberry Pi 4b or CM4 systems.

    It might be unnecessary to completely build recompile a kernel; maybe experiment a bit with it. 19. Installing and running. Make sure go is available in your PATH for root user; it is needed to compile polycubectl.

    sudo su -
    cd ~pi/polycube/build
    make install
    make install should finished a message of Installation completed successfully. Now we can run polycubed and it will find all the associated shared libraries for the functionality we will be investigating in the next post.
    sudo polycubed
    You should get output that looks like this:
    [2023-01-27 17:13:46.791] [polycubed] [info] loading configuration from /etc/polycube/polycubed.conf
    [2023-01-27 17:13:46.793] [polycubed] [info] configuration parameters:
    [2023-01-27 17:13:46.793] [polycubed] [info]  loglevel: info
    [2023-01-27 17:13:46.793] [polycubed] [info]  daemon: false
    [2023-01-27 17:13:46.793] [polycubed] [info]  pidfile: /var/run/polycube.pid
    [2023-01-27 17:13:46.793] [polycubed] [info]  port: 9000
    [2023-01-27 17:13:46.793] [polycubed] [info]  addr: localhost
    [2023-01-27 17:13:46.793] [polycubed] [info]  logfile: /var/log/polycube/polycubed.log
    [2023-01-27 17:13:46.794] [polycubed] [info]  cubes-dump-file: /etc/polycube/cubes.yaml
    [2023-01-27 17:13:46.794] [polycubed] [info]  cubes-dump-clean-init: false
    [2023-01-27 17:13:46.794] [polycubed] [info]  cubes-dump-enable: false
    [2023-01-27 17:13:46.794] [polycubed] [info] polycubed starting...
    [2023-01-27 17:13:46.794] [polycubed] [info] version v0.9.0+ [git: (branch/commit): master/75da2773]
    prog tag mismatch 3e70ec38a5f6710 1
    WARNING: cannot get prog tag, ignore saving source with program tag
    prog tag mismatch 1e2ac42799daebd8 1
    WARNING: cannot get prog tag, ignore saving source with program tag
    [2023-01-27 17:14:03.905] [polycubed] [info] rest server listening on ''
    [2023-01-27 17:14:03.906] [polycubed] [info] rest server starting ...
    [2023-01-27 17:14:04.010] [polycubed] [info] service bridge loaded using libpcn-bridge.so
    [2023-01-27 17:14:04.050] [polycubed] [info] service firewall loaded using libpcn-firewall.so
    [2023-01-27 17:14:04.149] [polycubed] [info] service nat loaded using libpcn-nat.so
    [2023-01-27 17:14:04.277] [polycubed] [info] service router loaded using libpcn-router.so
    [2023-01-27 17:14:04.340] [polycubed] [info] service simplebridge loaded using libpcn-simplebridge.so
    [2023-01-27 17:14:04.370] [polycubed] [info] service simpleforwarder loaded using libpcn-simpleforwarder.so
    [2023-01-27 17:14:04.413] [polycubed] [info] service iptables loaded using libpcn-iptables.so
    [2023-01-27 17:14:04.553] [polycubed] [info] service dynmon loaded using libpcn-dynmon.so
    [2023-01-27 17:14:04.554] [polycubed] [info] loading metrics from yang files