Network Testing PC

A NUC is a great platform for a small and portable network link testing device there you want to test apart from your production network.

Test a long copper link
Test a fiber link
Test a wireless link
Test throughput through a firewall
- For example, testing my pfSense and OPNsense firewalls

Features:

Plug-and-play headless server
DHCP server for ease of use
- Boot the NUC on one end of the link or WAN side of the firewall
- Connect workstation laptop on the other end or LAN side of the firewall
Multiple test methods
- HTML5 easy-to-use speedtest
- iPerf server
- Web server with sample pages for testing using wrk, wget, etc.

Ideas:

JTest to the server
Testing NUCs with 2.5Gb capable interfaces

If you are looking for a serious packet generator to test firewalls (e.g. an IMIX benchmark), use DPDK and Pktgen. A NUC isn't going to be able to drive traffic like a fully-featured packet generator.

Setup

Server Setup

Ubuntu 20.04 LTS is a solid server. The following is a “medium” difficult tutorial to configure it. We install our tools then turn it into a router.

Instructions:

Install Ubuntu 20.04 LTS
- Follow on-line tutorials to create a bootable USB stick (example)
- Connect the NUC to your local network for ease of setup
- Connect a monitor, keyboard, and mouse
- Insert USB stick into front USB port on NUC
- Power on the NUC
- Press F10 while the NUC is booting, and select to boot from the USB stick
- Follow Ubuntu installation prompts
  - It is recommended to install OpenSSH server when prompted. This is convenient when performing tests later.
  - yes, it will probably get stuck during a curtain update; select cancel updates and reboot
Update and install general packages
- login using the account information you chose during installation
- sudo apt update && sudo apt upgrade -y
- sudo apt install net-tools wireless-tools iftop wpasupplicant -y
Install iperf server as a service
- We are using iPerf (iPerf4) instead of iPerf3 (which does not support multithreading)
- sudo apt install iperf
- sudo vi /etc/systemd/system/iperf.service
  - ```
  [Unit]
  Description=iperf server
  After=syslog.target network.target auditd.service
  [Service]
  ExecStart=/usr/bin/iperf -s
  [Install]
  WantedBy=multi-user.target
```
- sudo chmod 644 /etc/systemd/system/iperf.service
- sudo systemctl daemon-reload
- sudo systemctl start iperf
- sudo systemctl status iperf
- sudo systemctl enable iperf
Install HTML5 Speedtest
- sudo apt install nginx php-fpm -y
- enable PHP in nginx (see details)
  - sudo vi /etc/nginx/sites-available/default
  - insert index.php before index.html
  - add (or replace existing location ~ \.php$ { config line)
    - location ~ \.php$ { include snippets/fastcgi-php.conf; fastcgi_pass unix:/var/run/php/php-fpm.sock; }
- sudo systemctl reload nginx
- cd ~
- git clone https://github.com/adolfintel/speedtest.git
- cd speedtest
- sudo cp -R backend example-singleserver-pretty.html *.js /var/www/html
- cd /var/www/html
- sudo mv example-singleserver-pretty.html index.html
- sudo chown -R www-data /var/www/html
Install a test site for testing using wrk and wget
- sudo mkdir /var/www/html/wrk
- cd /var/www/html/wrk
- sudo dd if=/dev/zero of=1kb.bin bs=1KB count=1
- sudo dd if=/dev/zero of=1mb.bin bs=1MB count=1
- sudo wget -r http://demo.borland.com/testsite/stadyn_largepagewithimages.html
- sudo vi index.html
  - ```
  <html>
  <head><title>Link</title></head>
  <body><a href="demo.borland.com/testsite/stadyn_largepagewithimages.html">link</a></body>
  </html>
```
- sudo chown -R www-data /var/www/html
Convert to static IP and DHCP server
- sudo vi /etc/netplan/00-installer-config.yaml
  - under the interface add
    - optional: yes
- sudo cp /etc/netplan/00-installer-config.yaml /etc/netplan/99-config.yaml
- sudo vi /etc/netplan/99-config.yaml
  - The interface name may vary. Record this interface name for later (or look is up using “ip a”). Edit the file to be similar to:
  - ```
  network:
    version: 2
    renderer: networkd
    ethernets:
      <interfacename>:
        optional: yes
        dhcp4: no
        addresses:
          - 192.168.178.1/24
        gateway4: 192.168.178.1
        nameservers:
          search: [localdomain]
          addresses: [192.168.178.1]
```
- NOTE: the unusual subnet was chose so as not to collide with commonly used network addresses
- NOTE: adding optional: yes will boot the server quickly even if there is no network link, but your DHCP server will not come up with the network being up. We add a crude watchdog later.
- TIP: don't run “sudo netplan apply” yet. If you do, connectivity to your network will be lost.
Install dhcp server
- sudo apt install isc-dhcp-server -y
- sudo systemctl start isc-dhcp-server
- sudo systemctl enable isc-dhcp-server
Configure dhcp server
- copy the interface name from 99-config.yaml or find it using “ip a”
- sudo vi /etc/default/isc-dhcp-server
  - ```
  INTERFACESv4="<interfacename>"
```
- sudo vi /etc/dhcp/dhcpd.conf
  - uncomment:
    - authoritative;
  - comment out:
    - option domain-name-servers ns1.example.com, ns2.example.org
  - change:
    - option domain-name “localdomain”;
    - default-lease-time 3600;
  - add:
    - subnet 192.168.178.0 netmask 255.255.255.0 { range 192.168.178.20 192.168.178.245; option routers 192.168.178.1; }
- sudo poweroff
- TIP: Do NOT run “sudo systemctl restart isc-dhcp-server” before powering off and disconnecting from your network. If you do, the NUC's DHCP server will interfere with your regular network! This is called a “rogue DHCP server” problem and it is a bad day.
- NOTE: lease file is /var/lib/dhcp/dhcpd.leases
Connect your workstation directly to the NUC's NIC
- Disconnect the now powered-off NUC from the local network
- Connect your test workstation directly to the NUC's network port
- Power on the NUC
- You can how test:
  - http://192.168.178.1
  - ssh 192.168.178.1
Auto restart DHCP server
- If the server boots without a link on the network interface, the DHCP server will not start. Adding a basic watchdog for the DHCP server to start or restart it.
- Add script /usr/local/bin/service.sh
- ```
#!/bin/env bash
#!/bin/bash
service=$@
/bin/systemctl -q is-active "$service.service"
status=$?
if [ "$status" != 0 ]; then
    /bin/systemctl start "$service.service"
fi
```
- Make the script executable
  - sudo chmod +x /usr/local/bin/service.sh
- Add the script to your root crontab to run every 2 minutes
  - */2 * * * * /usr/local/bin/service.sh isc-dhcp-server
- Or every minute
  - * * * * * /usr/local/bin/service.sh isc-dhcp-server
- Look up “Ubuntu watchdog service” for using a proper service watchdog
If you have more that one interface and want to enable routing across these interfaces
- temporary
  - check if routing enabled: sudo sysctl -a | grep net.ipv4.ip_forward
  - enable routing: sudo sysctl -w net.ipv4.ip_forward=1
- permanent
  - add to /etc/sysctl.conf: net.ipv4.ip_forward = 1
  - sudo sysctl -p

Setting Up the Workstation

I like to set up a laptop with a wired ethernet connection for this testing. A Windows workstation with WSL2 and Ubuntu is my preferred method. However, you don't need a virtual server to run the HTML5 speed test and iperf.

HTML5 SpeedTest

Point your browser the to IP address of the server. Click Start.

Results testing a high speed and low latency connection will likely vary wildly, even from test to test.

iperf

See https://iperf.fr/

Linux and WSL2 on windows: sudo apt install iperf

Or download the Windows binary.

Tests to run:

iperf -c <serverip>
iperf -c <serverip> -P 4
iperf -c <serverip> -e

-P 4 selects 4 parallel threads

-e selects enhanced reporting

wrk

Instructions are for a Linux environment such as WSL2 on Windows.

Prior to installation you will want to:

sudo apt install build-essential unzip

See https://github.com/wg/wrk

Instructions:

cd ~
git clone https://github.com/wg/wrk.git
cd wrk
make
- the usual sudo make install is apparently not needed

Example commands:

./wrk -t2 -c50 -d30s http://<ip address>/wrk/index.html
./wrk -t2 -c50 -d30s http://<ipaddress>/wrk/demo.borland.com/testsite/stadyn_largepagewithimages.html
./wrk -t2 -c50 -d30s http://<ipaddress>/wrk/1kb.bin
./wrk -t2 -c50 -d30s http://<ipaddress>/wrk/1mb.bin

-t 2 assigned 2 threads

-c50 assigns 50 connections

-d assigns 30 second duration

wget

Pull one or more files from the web server. Note that the recursion flag (-r) cannot be used with a special file like /dev/null.

Installation:

Linux - apt install wget
Windows - https://www.gnu.org/software/wget/

wget http://127.0.0.1/wrk/demo.borland.com/testsite/stadyn_largepagewithimages.html -O /dev/null
wget -r http://127.0.0.1/wrk/demo.borland.com/testsite/stadyn_largepagewithimages.html -O /tmp/blob.fil

How to Use

Test direct connection from test workstation to the server
- HTML5 bandwidth test: https://192.168.178.1
- iperf test
  - iperf -c 192.168.178.1 -e
  - iperf -c 192.168.178.1 -u -e
  - iperf -c 192.168.178.1 -P 10 -e
- wrk
  - ./wrk -t2 -c50 -d30s http:<ip address>/wrk/index.html * ./wrk -t2 -c50 -d30s http:<ipaddress>/wrk/demo.borland.com/testsite/stadyn_largepagewithimages.html
  - ./wrk -t2 -c50 -d30s http:<ipaddress>/wrk/1kb.bin * ./wrk -t2 -c50 -d30s http:<ipaddress>/wrk/1mb.bin
- wget
  - wget http://127.0.0.1/wrk/demo.borland.com/testsite/stadyn_largepagewithimages.html -O /dev/null
  - wget -r http://127.0.0.1/wrk/demo.borland.com/testsite/stadyn_largepagewithimages.html -O /tmp/blob.fil
- Check workstation and server interfaces for errors
  - Linux
    - ifconfig
    - ethtool -S <interfacename>
  - Windows
    - windows powershell: netsh interface ipv6 show ipstats | Select-String errors
  - Note that rx overruns can occur with load testing even on a perfect link
Test a link (fiber link, wireless link, wired trunk)
- HTML5 bandwidth test: https://192.168.178.1
- iperf test
  - iperf -c 192.168.178.1 -e
  - iperf -c 192.168.178.1 -u -e
  - iperf -c 192.168.178.1 -P 10 -e
- wrk
  - ./wrk -t2 -c50 -d30s http:<ip address>/wrk/index.html * ./wrk -t2 -c50 -d30s http:<ipaddress>/wrk/demo.borland.com/testsite/stadyn_largepagewithimages.html
  - ./wrk -t2 -c50 -d30s http:<ipaddress>/wrk/1kb.bin * ./wrk -t2 -c50 -d30s http:<ipaddress>/wrk/1mb.bin
- wget
  - wget http://127.0.0.1/wrk/demo.borland.com/testsite/stadyn_largepagewithimages.html -O /dev/null
  - wget -r http://127.0.0.1/wrk/demo.borland.com/testsite/stadyn_largepagewithimages.html -O /tmp/blob.fil
- Check switch(es) for interface errors
Test a firewall
- HTML5 bandwidth test: https://192.168.178.1
- iperf test
  - iperf -c 192.168.178.1 -e
  - iperf -c 192.168.178.1 -u -e
  - iperf -c 192.168.178.1 -P 10 -e
- wrk
  - ./wrk -t2 -c50 -d30s http:<ip address>/wrk/index.html * ./wrk -t2 -c50 -d30s http:<ipaddress>/wrk/demo.borland.com/testsite/stadyn_largepagewithimages.html
  - ./wrk -t2 -c50 -d30s http:<ipaddress>/wrk/1kb.bin * ./wrk -t2 -c50 -d30s http:<ipaddress>/wrk/1mb.bin
- wget
  - wget http://127.0.0.1/wrk/demo.borland.com/testsite/stadyn_largepagewithimages.html -O /dev/null
  - wget -r http://127.0.0.1/wrk/demo.borland.com/testsite/stadyn_largepagewithimages.html -O /tmp/blob.fil

Maintenance

Updating the Server

Most NUCs have only 1 RJ45 interface. Since your NUC currently runs a DHCP server on that interface, plugging it into your network could take your network down.

Here are some options to connect to your network for updates.

Shutdown the DHCP server and set the interface to get an IP via DHCP
- sudo systemctl stop isc-dhcp-server
- sudo mv /etc/netplan/99-config.yaml ~
- sudo netplan apply
- connect the NUC to your network
- when done
  - disconnect from your network
  - sudo mv ~/etc/netplan/99-config.yaml /etc/netplan/99-config.yaml
  - sudo netplan apply
  - sudo systemctl start isc-dhcp-server
Plug a USB RJ45 ethernet adapter into the NUC and configure this interface before connecting to your network.
- ip a
- sudo ip link set <interfacename> up
- sudo dhclient <interfacename>
- This interface is unmanaged (sudo networkctl) unless you configure it in netplan
If the NUC as a Wifi card, configure it and join your wifi network
- TODO - need to test
Plug in a USB wireless adapter and use that
- Details at https://schillernet.wordpress.com/2020/07/17/setting-up-a-usb-wifi-dongle-in-ubuntu-server-18-04-or-later/
- WARNING this is problematic
- lsusb
- lshw -C network
- ip link show
- iwconfig
  - record the name of the interface
- sudo vi /etc/netplan/01-netcfg.yaml
  - ```
  network:
    version: 2
    renderer: networkd
    wifis:
      <similarto:wlx00e0262f73ec>:
        optional: yes
        dhcp4: yes
        dhcp6: yes
        access-points:
          "ESSID":
            password: "yourpassphrasehere"
```
- sudo netplan –debug apply
- sudo dhclient <similarto:wlx00e0262f73ec>
- ip address show dev <similarto:wlx00e0262f73ec>
- sudo iwlist <similarto:wlx00e0262f73ec> scan
- troubleshooting
  - sudo ip link set <similarto:wlx00e0262f73ec> up
  - dmesg | grep -e wlp -e network
  - cat /var/log/syslog | grep -i network
  - networkctl

To update Ubuntu and its software packages:

sudo apt-get update && sudo apt-get upgrade -y

Server Performance

After building your server, you will want to test it from a directly connected workstation. See below on how to set up your workstation.

You can use top or htop in conjunction with iftop to view CPU and the amount of bandwidth you are able to drive to the NUC and if the CPU is the limiting factor. If you don't mind a little overhead, installing and running glances does both in a handy dashboard.

One reason to choose a NUC over a RaspberryPi for this use case is the drastic bandwidth limitation on the Pi's network interface being connected to the USB bus. It's too slow for this use case.

From your testing you will find the maximum bandwidth you can drive to the server, which you can compare to your tests through a firewall or over a network link.

UncleNUC Wiki

Table of Contents