Deploying Example eBPF Programs On Local Host
This section describes running bpfman and the example eBPF programs on a local host. When running bpfman, it can be run as a process or run as a systemd service. Examples run the same, independent of how bpfman is deployed.
Building
To build directly on a system, make sure all the prerequisites are met, then build.
Prerequisites
This assumes bpfman is already installed and running on the system. If not, see Setup and Building bpfman.
- All requirements defined by the
cilium/ebpfpackage - libbpf development package to get the required eBPF c headers
Fedora:
sudo dnf install libbpf-devel
Ubuntu:
sudo apt-get install libbpf-dev
- Cilium's
bpf2gobinary
go install github.com/cilium/ebpf/cmd/bpf2go@master
Building Locally
To build all the C based eBPF counter bytecode, run:
To build all the Userspace GO Client examples, run:
To build only a single example:
Running On Host
The most basic way to deploy this example is running directly on a host system.
First, start or ensure bpfman is up and running.
Tutorial will guide you through deploying bpfman.
In all the examples of running on a host system, a bpfman-client certificate is used
that is generated by bpfman to encrypt the application's connection to bpfman.
The diagram below shows go-xdp-counter example, but the go-tc-counter and
go-tracepoint-counter examples operate exactly the same way.
Following the diagram (Purple numbers):
- When
go-xdp-counteruserspace is started, it will send a gRPC request over unix socket tobpfmanrequestingbpfmanto load thego-xdp-countereBPF bytecode located on disk atbpfman/examples/go-xdp-counter/bpf_bpfel.oat a priority of 50 and on interfaceens3. These values are configurable as we will see later, but for now we will use the defaults (except interface, which is required to be entered). bpfmanwill load it'sdispatchereBPF program, which links to thego-xdp-countereBPF program and return a UUID referencing the running program.bpfman listcan be used to show that the eBPF program was loaded.- Once the
go-xdp-countereBPF bytecode is loaded, the eBPF program will write packet counts and byte counts to a shared map. go-xdp-counteruserspace program periodically reads counters from the shared map and logs the value.
Running Privileged
To run the go-xdp-counter program, determine the host interface to attach the eBPF
program to and then start the go program with:
or (NOTE: TC programs also require a direction, ingress or egress)
cd bpfman/examples/go-tc-counter/
sudo ./go-tc-counter -direction ingress -iface <INTERNET INTERFACE NAME>
or
The output should show the count and total bytes of packets as they pass through the interface as shown below:
sudo ./go-xdp-counter --iface vethff657c7
2023/07/17 17:43:58 Using Input: Interface=vethff657c7 Priority=50 Source=/home/<$USER>/src/bpfman/examples/go-xdp-counter/bpf_bpfel.o
2023/07/17 17:43:58 Unable to read /etc/bpfman/bpfman.toml, using default configuration values.
2023/07/17 17:43:58 Program registered with id 6211
2023/07/17 17:44:01 4 packets received
2023/07/17 17:44:01 580 bytes received
2023/07/17 17:44:04 4 packets received
2023/07/17 17:44:04 580 bytes received
2023/07/17 17:44:07 8 packets received
2023/07/17 17:44:07 1160 bytes received
:
Use the CLI to show the go-xdp-counter eBPF bytecode was loaded.
Finally, press <CTRL>+c when finished with go-xdp-counter.
:
2023/07/17 17:44:34 28 packets received
2023/07/17 17:44:34 4060 bytes received
^C2023/07/17 17:44:35 Exiting...
2023/07/17 17:44:35 Unloading Program: 6211
Passing eBPF Bytecode In A Container Image
bpfman can load eBPF bytecode from a container image built following the spec described in eBPF Bytecode Image Specifications. Pre-built eBPF container images for the examples can be loaded from:
quay.io/bpfman-bytecode/go-xdp-counter:latestquay.io/bpfman-bytecode/go-tc-counter:latestquay.io/bpfman-bytecode/go-tracepoint-counter:latest
To use the container image, pass the URL to the userspace program:
sudo ./go-xdp-counter -iface ens3 -image quay.io/bpfman-bytecode/go-xdp-counter:latest
2022/12/02 16:28:32 Unable to read /etc/bpfman/bpfman.toml, using default configuration values.
2022/12/02 16:28:32 Using Input: Interface=ens3 Priority=50 Source=quay.io/bpfman-bytecode/go-xdp-counter:latest
2022/12/02 16:28:34 Program registered with id 6223
2022/12/02 16:28:37 4 packets received
2022/12/02 16:28:37 580 bytes received
2022/12/02 16:28:40 4 packets received
2022/12/02 16:28:40 580 bytes received
^C2022/12/02 16:28:42 Exiting...
2022/12/02 16:28:42 Unloading Program: 6223
Building eBPF Bytecode Container Image
eBPF Bytecode Image Specifications provides detailed
instructions on building and shipping bytecode in a container image.
To build go-xdp-counter and go-tc-counter eBPF bytecode container image, first make sure the
bytecode has been built (i.e. bpf_bpfel.o has been built - see Building), then
run the build commands below:
cd bpfman/examples/go-xdp-counter/
go generate
docker build \
--build-arg PROGRAM_NAME=go-xdp-counter \
--build-arg BPF_FUNCTION_NAME=xdp_stats \
--build-arg PROGRAM_TYPE=xdp \
--build-arg BYTECODE_FILENAME=bpf_bpfel.o \
--build-arg KERNEL_COMPILE_VER=$(uname -r) \
-f ../../Containerfile.bytecode . -t quay.io/$USER/go-xdp-counter-bytecode:latest
and
cd bpfman/examples/go-tc-counter/
go generate
docker build \
--build-arg PROGRAM_NAME=go-tc-counter \
--build-arg BPF_FUNCTION_NAME=stats \
--build-arg PROGRAM_TYPE=tc \
--build-arg BYTECODE_FILENAME=bpf_bpfel.o \
--build-arg KERNEL_COMPILE_VER=$(uname -r) \
-f ../../Containerfile.bytecode . -t quay.io/$USER/go-tc-counter-bytecode:latest
and
cd bpfman/examples/go-tracepoint-counter/
go generate
docker build \
--build-arg PROGRAM_NAME=go-tracepoint-counter \
--build-arg BPF_FUNCTION_NAME=tracepoint_kill_recorder \
--build-arg PROGRAM_TYPE=tracepoint \
--build-arg BYTECODE_FILENAME=bpf_bpfel.o \
--build-arg KERNEL_COMPILE_VER=$(uname -r) \
-f ../../Containerfile.bytecode . -t quay.io/$USER/go-tracepoint-counter-bytecode:latest
bpfman currently does not provide a method for pre-loading bytecode images
(see issue #603), so push the bytecode image to a remote
repository.
For example:
docker login quay.io
docker push quay.io/$USER/go-xdp-counter-bytecode:latest
docker push quay.io/$USER/go-tc-counter-bytecode:latest
Then run with the privately built bytecode container image:
sudo ./go-tc-counter -iface ens3 -direction ingress -location image://quay.io/$USER/go-tc-counter-bytecode:latest
2022/12/02 16:38:44 Unable to read /etc/bpfman/bpfman.toml, using default configuration values.
2022/12/02 16:38:44 Using Input: Interface=ens3 Priority=50 Source=quay.io/$USER/go-tc-counter-bytecode:latest
2022/12/02 16:38:45 Program registered with id 6225
2022/12/02 16:38:48 4 packets received
2022/12/02 16:38:48 580 bytes received
2022/12/02 16:38:51 4 packets received
2022/12/02 16:38:51 580 bytes received
^C2022/12/02 16:38:51 Exiting...
2022/12/02 16:38:51 Unloading Program: 6225
Preloading eBPF Bytecode
Another way to load the eBPF bytecode is to pre-load the eBPF bytecode and
pass the associated bpfman program id to the userspace program.
This is similar to how eBPF programs will be loaded in Kubernetes, except kubectl commands will be
used to create Kubernetes CRD objects instead of using the CLI, but that is covered in the next section.
The userspace programs will skip the loading portion and use the program id to find the shared
map and continue from there.
Referring back to the diagram above, the load and unload are being done by the CLI and not
go-xdp-counter userspace program.
First, use the CLI to load the go-xdp-counter eBPF bytecode:
sudo bpfman load image --image-url quay.io/bpfman-bytecode/go-xdp-counter:latest xdp --iface ens3 --priority 50
Bpfman State
---------------
Name: xdp_stats
Image URL: quay.io/bpfman-bytecode/go-xdp-counter:latest
Pull Policy: IfNotPresent
Global: None
Metadata: None
Map Pin Path: /run/bpfman/fs/maps/6229
Map Owner ID: None
Map Used By: 6229
Priority: 50
Iface: ens3
Position: 0
Proceed On: pass, dispatcher_return
Kernel State
----------------------------------
ID: 6229
Name: xdp_stats
Type: xdp
Loaded At: 2023-07-17T17:48:10-0400
Tag: 4b9d1b2c140e87ce
GPL Compatible: true
Map IDs: [2724]
BTF ID: 2834
Size Translated (bytes): 168
JITed: true
Size JITed (bytes): 104
Kernel Allocated Memory (bytes): 4096
Verified Instruction Count: 21
Then run the go-xdp-counter userspace program, passing in the UUID:
sudo ./go-xdp-counter -iface ens3 -id 6229
2022/12/02 17:01:38 Using Input: Interface=ens3 Source=6229
2022/12/02 17:01:41 180 packets received
2022/12/02 17:01:41 26100 bytes received
2022/12/02 17:01:44 184 packets received
2022/12/02 17:01:44 26680 bytes received
^C2022/12/02 17:01:46 Exiting...
2022/12/02 17:01:46 Closing Connection for Program: 6229
Then use the CLI to unload the eBPF bytecode: