opentelemetry-network/renderc
Jonathan Perry a3dabc3ebb
feat(kernel-collector)!: support CO-RE (compile once run everywhere) by transitioning from bcc to libbpf (#350)
* add vmlinux.h submodule

* add libbpf cmake target and switch #includes to vmlinux.h

* fix static assert for clang / libbpf

* add missing include

* update the kernel version macros

* fix map definitions and their accessors

* fix perf ring event output in renderc

* fix include directory

* fix bpf_log line output

* refactor checking the error codes in bpf_map_lookup_elem to keep the same behavior

* remove old checking of delete unexpected return value

* migrate calls for ntohs to bpf_ntohs helper macros

* solve integer to non-integer cast warning with explicit cast

* migrate BPF_PROG_ARRAY to libbpf

* migrate tail calls

* add networking macros not in vmlinux.h

* add bpf_tracing.h helpers for PT_REGS_PARM

* solve unsigned-to-signed assignment warning

* migrate htons (we only changed ntohs earlier)

* port tcp processor's rings

* migrate DNS perf event output

* add another define for a macro missing from vmlinux.h

* port dns_message_array

* solve errors for missing define

* solve signed-unsigned conversion warnings

* add some netfilter structs missing from vmlinux.h

* move added kernel structs to vmlinux_extensions.h and added doc

* fix string_starts_with without strlen

* add structs from previous versions

* solve multiple kernel version compatibility and missing definition errors

* fix enum read from cgroup_subsys_id

* add enum declaration for cgroup_subsys_id

* convert another bcc map lookup to libbpf format

* remove preserve_access_index attribute from enum (not allowed)

* support configuration of constants using libbpf global variables. max_pid needs to be constant however

* fix string_starts_with

* fix number of parameters by inlining

* fix bpf_log exceeding number of function parameters by encoding fields onto stack to pass to bpf_log

* support functions with many parameters

* migrate userspace code to libbpf

* more migration work
 Please enter the commit message for your changes. Lines starting

* use forward declaration for the skeleton where possible

* removed redundant includes of the skel file

* clang-format

* reorder #includes for correct ordering

* fix the too long prototype for tcp_recvmsg

* fix permission check to use libbpf

* run clang-format

* add SEC annotations to bpf programs

* remove kernel header fetching

* update kernel header test to just run the collector

* run clang format

* statically link against libbpf

* add libelf1 to kernel collector dockerfile

* run a reducer when testing the kernel collector

* remove kernel fetching from the kernel-collector-test, add libelf1

* do not cancel running workflows

* mount sysfs inside the kernel collector container, for BTF

* add host networking so kernel-collector components can reach the test reducer

* remove double "--network=host"

* extract kprobe parameters using REGS_PARM for taskstats_exit

* add license to bpf code

* change to kernel-compatible license

* try to fix load test checks

* parse kprobe parameters from ctx, first update

* parse kprobe parameters from ctx, second update

* run clang format

* fix conversion typo

* support pre-mounted /sys in kernel-collector entrypoint

* give verifier hint as to tgid range in END_SAVE_ARGS

* try using u64 for bpf_trace_printk in END_SAVE_ARGS

* output pid_tgid in bpf_trace_printk in END_SAVE_ARGS to appease the verifier

* allow mounted /sys in kernel-collector-test

* fix typo

* add &= to satisfy verifier

* make new unsigned variable for printing

* fix missing escape

* remove tgid printing in macro (cannot satisfty verifier)

* try to re-get pid_tgid

* different printk format string

* add explicit cast

* try signed printing

* print dummy

* give up on getting verifier to agree to trace in macro, count in global variables instead.

* make reads verifier safe and remove BPF_REGS_PARMS with KERNEL_VERSION (not well supported it seems)

* run clang-format

* use nc instead of reducer

* increase simple test time to 30 seconds

* switch some paths from kernel version to bpf_core_field_exists

* remove getting fd of bpf program

* remove kernel symbol resolution lookup

* remove unneeded fd tracking for kprobes

* install netcat for simple test

* add missing SEC markings

* fix program parameter parsing to use PT_REGS_PARM*

* run clang-format

* fix netcat fetching

* use CORE for msg_iter

* fix duplicate SEC markings

* fix probe naming

* annotate onret_cgroup_control with SEC

* add more missing SEC annotations

* fix PT_REGS_PARM and BPF_CORE_READ in new SEC markings

* clang format

* add BPF_CORE_READ where appropriate in udp_update_stats_impl()

* use the openbsd version of netcat

* move udp_update_stats to always inline

* simplify BPF_CORE_READ

* explicitly disable UDP tracing (was already disabled)

* add BPF_CORE_READ where appropriate

* enhance error reporting when loading tail calls

* mark tail calls with SEC("kprobe")

* add BPF_CORE_READ and PT_REGS_PARM to tail probes in render_bpf.c

* fix verifier in DNS reporting

* move reading of struct iov closer to its origin to help verifier

* fix verifier issues with string_starts_with

* run clang-format

* tackle verifier error on 5.10

* remove bpf-to-bpf calls in functions that also use tail calls so 5.4 kernels will verify

* enumerate existing cgroups using cgroup_get_from_fd

* run clang-format

* fail the simple tests if the word "error" appears in the kernel-collector output

* add __always_inline to avoid mixing bpf-to-bpf calls with tail calls (forbidden in older kernels)

* change failing BPF_CORE_READ of type to bpf_probe_read_kernel

* fix bpf_log variable

* add more __always_inline directives to allow tail calls

* run clang-format

* add __always_inline to another function where compiler complained

* add __always_inline to all functions with 6 or more parameters

* add explicit null checks after casting to pre 5.14 msg struct

* simplify verification for 5.10 kernel

* add __always_inline to functions that might be called from handle_receive_udp_skb to allow tail calls on 5.4 kernels

* simplify backwards compat of handle_kprobe__tcp_sendmsg

* run clang-format

* add more __always_inline to enable tail calls in continue_tcp_sendmsg on 5.4 kernels

* fix printouts during simple run

* remove debug print

* simplify backwards compat code in handle_kprobe__tcp_recvmsg to aid verifier on 5.10 kernels

* run clang-format

* add more __always_inline on functions we missed, for continue_tcp_sendmsg on 5.4 kernels

* fix the bpf_log in handle_kprobe__tcp_recvmsg

* remove the 4.19 kernel from the test matrix since it does not support BTF

* help verifier on 5.4 kernel limit iteration size in string_starts_with

* move bpf configuration to libbpf and remove the bpf code string handling

* fix kernel_collector_test unused variable

* remove dead code

* simplify function name and remove irrelevant docs

* remove `report_debug_events` global variable -- now unused

* remove unneded defines and pragmas

* remove old comment

* remove tcp_*_handler wrappers, unneeded

* remove redundant LINUX_KERNEL_VERSION externs

* Simplify parameter loading in tcp_recvmsg kprobe

* simplify parameter loading in tcp_sendmsg kprobe

* remove dead compat code

* remove bcc from makefiles

* remove compilation and header fetching errors from troubleshooting

* fix bcc-related code comments

* add deprecation warning to the bcc-based tcp-processor python script

* remove kprobe cleanup -- unnecessary with libbpf

* simplify flow

* remove entrypoint error reporting

* remove probe_handler class member -- unneeded

* run clang-format

* remove backwards compatibility artifact

* simplify cgroup selection

* move variables to right scope

* remove dead code

* remove dead code

* remove prefix __ from __onret_udp_get_port_impl

* remove unused parameters

* don't re-read sk twice in on_ip_send_skb

* fix wrong porting bcc->libbpf hidden behind #ifdef

* simplify on_skb_free_datagram_locked

* remove some pre 3.12 cgroup support

* run clang-format

* simplify backwards compatibility

* simplify get_css_id

* rename handle_cgroup_processing -> handle_existing_cgroup

* simplify backwards compatibility in onret_cgroup_control

* suppress libbpf prints

* remove entrypoint_error.h include in reducer

* run clang-format

* keep entrypoint_error message printing, but do not require the enum values

* clang format

* run kernel-collector tests on every gha run

* only check out submodules in ext/

* remove extraneous dependency of kct test on kernel-collector

* fix removal of change set

* Use CMAKE_HOST_SYSTEM_PROCESSOR instead of `uname -m`

* remove redundant cmd_args parameters in entrypoint_kct.sh

* write all libpf messages as debug in the BPF logging kind

* clang-format

* remove redundant __attribute__((preserve_access_index))

* write libbpf logs in default debug mode

* increase libbpf debug buffer size

* move LIBBPF_DEBUG messages to LOG::trace

* clang-format
2025-08-10 02:35:43 -05:00
..
gradle chore(deps): update renderc for gradle 8 and update gradle dependency versions (#346) 2025-07-25 13:36:09 -05:00
io.opentelemetry.render feat(kernel-collector)!: support CO-RE (compile once run everywhere) by transitioning from bcc to libbpf (#350) 2025-08-10 02:35:43 -05:00
io.opentelemetry.render.standalone chore(deps): update renderc for gradle 8 and update gradle dependency versions (#346) 2025-07-25 13:36:09 -05:00
test ci: use build tools with distro packages of curl, openssl, grpc, and abseil (#338) 2025-07-25 08:37:12 -05:00
.gitignore Flowmill open-source release. 2021-03-28 11:37:52 -07:00
CMakeLists.txt Refactoring renderc's SpanAutoDependencies. 2022-10-10 16:31:12 -05:00
build.gradle chore(deps): update renderc for gradle 8 and update gradle dependency versions (#346) 2025-07-25 13:36:09 -05:00
gradlew Flowmill open-source release. 2021-03-28 11:37:52 -07:00
gradlew.bat Flowmill open-source release. 2021-03-28 11:37:52 -07:00
settings.gradle change renderc source packages and build 'net.flowmill' -> 'io.opentelemetry' 2022-08-01 13:35:55 -05:00