[CROSS-COMPILE] BPF JIT verifier return -EACESS in arm 32-bit

[LinuxZrc]

Hello, I'm trying to debug scheduling latency on an ARM 32-bit system. I've tried porting runqlat in 32-bit system, but haven't found any issues. When I tried running bootstrap.bpf.c transpoint, accessing sask_struct.pid via BPF_CORE_READ, the JIT verifier returned error code -EACCES. I suspect this might be a problem with 64-bit register access.
I'm not sure if my understanding is correct, but I suspect that BPF register access is 64-bit, while ARM32 uses 32-bit access, leading to incompatibility. Below are my operating environment and error messages. How can I troubleshoot this type of problem using bpftool? Or is there a way to correct the clang compilation parameters or my BPF program? Thank you very much.

Kernel: linux-5.4.61（BTF）
CLANG-18， -bpf ARM
arm-linux-gnueabihf 7.3

[anakryiko]

yes, this is probably due to 32-bit architecture. You can try declaring

int BPF_PROG(sched_wakeup, u32 p32)
{
    struct task_struct *p =(void*)(long)p32;
   ...
}

That should help with reading 32-bit pointer, I think.

[LinuxZrc]

@anakryiko
thank you for replay，I believe I have found the cause of the problem，It is a usage error of "tp" and "tp_raw"，"tp" could not read "struct task_struct *p" directly. After changing the SEC method to ""tp_raw"", the problem was solved. Then I dealt with some compatibility issues， and my bpf program can run now.

old：

SEC("tracepoint/sched/sched_wakeup")
int BPF_PROG(sched_wakeup, struct task_struct *p)
{
	u32 pid, tgid;

	tgid= BPF_CORE_READ(p, tgid);
	pid = BPF_CORE_READ(p, pid);

	return trace_enqueue(p->tgid, p->pid);
}

new：

SEC("raw_tp/sched_wakeup")
int BPF_PROG(sched_wakeup, struct task_struct *p)
{
	u32 pid, tgid;
	
	tgid= BPF_CORE_READ(p, tgid);
	pid = BPF_CORE_READ(p, pid);
	
	return trace_enqueue(tgid, pid);
}

[LinuxZrc]

However, a fatal issue arose, whether I use tp or tp_raw to access schedw_akeup, after running for a short period of time, the kernel will crash. I wrote the simplest program for testing, and the kernel still freezes without any panic information.

#include <vmlinux.h>
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
#include <bpf/bpf_core_read.h>

char LICENSE[] SEC("license") = "Dual BSD/GPL";

SEC("raw_tp/sched_wakeup")
int BPF_PROG(sched_wakeup, struct task_struct *p)
{
	long err = 0;
	int ppid;
	int pid = bpf_get_current_pid_tgid() >> 32;

	err = bpf_probe_read_kernel(&ppid, sizeof(ppid), &p->pid);
	if (err)
	{
		return 0;
	}
	bpf_printk("BPF triggered from : %d. %d\n", pid, ppid);

	return 0;
}

# ./minimal 
libbpf: loading object 'minimal_bpf' from buffer
libbpf: elf: section(2) .symtab, size 216, link 1, flags 0, type=2
libbpf: elf: section(3) raw_tp/sched_wakeup, size 192, link 0, flags 6, type=1
libbpf: sec 'raw_tp/sched_wakeup': found program 'sched_wakeup' at insn offset 0 (0 bytes), code size 24 insns (192 bytes)
libbpf: elf: section(4) license, size 13, link 0, flags 3, type=1
libbpf: license of minimal_bpf is Dual BSD/GPL
libbpf: elf: section(5) .bss, size 4, link 0, flags 3, type=8
libbpf: elf: section(6) .rodata, size 29, link 0, flags 2, type=1
libbpf: elf: section(7) .relraw_tp/sched_wakeup, size 16, link 2, flags 40, type=9
libbpf: elf: section(8) .BTF, size 21067, link 0, flags 0, type=1
libbpf: elf: section(9) .BTF.ext, size 236, link 0, flags 0, type=1
libbpf: looking for externs among 9 symbols...
libbpf: collected 0 externs total
libbpf: map 'minimal_.bss' (global data): at sec_idx 5, offset 0, flags 400.
libbpf: map 0 is "minimal_.bss"
libbpf: map 'minimal_.rodata' (global data): at sec_idx 6, offset 0, flags 80.
libbpf: map 1 is "minimal_.rodata"
libbpf: sec '.relraw_tp/sched_wakeup': collecting relocation for section(3) 'raw_tp/sched_wakeup'
libbpf: sec '.relraw_tp/sched_wakeup': relo #0: insn #17 against '.rodata'
libbpf: prog 'sched_wakeup': found data map 1 (minimal_.rodata, sec 6, off 0) for insn 17
libbpf: object 'minimal_bpf': failed (-22) to create BPF token from '/sys/fs/bpf', skipping optional step...
libbpf: loaded kernel BTF from '/sys/kernel/btf/vmlinux'
libbpf: sec 'raw_tp/sched_wakeup': found 1 CO-RE relocations
libbpf: CO-RE relocating [6] struct task_struct: found target candidate [192] struct task_struct in [vmlinux]
libbpf: CO-RE relocating [6] struct task_struct: found target candidate [17588] struct task_struct in [vmlinux]
libbpf: prog 'sched_wakeup': relo #0: <byte_off> [6] struct task_struct.pid (0:63 @ offset 1192)
libbpf: prog 'sched_wakeup': relo #0: matching candidate #0 <byte_off> [192] struct task_struct.pid (0:63 @ offset 880)
libbpf: prog 'sched_wakeup': relo #0: matching candidate #1 <byte_off> [17588] struct task_struct.pid (0:63 @ offset 880)
libbpf: prog 'sched_wakeup': relo #0: patched insn #7 (ALU/ALU64) imm 1192 -> 880
libbpf: map 'minimal_.bss': created successfully, fd=3
libbpf: map 'minimal_.rodata': created successfully, fd=4
Successfully started! Please run `sudo cat /sys/kernel/debug/tracing/trace_pipe` to see output of the BPF programs.
.............
The semaphore timeout period has expired.