This commit adds a new BPF specific pass named
BPFContextAccessMarkerPass. The goal of this pass is to restraint
the SimplifyCFGPass optimization pass to avoid generation of Kernel
BPF verifier unfriendly context access pattern.
Context access means an access to the context register, R1, the first
parameter of a BPF program. This patch assumes that context parameters
are marked using the following attribute:
__attribute__((btf_decl_tag("ctx")))Kernel BPF verifier only allows accesses of form BASE + static-offset
if BASE is the context register (or it's alias).
For access to function arguments, CLang frontend generates IR that is
translated in the conforming way. However the SimplifyCFGPass might
move some instructions and add intermediate values that would lead to
dynamic base offset computation. This is could be illustrated by the
next example:
struct bpf_sock { ... }
struct bpf_sockopt { ... }
extern int f(int x);
int _getsockopt(struct bpf_sockopt *ctx __attribute__((btf_decl_tag("ctx"))))
{
unsigned g = 0;
switch (ctx->level) {
case 10:
g = f(ctx->sk->family);
break;
case 20:
g = f(ctx->optlen);
break;
}
return g % 2;
}The following (simplified) IR is generated for function _getsockopt:
define dso_local i32 @_getsockopt(ptr noundef %ctx)
...
sw.bb:
%1 = load ptr, ptr %ctx ;; access
%family = getelementptr ;; to ctx->sk->family
inbounds %struct.bpf_sock, ptr %1 ;; (a)
%2 = load i32, ptr %family ;;
%call = call i32 @f(i32 noundef %2)
br label %sw.epilog
sw.bb1:
%optlen = getelementptr ;; access
inbounds %struct.bpf_sockopt, ptr %ctx ;; to ctx->optlen
%3 = load i32, ptr %optlen ;; (b)
%call2 = call i32 @f(i32 noundef %3)
br label %sw.epilog
sw.epilog:
...W/o SimplifyCFGPass machine code for (a) and (b) looks as follows:
$r1 = LDW $r1, 4 ;; for ctx->sk->family $r1 = LDW $r1, 12 ;; for ctx->optlen
Which is allowed by BPF verifier. However, if SimplifyCFGPass is
executed the code is transformed as follows:
...
sw.bb:
%1 = load ptr, ptr %ctx
%family = getelementptr inbounds %struct.bpf_sock, ptr %1
br label %sw.epilog.sink.split
sw.bb1:
%optlen = getelementptr inbounds %struct.bpf_sockopt, ptr %ctx
br label %sw.epilog.sink.split
sw.epilog.sink.split:
%optlen.sink = phi
ptr [ %optlen, %sw.bb1 ], [ %family, %sw.bb ]
%2 = load i32, ptr %optlen.sink ;; (c)
%call2 = call fastcc i32 @f(i32 noundef %2)
br label %sw.epilog
sw.epilog:
...Note that load instructions (a) and (b) are replaced by a single load
instruction (c) that gets it's value from a PHI node. This is done by
a code sinking part of the SimplifyCFGPass. This leads to the
following machine code:
bb.2.sw.bb: $r1 = LDD $r1, 0 $r1 = ADD_ri $r1, 4 JMP %bb.4 bb.3.sw.bb1: $r1 = ADD_ri $r1, 12 bb.4.sw.epilog.sink.split: $r1 = LDW $r1, 0 JAL @f
Here the offset is dynamically added to r1 (context register), this
access pattern is not allowed by BPF verifier.
To prevent the undesired code motion the BPFContextAccessMarkerPass
inserts a call to llvm.bpf.passthrough function after each load of a
value from location that aliases the context parameter.
llvm.bpf.passthrough accepts a unique integer constant as one of the
parameters, thus preventing the common code sinking after certain
position.
The IR from above is transformed as follows:
sw.bb: %2 = load ptr, ptr %ctx, align 8 %family = getelementptr inbounds %struct.bpf_sock, ptr %2 %3 = load i32, ptr %family, align 4 %4 = call i32 @llvm.bpf.passthrough.i32.i32(i32 2, i32 %3) ;; <-- added call %call = call i32 @f(i32 noundef %4) br label %sw.epilog sw.bb1: %optlen = getelementptr inbounds %struct.bpf_sockopt, ptr %ctx %5 = load i32, ptr %optlen %6 = call i32 @llvm.bpf.passthrough.i32.i32(i32 3, i32 %5) ;; <-- added call %call2 = call i32 @f(i32 noundef %6) br label %sw.epilog sw.epilog: ...
This addresses the issue reported by the following thread: