Page MenuHomePhabricator

[BPF] Attribute btf_decl_tag("ctx") for structs
DraftPublic

Authored by eddyz87 on Sep 6 2022, 8:31 AM.
This is a draft revision that has not yet been submitted for review.

Details

Reviewers
aaron.ballman
Summary

This commit adds a new BPF specific structure attribte
__attribute__((btf_decl_tag("ctx"))) and a pass to deal with it.

This attribute may be attached to a struct or union declaration, where
it notifies the compiler that this structure is a "context" structure.
The following limitations apply to context structures:

  • runtime environment might patch access to the fields of this type by updating the field offset;

    BPF verifier limits access patterns allowed for certain data types. E.g. struct __sk_buff and struct bpf_sock_ops. For these types only LD/ST <reg> <static-offset> memory loads and stores are allowed.

    This is so because offsets of the fields of these structures do not match real offsets in the running kernel. During BPF program load/verification loads and stores to the fields of these types are rewritten so that offsets match real offsets. For this rewrite to happen static offsets have to be encoded in the instructions.

    See kernel/bpf/verifier.c:convert_ctx_access function in the Linux kernel source tree for details.
  • runtime environment might disallow access to the field of the type through modified pointers.

    During BPF program verification a tag PTR_TO_CTX is tracked for register values. In case if register with such tag is modified BPF programs are not allowed to read or write memory using register. See kernel/bpf/verifier.c:check_mem_access function in the Linux kernel source tree for details.

Access to the structure fields is translated to IR as a sequence:

  • (load (getelementptr %ptr %offset)) or
  • (store (getelementptr %ptr %offset))

During instruction selection phase such sequences are translated as a
single load instruction with embedded offset, e.g. LDW %ptr, %offset,
which matches access pattern necessary for the restricted
set of types described above (when %offset is static).

Multiple optimizer passes might separate these instructions, this
includes:

  • SimplifyCFGPass (sinking)
  • InstCombine (sinking)
  • GVN (hoisting)

The btf_decl_tag("ctx") attribute marks structures for which the
following transformations happen:

  • at the early IR processing stage:
    • (load (getelementptr ...)) replaced by call to intrinsic llvm.bpf.getelementptr.and.load;
    • (store (getelementptr ...)) replaced by call to intrinsic llvm.bpf.getelementptr.and.store;
  • at the late IR processing stage this modification is undone.

Such handling prevents various optimizer passes from generating
sequences of instructions that would be rejected by BPF verifier.

The attribute((btf_decl_tag("ctx"))) has a priority over
attribute((preserve_access_index)). When "ctx" attribute is
present preserve access index transformations are not applied.

To ensure this, handling of the BTFDeclTagAttr in SemaDeclAttr.cpp is
updated to recursively propagate "ctx" attribute to nested structure
declarations as it is done for BPFPreserveAccessIndexAttr.

This addresses the issue reported by the following thread:

https://lore.kernel.org/bpf/CAA-VZPmxh8o8EBcJ=m-DH4ytcxDFmo0JKsm1p1gf40kS0CE3NQ@mail.gmail.com/T/#m4b9ce2ce73b34f34172328f975235fc6f19841b6

Diff Detail

Event Timeline

eddyz87 created this revision.Sep 6 2022, 8:31 AM
Herald added a project: Restricted Project. · View Herald Transcript
eddyz87 updated this revision to Diff 459560.Sep 12 2022, 2:32 PM
  • Added folding for GEP chains with constant indices:
    • inline anonymous unions work as expected
    • type casts to char* with consequitive array access work as expected
  • Moved context rewrite to a late callback function processing pipeline stage:
    • necessary to run the rewrite after loop unrolling, as rewrite can't handle non-constant indices for non structural GEP chains
eddyz87 updated this revision to Diff 460356.Sep 15 2022, 3:44 AM
  • added GVN pass after context rewrite to handle cases clobbered by marker calls
  • added marker simplification and context rewrite passes for opt pipeline
  • more test cases
eddyz87 updated this revision to Diff 461103.Sep 18 2022, 4:42 PM

Merged simplify and rewrite passes, various adjustments to unclobber CSE and GVN opportunities

eddyz87 updated this revision to Diff 461317.Sep 19 2022, 12:43 PM

Merge getelementptr and get.and.load/store calls for function inlining

eddyz87 updated this revision to Diff 462038.Sep 21 2022, 4:23 PM

Force doesNotAccessMemory for context marker function

eddyz87 updated this revision to Diff 462709.Sep 25 2022, 3:24 AM

Moved readonly/writeonly attrs as properties of call site to handle volatile load and stores