[BPF] Preserve and make bitfield access relocatable

[yonghong-song]

Summary

Previous work for BPF CO-RE (compile once, run everywhere)
tries to record structure, union and array accesses
in order to make addresses relocatable. Bitfield is
not handled as you cannot really take an address of
a bitfield.

Internally, for any bitfield access, llvm will generate
a GEP to the first bitfield for a contiguous run of
bitfields, and then perform proper load and bit
manipulation from that address.

In this patch, a clang intrinsic is introduced:

void * __builtin_preserve_bitfield_info(expr, unsigned *buf)

Given a bitfield access, the builtin will return the
address of the first bitfield for the contiguous run
of bitfields. The "buf" will have

. the signness of the bitfield
. the size of bitfield
. the offset within the contiguous run of bitfields

These information will be sufficient for bpf program
to retrieve or assign the bitfield values.

The BPF backend will generate an offset relocation
for the leading bitfield and also record all bitfield
accesses happening in this group. So if structure changed
outside the bitfield group, relocation will handle it.
If the structure changes inside the bitfield group,
the bpf loader may reject the program if it deems previous
bitfield extraction might get the wrong result.

For the case of bitfield access without
builtin_preserve_bitfield_info() to get an address,
builtin_preserve_access_index() can be used to enclose
the code. The same offset relocation will be generated.

Please see added tests on how to use the new intrinsic and
what the new offset relocation looks like.

TODO: due to IR intrinsic __builtin_preserve_struct_access_index()
signature change, most existing CORE tests failed. Will fix it later.

[yonghong-song]

fix a bug like foo(..., bar(&value), value) where value is set in function bar. This is not right and depends on compiler optimization as there is not guarantee that arguments are evaluated from left to right. The new code looks like a = bar(&value); foo(..., a, value) ...

[yonghong-song]

We will take a different approach. So abandon this revision.

clang/docs/LanguageExtensions.rst

union {
       int a;
       int b;
     } c[4];
   };
   struct t *v = ...;
   int *pb =__builtin_preserve_access_index(&v->c[3].b);
   __builtin_preserve_access_index(v->j);
 
+``__builtin_preserve_bitfield_info``
+------------------------------------
+
+``__builtin_preserve_bitfield_info`` permits the structure bitfield access
+to be relocatable/verifiable under bpf compile-once run-everywhere framework.
+Debuginfo (typically with ``-g``) is needed, otherwise, the compiler will
+exit with an error. The intrinsic returns the memory address of the first
+bitfield in the group of contiguous run of bitfields including the one
+expressed in the first argument. The second argument returns the following
+information which can be used by application for a generic way to extract
+the bitfield values. The semantics of these fields are similar to the
+definition in ``clang::CodeGen::CGFieldInfo``.
+ * the signness of the bitfield
+ * the size of the bitfield
+ * the offset within the contiguous run of bitfields
+
+**Syntax**:
+
+.. code-block:: c
+
+  void * __builtin_preserve_bitfield_info(expr, unsigned *buf)
+
+**Example of Use**:
+
+.. code-block:: c
+
+  struct t {
+    int a;
+    int f1:8;
+    int f2:1;
+    int f3:2;
+    int b;
+  };
+  struct t *v = ...;
+  unsigned info;
+  void *p =__builtin_preserve_bitfield_info(v->f3, &info);
+  // p = (void *)v + 4, pointing to field f1
+  // signness = info >> 31
+  // size = (info >> 16) & 0x7fff
+  // offset = info & 0xffff
+
 Multiprecision Arithmetic Builtins
 ----------------------------------
 
 Clang provides a set of builtins which expose multiprecision arithmetic in a
 manner amenable to C. They all have the following form:
 
 .. code-block:: c
 

clang/include/clang/Basic/Builtins.def

 
 // Clang builtins (not available in GCC).
 BUILTIN(__builtin_addressof, "v*v&", "nct")
 BUILTIN(__builtin_operator_new, "v*z", "tc")
 BUILTIN(__builtin_operator_delete, "vv*", "tn")
 BUILTIN(__builtin_char_memchr, "c*cC*iz", "n")
 BUILTIN(__builtin_dump_struct, "ivC*v*", "tn")
 BUILTIN(__builtin_preserve_access_index, "v.", "t")
+BUILTIN(__builtin_preserve_bitfield_info, "v*.", "t")
 
 // Safestack builtins
 BUILTIN(__builtin___get_unsafe_stack_start, "v*", "Fn")
 BUILTIN(__builtin___get_unsafe_stack_bottom, "v*", "Fn")
 BUILTIN(__builtin___get_unsafe_stack_top, "v*", "Fn")
 BUILTIN(__builtin___get_unsafe_stack_ptr, "v*", "Fn")
 
 // Nontemporal loads/stores builtins

clang/include/clang/Basic/DiagnosticSemaKinds.td

 def err_builtin_launder_invalid_arg : Error<
   "%select{non-pointer|function pointer|void pointer}0 argument to "
   "'__builtin_launder' is not allowed">;
 
 def err_bit_cast_non_trivially_copyable : Error<
   "__builtin_bit_cast %select{source|destination}0 type must be trivially copyable">;
 def err_bit_cast_type_size_mismatch : Error<
   "__builtin_bit_cast source size does not equal destination size (%0 vs %1)">;
+
+def err_preserve_bitfield_info_not_bitfield : Error<
+  "__builtin_preserve_bitfield_info arg %0 not a bitfield access">;
+def err_preserve_bitfield_info_not_ptr: Error<
+  "__builtin_preserve_bitfield_info arg %0 not a pointer">;
+def err_preserve_bitfield_info_ptr_invalid_align: Error<
+  "__builtin_preserve_bitfield_info arg %0 must be a pointer"
+  " with memory alignment of %1-bit">;
+def err_preserve_bitfield_info_ptr_invalid_size: Error<
+  "__builtin_preserve_bitfield_info arg %0 must be a pointer"
+  " with pointee size of %1-bit instead of %2-bit">;
 } // end of sema component.

clang/lib/CodeGen/CGBuilder.h

llvm::CallInst *CreateMemSet(Address Dest, llvm::Value *Value,
     return CreateMemSet(Dest.getPointer(), Value, Size,
                         Dest.getAlignment().getQuantity(), IsVolatile);
   }
 
   using CGBuilderBaseTy::CreatePreserveStructAccessIndex;
   Address CreatePreserveStructAccessIndex(Address Addr,
                                           unsigned Index,
                                           unsigned FieldIndex,
+                                          unsigned PreservedIndex,
                                           llvm::MDNode *DbgInfo) {
     llvm::StructType *ElTy = cast<llvm::StructType>(Addr.getElementType());
     const llvm::DataLayout &DL = BB->getParent()->getParent()->getDataLayout();
     const llvm::StructLayout *Layout = DL.getStructLayout(ElTy);
     auto Offset = CharUnits::fromQuantity(Layout->getElementOffset(Index));
 
     return Address(CreatePreserveStructAccessIndex(Addr.getPointer(),
-                                                   Index, FieldIndex, DbgInfo),
+                                                   Index, FieldIndex,
+                                                   PreservedIndex,
+                                                   DbgInfo),
                    Addr.getAlignment().alignmentAtOffset(Offset));
   }
 };
 
 }  // end namespace CodeGen
 }  // end namespace clang
 
 #endif

clang/lib/CodeGen/CGBuiltin.cpp

case Builtin::BI__builtin_preserve_access_index: {
     // Only enabled preserved access index region when debuginfo
     // is available as debuginfo is needed to preserve user-level
     // access pattern.
     if (!getDebugInfo()) {
       CGM.Error(E->getExprLoc(), "using builtin_preserve_access_index() without -g");
       return RValue::get(EmitScalarExpr(E->getArg(0)));
     }
 
-    // Nested builtin_preserve_access_index() not supported
+    // Nested builtin_preserve_{access_index, bitfield_info}() not supported
     if (IsInPreservedAIRegion) {
-      CGM.Error(E->getExprLoc(), "nested builtin_preserve_access_index() not supported");
+      CGM.Error(E->getExprLoc(), "nested builtin_preserve_*() not supported");
       return RValue::get(EmitScalarExpr(E->getArg(0)));
     }
 
     IsInPreservedAIRegion = true;
     Value *Res = EmitScalarExpr(E->getArg(0));
     IsInPreservedAIRegion = false;
     return RValue::get(Res);
   }
 
+  case Builtin::BI__builtin_preserve_bitfield_info: {
+    if (!getDebugInfo()) {
+      CGM.Error(E->getExprLoc(), "using builtin_preserve_bitfield_info() without -g");
+      return RValue::get(EmitLValue(E->getArg(0)).getBitFieldPointer());
+    }
+
+    if (IsInPreservedAIRegion) {
+      CGM.Error(E->getExprLoc(), "nested builtin_preserve_*() not supported");
+      return RValue::get(EmitLValue(E->getArg(0)).getBitFieldPointer());
+    }
+
+    IsInPreservedAIRegion = true;
+
+    // Emit relocation for the first member of bitfield group.
+    LValue LV = EmitLValue(E->getArg(0));
+    Value *BitFieldPtr = LV.getBitFieldPointer();
+    unsigned AS = cast<llvm::PointerType>(BitFieldPtr->getType())->getAddressSpace();
+    Value *Res = Builder.CreateBitCast(BitFieldPtr, Int8Ty->getPointerTo(AS));
+
+    // Store the bitfield info in user memory.
+    const CGBitFieldInfo &Info = LV.getBitFieldInfo();
+    uint32_t Val = Info.IsSigned << 31 |
+                   Info.Size << 16 |
+                   Info.Offset;
+    Value *InfoVal = ConstantInt::get(Int32Ty, Val);
+    Address InfoPtr = EmitPointerWithAlignment(E->getArg(1));
+    llvm::PointerType *Int32PtrTy =
+        Int32Ty->getPointerTo(InfoPtr.getAddressSpace());
+    Builder.CreateStore(InfoVal,
+                        Builder.CreateBitCast(InfoPtr, Int32PtrTy));
+
+    IsInPreservedAIRegion = false;
+    return RValue::get(Res);
+  }
+
   case Builtin::BI__builtin_cimag:
   case Builtin::BI__builtin_cimagf:
   case Builtin::BI__builtin_cimagl:
   case Builtin::BIcimag:
   case Builtin::BIcimagf:
   case Builtin::BIcimagl: {
     ComplexPairTy ComplexVal = EmitComplexExpr(E->getArg(0));
     return RValue::get(ComplexVal.second);

clang/lib/CodeGen/CGExpr.cpp

LValue CodeGenFunction::EmitLValueForLambdaField(const FieldDecl *Field) {
   assert(cast<CXXMethodDecl>(CurCodeDecl)->getParent() == Field->getParent());
   QualType LambdaTagType =
     getContext().getTagDeclType(Field->getParent());
   LValue LambdaLV = MakeNaturalAlignAddrLValue(CXXABIThisValue, LambdaTagType);
   return EmitLValueForField(LambdaLV, Field);
 }
 
 /// Get the field index in the debug info. The debug info structure/union
-/// will ignore the unnamed bitfields.
+/// will ignore the unnamed bitfields. The \p PreservedIndex is the
+/// field index recorded as relocation for structures.
 unsigned CodeGenFunction::getDebugInfoFIndex(const RecordDecl *Rec,
-                                             unsigned FieldIndex) {
-  unsigned I = 0, Skipped = 0;
+                                             unsigned FieldIndex,
+                                             unsigned &PreservedIndex) {
+  unsigned I = 0, Skipped = 0, BitFieldsInGroup = 0;
 
   for (auto F : Rec->getDefinition()->fields()) {
     if (I == FieldIndex)
       break;
     if (F->isUnnamedBitfield())
       Skipped++;
+    else
+      BitFieldsInGroup = F->isBitField() ? BitFieldsInGroup + 1 : 0;
     I++;
   }
 
+  PreservedIndex = FieldIndex - Skipped - BitFieldsInGroup;
   return FieldIndex - Skipped;
 }
 
 /// Get the address of a zero-sized field within a record. The resulting
 /// address doesn't necessarily have the right type.
 static Address emitAddrOfZeroSizeField(CodeGenFunction &CGF, Address Base,
                                        const FieldDecl *Field) {
   CharUnits Offset = CGF.getContext().toCharUnitsFromBits(
static Address emitPreserveStructAccess(CodeGenFunction &CGF, Address base,
                                         const FieldDecl *field) {
   const RecordDecl *rec = field->getParent();
   llvm::DIType *DbgInfo = CGF.getDebugInfo()->getOrCreateRecordType(
       CGF.getContext().getRecordType(rec), rec->getLocation());
 
   unsigned idx =
       CGF.CGM.getTypes().getCGRecordLayout(rec).getLLVMFieldNo(field);
 
-  return CGF.Builder.CreatePreserveStructAccessIndex(
-      base, idx, CGF.getDebugInfoFIndex(rec, field->getFieldIndex()), DbgInfo);
+  unsigned FieldIndex = field->getFieldIndex(), PreservedIndex;
+  unsigned DIFieldIndex = CGF.getDebugInfoFIndex(rec, FieldIndex,
+                                                 PreservedIndex);
+  return CGF.Builder.CreatePreserveStructAccessIndex(base, idx,
+      DIFieldIndex, PreservedIndex, DbgInfo);
 }
 
 static bool hasAnyVptr(const QualType Type, const ASTContext &Context) {
   const auto *RD = Type.getTypePtr()->getAsCXXRecordDecl();
   if (!RD)
     return false;
 
   if (RD->isDynamicClass())
LValue CodeGenFunction::EmitLValueForField(LValue base,
   LValueBaseInfo BaseInfo = base.getBaseInfo();
 
   if (field->isBitField()) {
     const CGRecordLayout &RL =
       CGM.getTypes().getCGRecordLayout(field->getParent());
     const CGBitFieldInfo &Info = RL.getBitFieldInfo(field);
     Address Addr = base.getAddress();
     unsigned Idx = RL.getLLVMFieldNo(field);
+    if (!IsInPreservedAIRegion) {
       if (Idx != 0)
         // For structs, we GEP to the field that the record layout suggests.
         Addr = Builder.CreateStructGEP(Addr, Idx, field->getName());
+    } else {
+       const RecordDecl *rec = field->getParent();
+       llvm::DIType *DbgInfo = getDebugInfo()->getOrCreateRecordType(
+           getContext().getRecordType(rec), rec->getLocation());
+
+       // Preserve access index for the first bitfield member
+       // in the contiguous run of bitfields with this field.
+       unsigned FieldIndex = field->getFieldIndex();
+       unsigned PreservedIndex;
+       unsigned DIFieldIndex = getDebugInfoFIndex(rec, FieldIndex,
+                                                  PreservedIndex);
+       Addr = Builder.CreatePreserveStructAccessIndex(Addr, Idx,
+           DIFieldIndex, PreservedIndex, DbgInfo);
+    }
+
     // Get the access type.
     llvm::Type *FieldIntTy =
       llvm::Type::getIntNTy(getLLVMContext(), Info.StorageSize);
     if (Addr.getElementType() != FieldIntTy)
       Addr = Builder.CreateElementBitCast(Addr, FieldIntTy);
 
     QualType fieldType =
       field->getType().withCVRQualifiers(base.getVRQualifiers());
if (CGM.getCodeGenOpts().StrictVTablePointers &&
       // a barrier every time CXXRecord field with vptr is referenced.
       addr = Address(Builder.CreateLaunderInvariantGroup(addr.getPointer()),
                      addr.getAlignment());
 
     if (IsInPreservedAIRegion) {
       // Remember the original union field index
       llvm::DIType *DbgInfo = getDebugInfo()->getOrCreateRecordType(
           getContext().getRecordType(rec), rec->getLocation());
+      unsigned PreservedIndex;
       addr = Address(
           Builder.CreatePreserveUnionAccessIndex(
-              addr.getPointer(), getDebugInfoFIndex(rec, field->getFieldIndex()), DbgInfo),
+              addr.getPointer(),
+              getDebugInfoFIndex(rec, field->getFieldIndex(), PreservedIndex),
+              DbgInfo),
           addr.getAlignment());
     }
 
     if (FieldType->isReferenceType())
       addr = Builder.CreateElementBitCast(
           addr, CGM.getTypes().ConvertTypeForMem(FieldType), field->getName());
   } else {
     if (!IsInPreservedAIRegion)

clang/lib/CodeGen/CodeGenFunction.h

   llvm::Value *EmitScalarPrePostIncDec(const UnaryOperator *E, LValue LV,
                                        bool isInc, bool isPre);
   ComplexPairTy EmitComplexPrePostIncDec(const UnaryOperator *E, LValue LV,
                                          bool isInc, bool isPre);
 
   /// Converts Location to a DebugLoc, if debug information is enabled.
   llvm::DebugLoc SourceLocToDebugLoc(SourceLocation Location);
 
-  /// Get the record field index as represented in debug info.
-  unsigned getDebugInfoFIndex(const RecordDecl *Rec, unsigned FieldIndex);
-
+  /// Get the record field index as represented in debug info and
+  /// the record field index used for relocation.
+  unsigned getDebugInfoFIndex(const RecordDecl *Rec, unsigned FieldIndex,
+                              unsigned &PreservedIndex);
 
   //===--------------------------------------------------------------------===//
   //                            Declaration Emission
   //===--------------------------------------------------------------------===//
 
   /// EmitDecl - Emit a declaration.
   ///
   /// This function can be called with a null (unreachable) insert point.

clang/lib/Sema/SemaChecking.cpp

 static bool SemaBuiltinPreserveAI(Sema &S, CallExpr *TheCall) {
   if (checkArgCount(S, TheCall, 1))
     return true;
 
   TheCall->setType(TheCall->getArg(0)->getType());
   return false;
 }
 
+static bool SemaBuiltinPreserveBI(Sema &S, CallExpr *TheCall) {
+  if (checkArgCount(S, TheCall, 2))
+    return true;
+
+  // The first argument must be a bitfield access
+  Expr *Arg = TheCall->getArg(0);
+  if (Arg->getType()->getAsPlaceholderType() ||
+      Arg->IgnoreParens()->getObjectKind() != OK_BitField) {
+    S.Diag(Arg->getBeginLoc(), diag::err_preserve_bitfield_info_not_bitfield)
+        << 1 /* first argument */ << Arg->getSourceRange();
+    return true;
+  }
+
+  // The second argument must be a pointer and aligned at 32-bit
+  // boundary as later a 32-bit store insn will be generated.
+  // Note that higher alignment (e.g. 64-bit) is okay.
+  // The size of pointee must be 32-bit.
+  Arg = TheCall->getArg(1);
+  QualType ArgType = Arg->getType();
+  if (ArgType->isPointerType()) {
+    bool IsAligned = false;
+    QualType Pointee = ArgType->getPointeeType();
+    unsigned IntAlign = S.Context.getTargetInfo().getIntAlign();
+    if (!Pointee->isIncompleteType()) {
+      unsigned PointeeAlign = S.Context.getTypeAlign(Pointee);
+      if (!(PointeeAlign & (IntAlign - 1)))
+        IsAligned = true;
+    }
+    if (!IsAligned) {
+      S.Diag(Arg->getBeginLoc(), diag::err_preserve_bitfield_info_ptr_invalid_align)
+          << 2 /* second argument */
+          << IntAlign
+          << Arg->getSourceRange();
+      return true;
+    }
+
+    unsigned IntSize = S.Context.getTargetInfo().getIntWidth();
+    unsigned PointeeSize = S.Context.getTypeSize(Pointee);
+    if (IntSize != PointeeSize) {
+      S.Diag(Arg->getBeginLoc(), diag::err_preserve_bitfield_info_ptr_invalid_size)
+          << 2 /* second argument */
+          << IntSize << PointeeSize
+          << Arg->getSourceRange();
+      return true;
+    }
+  } else {
+    S.Diag(Arg->getBeginLoc(), diag::err_preserve_bitfield_info_not_ptr)
+        << 2 /* second argument */ << Arg->getSourceRange();
+    return true;
+  }
+
+  TheCall->setType(S.Context.VoidPtrTy);
+  return false;
+}
+
 static bool SemaBuiltinOverflow(Sema &S, CallExpr *TheCall) {
   if (checkArgCount(S, TheCall, 3))
     return true;
 
   // First two arguments should be integers.
   for (unsigned I = 0; I < 2; ++I) {
     ExprResult Arg = TheCall->getArg(I);
     QualType Ty = Arg.get()->getType();
case Builtin::BI__builtin_dump_struct: {
 
     TheCall->setType(Context.IntTy);
     break;
   }
   case Builtin::BI__builtin_preserve_access_index:
     if (SemaBuiltinPreserveAI(*this, TheCall))
       return ExprError();
     break;
+  case Builtin::BI__builtin_preserve_bitfield_info:
+    if (SemaBuiltinPreserveBI(*this, TheCall))
+      return ExprError();
+    break;
   case Builtin::BI__builtin_call_with_static_chain:
     if (SemaBuiltinCallWithStaticChain(*this, TheCall))
       return ExprError();
     break;
   case Builtin::BI__exception_code:
   case Builtin::BI_exception_code:
     if (SemaBuiltinSEHScopeCheck(*this, TheCall, Scope::SEHExceptScope,
                                  diag::err_seh___except_block))

clang/test/CodeGen/builtin-preserve-bitfield-info-2.c

+// RUN: %clang -target x86_64 -emit-llvm -O2 -S -g %s -o - | FileCheck %s
+
+struct s {
+  int a;
+  int b1:8;
+  int b2:3;
+  int b3:4;
+};
+
+#define _(x, y) __builtin_preserve_bitfield_info((x), (y))
+void process(void *);
+unsigned test(struct s *arg) {
+  unsigned info1, info2;
+  void *p1, *p2;
+
+  p1 = _(arg->b2, &info1);
+  p2 = _(arg->b3, &info2);
+  process(p1);
+  process(p2);
+  // the last 16 bit is the offset, so
+  // return value should be 8 + 11 = 19
+  // At -O2, compiler should do this optimization.
+  return (info1 & 0xffff) + (info2 & 0xffff);
+}
+
+// CHECK: define dso_local i32 @test
+// CHECK: tail call i16* @llvm.preserve.struct.access.index.p0i16.p0s_struct.ss(%struct.s* %{{[0-9a-z]+}}, i32 1, i32 2, i32 1), !dbg !{{[0-9]+}}, !llvm.preserve.access.index ![[STRUCT_S:[0-9]+]]
+// CHECK: tail call i16* @llvm.preserve.struct.access.index.p0i16.p0s_struct.ss(%struct.s* %{{[0-9a-z]+}}, i32 1, i32 3, i32 1), !dbg !{{[0-9]+}}, !llvm.preserve.access.index ![[STRUCT_S:[0-9]+]]
+// CHECK: ret i32 19
+//
+// CHECK: ![[STRUCT_S]] = distinct !DICompositeType(tag: DW_TAG_structure_type, name: "s"

clang/test/CodeGen/builtin-preserve-bitfield-info.c

+// RUN: %clang -target x86_64 -emit-llvm -S -g %s -o - | FileCheck %s
+
+struct s {
+  int a;
+  int b1:8;
+  int b2:3;
+  int b3:4;
+  int c;
+};
+
+extern int bpf_probe_read(void *, unsigned, void *);
+int test(struct s *arg, int generic) {
+  unsigned info, sign, size, offset;
+  union {
+    unsigned char uc;
+    char sc;
+  } v;
+  void *p, *s;
+  int ret;
+  char c;
+
+  p = __builtin_preserve_bitfield_info(arg->b3, &info);
+  if (generic) {
+    // try to be generic
+    // Assume all bitfields can be read within a byte.
+    sign = info >> 31;
+    size = (info >> 16) & 0x7fff;
+    offset = info & 0xffff;
+    s = p + (offset >> 3);
+
+    bpf_probe_read(&v, sizeof(v), s);
+    if (sign) {
+      v.sc = v.sc << (offset & 0x7);
+      ret = v.sc >> (8 - size);
+    } else {
+      v.uc = v.uc << (offset & 0x7);
+      ret = v.uc >> (8 - size);
+    }
+  } else {
+    // use specific knowledge for this field
+    bpf_probe_read(&c, sizeof(c), p + 1);
+    c = c << 3;
+    ret = c >> 4;
+  }
+
+  return ret;
+}
+
+// CHECK: define dso_local i32 @test
+// CHECK: call i16* @llvm.preserve.struct.access.index.p0i16.p0s_struct.ss(%struct.s* %{{[0-9a-z]+}}, i32 1, i32 3, i32 1), !dbg !{{[0-9]+}}, !llvm.preserve.access.index ![[STRUCT_S:[0-9]+]]
+// CHECK: store i32 -2147221493, i32* %{{[0-9a-z]+}}
+//
+// CHECK: ![[STRUCT_S]] = distinct !DICompositeType(tag: DW_TAG_structure_type, name: "s"
+

llvm/docs/LangRef.rst

 
 Syntax:
 """""""
 ::
 
       declare <ret_type>
       @llvm.preserve.struct.access.index.p0i8.p0s_struct.anon.0s(<type> base,
                                                                  i32 gep_index,
-                                                                 i32 di_index)
+                                                                 i32 di_index,
+                                                                 i32 preserved_index)
 
 Overview:
 """""""""
 
 The '``llvm.preserve.struct.access.index``' intrinsic returns the getelementptr address
 based on struct base ``base`` and IR struct member index ``gep_index``.
 The ``llvm.preserve.access.index`` type of metadata is attached to this call instruction
 to provide struct debuginfo type.
 The metadata is a ``DICompositeType`` representing the debuginfo version of ``type``.
 The return type ``ret_type`` is a pointer type to the structure member.
 
 Arguments:
 """"""""""
 
 The ``base`` is the structure base address. The ``gep_index`` is the struct member index
 based on IR structures. The ``di_index`` is the struct member index based on debuginfo.
+The ``preserved_index`` is the debuginfo struct member index which corresponds to
+``gep_index``. The ``preserved_index`` is the same as ``di_index`` for non bitfield members.
+For bitfield members, the ``preserved_index`` corresponds to the first bitfield member
+in the contiguous run of bitfields include ``di_index``.
 
 Semantics:
 """"""""""
 
 The '``llvm.preserve.struct.access.index``' intrinsic produces the same result
 as a getelementptr with base ``base`` and access operands ``{0, gep_index}``.

llvm/include/llvm/IR/IRBuilder.h

CallInst *Fn =
         CreateCall(FnPreserveUnionAccessIndex, {Base, DIIndex});
     if (DbgInfo)
       Fn->setMetadata(LLVMContext::MD_preserve_access_index, DbgInfo);
 
     return Fn;
   }
 
   Value *CreatePreserveStructAccessIndex(Value *Base, unsigned Index,
-                                         unsigned FieldIndex, MDNode *DbgInfo) {
+                                         unsigned FieldIndex,
+                                         unsigned PreservedIndex,
+                                         MDNode *DbgInfo) {
     assert(isa<PointerType>(Base->getType()) &&
            "Invalid Base ptr type for preserve.struct.access.index.");
     auto *BaseType = Base->getType();
 
     Value *GEPIndex = getInt32(Index);
     Constant *Zero = ConstantInt::get(Type::getInt32Ty(Context), 0);
     Type *ResultType =
         GetElementPtrInst::getGEPReturnType(Base, {Zero, GEPIndex});
 
     Module *M = BB->getParent()->getParent();
     Function *FnPreserveStructAccessIndex = Intrinsic::getDeclaration(
         M, Intrinsic::preserve_struct_access_index, {ResultType, BaseType});
 
     Value *DIIndex = getInt32(FieldIndex);
+    Value *PreservedDIIndex = getInt32(PreservedIndex);
     CallInst *Fn = CreateCall(FnPreserveStructAccessIndex,
-                              {Base, GEPIndex, DIIndex});
+                              {Base, GEPIndex, DIIndex, PreservedDIIndex});
     if (DbgInfo)
       Fn->setMetadata(LLVMContext::MD_preserve_access_index, DbgInfo);
 
     return Fn;
   }
 
 private:
   /// Helper function that creates an assume intrinsic call that

llvm/include/llvm/IR/Intrinsics.td

def int_preserve_array_access_index : Intrinsic<[llvm_anyptr_ty],
                                                 [llvm_anyptr_ty, llvm_i32_ty,
                                                  llvm_i32_ty],
                                                 [IntrNoMem, ImmArg<1>, ImmArg<2>]>;
 def int_preserve_union_access_index : Intrinsic<[llvm_anyptr_ty],
                                                 [llvm_anyptr_ty, llvm_i32_ty],
                                                 [IntrNoMem, ImmArg<1>]>;
 def int_preserve_struct_access_index : Intrinsic<[llvm_anyptr_ty],
                                                  [llvm_anyptr_ty, llvm_i32_ty,
-                                                  llvm_i32_ty],
+                                                  llvm_i32_ty, llvm_i32_ty],
                                                  [IntrNoMem, ImmArg<1>,
-                                                  ImmArg<2>]>;
+                                                  ImmArg<2>, ImmArg<3>]>;
 
 //===----------------------------------------------------------------------===//
 // Target-specific intrinsics
 //===----------------------------------------------------------------------===//
 
 include "llvm/IR/IntrinsicsPowerPC.td"
 include "llvm/IR/IntrinsicsX86.td"
 include "llvm/IR/IntrinsicsARM.td"

llvm/lib/Target/BPF/BPFAbstractMemberAccess.cpp

 //
 // By repeating the member access on the host, the bpf loader can
 // adjust "r = 4" as "r = 8".
 //
 // This feature relies on the following three intrinsic calls:
 //   addr = preserve_array_access_index(base, dimension, index)
 //   addr = preserve_union_access_index(base, di_index)
 //          !llvm.preserve.access.index <union_ditype>
-//   addr = preserve_struct_access_index(base, gep_index, di_index)
+//   addr = preserve_struct_access_index(base, gep_index, di_index,
+//                                       preserved_index)
 //          !llvm.preserve.access.index <struct_ditype>
 //
 //===----------------------------------------------------------------------===//
 
 #include "BPF.h"
 #include "BPFCORE.h"
 #include "BPFTargetMachine.h"
 #include "llvm/IR/DebugInfoMetadata.h"
 #include "llvm/IR/GlobalVariable.h"
 #include "llvm/IR/Instruction.h"
 #include "llvm/IR/Instructions.h"
 #include "llvm/IR/Module.h"
 #include "llvm/IR/Type.h"
 #include "llvm/IR/User.h"
 #include "llvm/IR/Value.h"
 #include "llvm/Pass.h"
 #include "llvm/Transforms/Utils/BasicBlockUtils.h"
+#include <set>
 #include <stack>
 
 #define DEBUG_TYPE "bpf-abstract-member-access"
 
 namespace llvm {
 const std::string BPFCoreSharedInfo::AmaAttr = "btf_ama";
+const std::string BPFCoreSharedInfo::BitFieldAttr = "btf_bitfield_di_findex";
 const std::string BPFCoreSharedInfo::PatchableExtSecName =
     ".BPF.patchable_externs";
 } // namespace llvm
 
 using namespace llvm;
 
 namespace {
 
private:
 
   std::map<std::string, GlobalVariable *> GEPGlobals;
   // A map to link preserve_*_access_index instrinsic calls.
   std::map<CallInst *, std::pair<CallInst *, uint32_t>> AIChain;
   // A map to hold all the base preserve_*_access_index instrinsic calls.
   // The base call is not an input of any other preserve_*_access_index
   // intrinsics.
   std::map<CallInst *, uint32_t> BaseAICalls;
+  std::map<GlobalVariable *, std::set<uint32_t>> BitfieldAccesses;
 
   bool doTransformation(Module &M);
 
   void traceAICall(CallInst *Call, uint32_t Kind, const MDNode *ParentMeta,
                    uint32_t ParentAI);
   void traceBitCast(BitCastInst *BitCast, CallInst *Parent, uint32_t Kind,
                     const MDNode *ParentMeta, uint32_t ParentAI);
   void traceGEP(GetElementPtrInst *GEP, CallInst *Parent, uint32_t Kind,
                 const MDNode *ParentMeta, uint32_t ParentAI);
   void collectAICallChains(Module &M, Function &F);
 
   bool IsPreserveDIAccessIndexCall(const CallInst *Call, uint32_t &Kind,
                                    const MDNode *&TypeMeta, uint32_t &AccessIndex);
   bool IsValidAIChain(const MDNode *ParentMeta, uint32_t ParentAI,
                       const MDNode *ChildMeta);
   bool removePreserveAccessIndexIntrinsic(Module &M);
   void replaceWithGEP(std::vector<CallInst *> &CallList,
                       uint32_t NumOfZerosIndex, uint32_t DIIndex);
 
   Value *computeBaseAndAccessKey(CallInst *Call, std::string &AccessKey,
+                                 uint32_t &DIFieldIndex,
                                  uint32_t Kind, MDNode *&BaseMeta);
   bool getAccessIndex(const Value *IndexValue, uint64_t &AccessIndex);
   bool transformGEPChain(Module &M, CallInst *Call, uint32_t Kind);
 };
 } // End anonymous namespace
 
 char BPFAbstractMemberAccess::ID = 0;
 INITIALIZE_PASS(BPFAbstractMemberAccess, DEBUG_TYPE,
bool BPFAbstractMemberAccess::removePreserveAccessIndexIntrinsic(Module &M) {
 
   // do the following transformation:
   // . addr = preserve_array_access_index(base, dimension, index)
   //   is transformed to
   //     addr = GEP(base, dimenion's zero's, index)
   // . addr = preserve_union_access_index(base, di_index)
   //   is transformed to
   //     addr = base, i.e., all usages of "addr" are replaced by "base".
-  // . addr = preserve_struct_access_index(base, gep_index, di_index)
+  // . addr = preserve_struct_access_index(base, gep_index, di_index,
+  //                                       preserved_index)
   //   is transformed to
   //     addr = GEP(base, 0, gep_index)
   replaceWithGEP(PreserveArrayIndexCalls, 1, 2);
   replaceWithGEP(PreserveStructIndexCalls, 0, 1);
   for (auto Call : PreserveUnionIndexCalls) {
     Call->replaceAllUsesWith(Call->getArgOperand(0));
     Call->eraseFromParent();
   }
bool BPFAbstractMemberAccess::getAccessIndex(const Value *IndexValue,
   return true;
 }
 
 /// Compute the base of the whole preserve_*_access_index chains, i.e., the base
 /// pointer of the first preserve_*_access_index call, and construct the access
 /// string, which will be the name of a global variable.
 Value *BPFAbstractMemberAccess::computeBaseAndAccessKey(CallInst *Call,
                                                         std::string &AccessKey,
+                                                        uint32_t &DIFieldIndex,
                                                         uint32_t Kind,
                                                         MDNode *&TypeMeta) {
   Value *Base = nullptr;
   std::string TypeName;
   std::stack<std::pair<CallInst *, uint32_t>> CallStack;
 
   // Put the access chain into a stack with the top as the head of the chain.
   while (Call) {
Value *BPFAbstractMemberAccess::computeBaseAndAccessKey(CallInst *Call,
   while (CallStack.size()) {
     auto StackElem = CallStack.top();
     Call = StackElem.first;
     Kind = StackElem.second;
     CallStack.pop();
 
     // Access Index
     uint64_t AccessIndex;
-    uint32_t ArgIndex = (Kind == BPFPreserveUnionAI) ? 1 : 2;
+    uint32_t ArgIndex;
+    if (Kind == BPFPreserveUnionAI)
+      ArgIndex = 1;
+    else if (Kind == BPFPreserveStructAI)
+      ArgIndex = 3;
+    else
+      ArgIndex = 2;
+
     if (!getAccessIndex(Call->getArgOperand(ArgIndex), AccessIndex))
       return nullptr;
     AccessKey += ":" + std::to_string(AccessIndex);
 
     MDNode *MDN = Call->getMetadata(LLVMContext::MD_preserve_access_index);
     // At this stage, it cannot be pointer type.
     auto *CTy = cast<DICompositeType>(stripQualifiers(cast<DIType>(MDN)));
     uint32_t Tag = CTy->getTag();
     if (Tag == dwarf::DW_TAG_structure_type) {
       auto *MemberTy = cast<DIDerivedType>(CTy->getElements()[AccessIndex]);
       AccessOffset += MemberTy->getOffsetInBits() >> 3;
     } else if (Tag == dwarf::DW_TAG_array_type) {
       auto *EltTy = stripQualifiers(CTy->getBaseType());
       AccessOffset += AccessIndex * calcArraySize(CTy, 1) *
                       EltTy->getSizeInBits() >> 3;
     }
+
+    if (Kind == BPFPreserveStructAI) {
+      ArgIndex = 2;
+      uint64_t DbgFieldIndex;
+      if (!getAccessIndex(Call->getArgOperand(ArgIndex), DbgFieldIndex))
+        return nullptr;
+      if (DbgFieldIndex != AccessIndex) {
+        // A bitfield access which is not the leading bitfield in
+        // the bitfield group.
+        DIFieldIndex = DbgFieldIndex;
+      }
+    }
   }
 
   // Access key is the type name + access string, uniquely identifying
   // one kernel memory access.
   AccessKey = TypeName + ":" + std::to_string(AccessOffset) + "$" + AccessKey;
 
   return Base;
 }
 
 /// Call/Kind is the base preserve_*_access_index() call. Attempts to do
 /// transformation to a chain of relocable GEPs.
 bool BPFAbstractMemberAccess::transformGEPChain(Module &M, CallInst *Call,
                                                 uint32_t Kind) {
   std::string AccessKey;
   MDNode *TypeMeta;
+  uint32_t DIFieldIndex = 0;
   Value *Base =
-      computeBaseAndAccessKey(Call, AccessKey, Kind, TypeMeta);
+      computeBaseAndAccessKey(Call, AccessKey, DIFieldIndex, Kind, TypeMeta);
   if (!Base)
     return false;
 
   // Do the transformation
   // For any original GEP Call and Base %2 like
   //   %4 = bitcast %struct.net_device** %dev1 to i64*
   // it is transformed to:
   //   %6 = load sk_buff:50:$0:0:0:2:0
if (GEPGlobals.find(AccessKey) == GEPGlobals.end()) {
     GV = new GlobalVariable(M, Type::getInt64Ty(BB->getContext()), false,
                             GlobalVariable::ExternalLinkage, NULL, AccessKey);
     GV->addAttribute(BPFCoreSharedInfo::AmaAttr);
     GV->setMetadata(LLVMContext::MD_preserve_access_index, TypeMeta);
     GEPGlobals[AccessKey] = GV;
   } else {
     GV = GEPGlobals[AccessKey];
   }
+  if (DIFieldIndex)
+    BitfieldAccesses[GV].insert(DIFieldIndex);
 
   // Load the global variable.
   auto *LDInst = new LoadInst(Type::getInt64Ty(BB->getContext()), GV);
   BB->getInstList().insert(Call->getIterator(), LDInst);
 
   // Generate a BitCast
   auto *BCInst = new BitCastInst(Base, Type::getInt8PtrTy(BB->getContext()));
   BB->getInstList().insert(Call->getIterator(), BCInst);
for (Function &F : M) {
     // The call chains will be used to generate the access
     // patterns similar to GEP.
     collectAICallChains(M, F);
 
     for (auto &C : BaseAICalls)
       Transformed = transformGEPChain(M, C.first, C.second) || Transformed;
   }
 
+  // Set bitfield attributes for globals.
+  for (auto &B : BitfieldAccesses) {
+    GlobalVariable *GV = B.first;
+    std::string AttrStr;
+    for (auto &DIFieldIndex : B.second)
+      AttrStr += std::to_string(DIFieldIndex) + "$";
+    AttrStr.pop_back();
+    GV->addAttribute(BPFCoreSharedInfo::BitFieldAttr, AttrStr);
+  }
+
   return removePreserveAccessIndexIntrinsic(M) || Transformed;
 }

llvm/lib/Target/BPF/BPFCORE.h

 #define LLVM_LIB_TARGET_BPF_BPFCORE_H
 
 namespace llvm {
 
 class BPFCoreSharedInfo {
 public:
   /// The attribute attached to globals representing a member offset
   static const std::string AmaAttr;
+  /// The attribute attached to globals representing dbg bitfield indices
+  static const std::string BitFieldAttr;
   /// The section name to identify a patchable external global
   static const std::string PatchableExtSecName;
 };
 
 } // namespace llvm
 
 #endif

llvm/lib/Target/BPF/BTF.h

enum {
   BTFParamSize = 8,
   BTFDataSecVarSize = 12,
   SecFuncInfoSize = 8,
   SecLineInfoSize = 8,
   SecOffsetRelocSize = 8,
   SecExternRelocSize = 8,
   BPFFuncInfoSize = 8,
   BPFLineInfoSize = 16,
-  BPFOffsetRelocSize = 12,
+  BPFOffsetRelocSize = 16,
   BPFExternRelocSize = 8,
 };
 
 /// The .BTF section header definition.
 struct Header {
   uint16_t Magic;  ///< Magic value
   uint8_t Version; ///< Version number
   uint8_t Flags;   ///< Extra flags

 /// Specifying line info's in one section.
 struct SecLineInfo {
   uint32_t SecNameOff;  ///< Section name index in the .BTF string table
   uint32_t NumLineInfo; ///< Number of line info's in this section
 };
 
 /// Specifying one offset relocation.
 struct BPFOffsetReloc {
-  uint32_t InsnOffset;    ///< Byte offset in this section
-  uint32_t TypeID;        ///< TypeID for the relocation
-  uint32_t OffsetNameOff; ///< The string to traverse types
+  uint32_t InsnOffset;      ///< Byte offset in this section
+  uint32_t TypeID;          ///< TypeID for the relocation
+  uint32_t OffsetNameOff;   ///< The string to traverse types
+  uint32_t BitfieldMembers; ///< The string to encode all accessed bitfield members
 };
 
 /// Specifying offset relocation's in one section.
 struct SecOffsetReloc {
   uint32_t SecNameOff;     ///< Section name index in the .BTF string table
   uint32_t NumOffsetReloc; ///< Number of offset reloc's in this section
 };
 

llvm/lib/Target/BPF/BTFDebug.h

struct BTFLineInfo {
   uint32_t FileNameOff; ///< file name offset in the .BTF string table
   uint32_t LineOff;     ///< line offset in the .BTF string table
   uint32_t LineNum;     ///< the line number
   uint32_t ColumnNum;   ///< the column number
 };
 
 /// Represent one offset relocation.
 struct BTFOffsetReloc {
-  const MCSymbol *Label;  ///< MCSymbol identifying insn for the reloc
-  uint32_t TypeID;        ///< Type ID
-  uint32_t OffsetNameOff; ///< The string to traverse types
+  const MCSymbol *Label;    ///< MCSymbol identifying insn for the reloc
+  uint32_t TypeID;          ///< Type ID
+  uint32_t OffsetNameOff;   ///< The string to traverse types
+  uint32_t BitfieldMembers; ///< The string to encode all accessed bitfield members
 };
 
 /// Represent one extern relocation.
 struct BTFExternReloc {
   const MCSymbol *Label;  ///< MCSymbol identifying insn for the reloc
   uint32_t ExternNameOff; ///< The extern variable name
 };
 
class BTFDebug : public DebugHandlerBase {
   void constructLineInfo(const DISubprogram *SP, MCSymbol *Label, uint32_t Line,
                          uint32_t Column);
 
   /// Generate types and variables for globals.
   void processGlobals(bool ProcessingMapDef);
 
   /// Generate one offset relocation record.
   void generateOffsetReloc(const MachineInstr *MI, const MCSymbol *ORSym,
-                           DIType *RootTy, StringRef AccessPattern);
+                           DIType *RootTy, StringRef AccessPattern,
+                           StringRef BitFieldAccesses);
 
   /// Populating unprocessed struct type.
   unsigned populateStructType(const DIType *Ty);
 
   /// Process LD_imm64 instructions.
   void processLDimm64(const MachineInstr *MI);
 
   /// Emit common header of .BTF and .BTF.ext sections.

llvm/lib/Target/BPF/BTFDebug.cpp

for (const auto &OffsetRelocSec : OffsetRelocTable) {
       OS.AddComment("Offset reloc section string offset=" +
                     std::to_string(OffsetRelocSec.first));
       OS.EmitIntValue(OffsetRelocSec.first, 4);
       OS.EmitIntValue(OffsetRelocSec.second.size(), 4);
       for (const auto &OffsetRelocInfo : OffsetRelocSec.second) {
         Asm->EmitLabelReference(OffsetRelocInfo.Label, 4);
         OS.EmitIntValue(OffsetRelocInfo.TypeID, 4);
         OS.EmitIntValue(OffsetRelocInfo.OffsetNameOff, 4);
+        OS.EmitIntValue(OffsetRelocInfo.BitfieldMembers, 4);
       }
     }
   }
 
   // Emit extern reloc table.
   if (ExternRelocLen) {
     OS.AddComment("ExternReloc");
     OS.EmitIntValue(BTF::BPFExternRelocSize, 4);
unsigned BTFDebug::populateStructType(const DIType *Ty) {
   for (const auto &TypeEntry : TypeEntries)
     TypeEntry->completeType(*this);
   return Id;
 }
 
 /// Generate a struct member offset relocation.
 void BTFDebug::generateOffsetReloc(const MachineInstr *MI,
                                    const MCSymbol *ORSym, DIType *RootTy,
-                                   StringRef AccessPattern) {
+                                   StringRef AccessPattern,
+                                   StringRef BitFieldAccesses) {
   unsigned RootId = populateStructType(RootTy);
   size_t FirstDollar = AccessPattern.find_first_of('$');
   size_t FirstColon = AccessPattern.find_first_of(':');
   StringRef IndexPattern = AccessPattern.substr(FirstDollar + 1);
   StringRef OffsetStr = AccessPattern.substr(FirstColon + 1,
       FirstDollar - FirstColon);
 
   BTFOffsetReloc OffsetReloc;
   OffsetReloc.Label = ORSym;
   OffsetReloc.OffsetNameOff = addString(IndexPattern);
   OffsetReloc.TypeID = RootId;
+  OffsetReloc.BitfieldMembers = addString(BitFieldAccesses);
   AccessOffsets[AccessPattern.str()] = std::stoi(OffsetStr);
   OffsetRelocTable[SecNameOff].push_back(OffsetReloc);
 }
 
 void BTFDebug::processLDimm64(const MachineInstr *MI) {
   // If the insn is an LD_imm64, the following two cases
   // will generate an .BTF.ext record.
   //
void BTFDebug::processLDimm64(const MachineInstr *MI) {
   const MachineOperand &MO = MI->getOperand(1);
   if (MO.isGlobal()) {
     const GlobalValue *GVal = MO.getGlobal();
     auto *GVar = dyn_cast<GlobalVariable>(GVal);
     if (GVar && GVar->hasAttribute(BPFCoreSharedInfo::AmaAttr)) {
       MCSymbol *ORSym = OS.getContext().createTempSymbol();
       OS.EmitLabel(ORSym);
 
+      StringRef BitFieldAccesses;
+      BitFieldAccesses =
+          GVar->getAttribute(BPFCoreSharedInfo::BitFieldAttr).getValueAsString();
+
       MDNode *MDN = GVar->getMetadata(LLVMContext::MD_preserve_access_index);
       DIType *Ty = dyn_cast<DIType>(MDN);
-      generateOffsetReloc(MI, ORSym, Ty, GVar->getName());
+      generateOffsetReloc(MI, ORSym, Ty, GVar->getName(), BitFieldAccesses);
     } else if (GVar && !GVar->hasInitializer() && GVar->hasExternalLinkage() &&
                GVar->getSection() == BPFCoreSharedInfo::PatchableExtSecName) {
       MCSymbol *ORSym = OS.getContext().createTempSymbol();
       OS.EmitLabel(ORSym);
 
       BTFExternReloc ExternReloc;
       ExternReloc.Label = ORSym;
       ExternReloc.ExternNameOff = addString(GVar->getName());

llvm/test/CodeGen/BPF/CORE/intrinsic-bitfield.ll

+; RUN: llc -march=bpfel -filetype=asm -o - %s | FileCheck -check-prefixes=CHECK %s
+; RUN: llc -march=bpfeb -filetype=asm -o - %s | FileCheck -check-prefixes=CHECK %s
+; Source code:
+;   struct s {
+;     int a;
+;     int b1:8;
+;     int b2:3;
+;     int b3:4;
+;   };
+;   #define _(x) __builtin_preserve_access_index(x)
+;   int test(struct s *arg) { return _(arg->b2 + arg->b3); }
+; Compilation flag:
+;   clang -target bpf -O2 -g -S -emit-llvm test.c
+
+%struct.s = type { i32, i16 }
+
+; Function Attrs: nounwind readonly
+define dso_local i32 @test(%struct.s* readonly %arg) local_unnamed_addr #0 !dbg !7 {
+entry:
+  call void @llvm.dbg.value(metadata %struct.s* %arg, metadata !19, metadata !DIExpression()), !dbg !20
+  %0 = tail call i16* @llvm.preserve.struct.access.index.p0i16.p0s_struct.ss(%struct.s* %arg, i32 1, i32 2, i32 1), !dbg !21, !llvm.preserve.access.index !12
+  %bf.load = load i16, i16* %0, align 4, !dbg !21
+  %bf.shl = shl i16 %bf.load, 5, !dbg !21
+  %bf.ashr = ashr i16 %bf.shl, 13, !dbg !21
+  %1 = tail call i16* @llvm.preserve.struct.access.index.p0i16.p0s_struct.ss(%struct.s* %arg, i32 1, i32 3, i32 1), !dbg !21, !llvm.preserve.access.index !12
+  %bf.load1 = load i16, i16* %1, align 4, !dbg !21
+  %bf.shl2 = shl i16 %bf.load1, 1, !dbg !21
+  %bf.ashr3 = ashr i16 %bf.shl2, 12, !dbg !21
+  %narrow = add nsw i16 %bf.ashr3, %bf.ashr, !dbg !21
+  %add = sext i16 %narrow to i32, !dbg !21
+  ret i32 %add, !dbg !22
+}
+
+; CHECK-LABEL:  test
+; CHECK:        r2 = 4
+; CHECK:        r1 += r2
+;
+; CHECK:        .long   1                       # BTF_KIND_STRUCT(id = 2)
+;
+; CHECK:        .byte   115                     # string offset=1
+; CHECK:        .ascii  ".text"                 # string offset=27
+; CHECK:        .ascii  "0:1"                   # string offset=33
+; CHECK:        .ascii  "2$3"                   # string offset=37
+
+; CHECK:        .long   16                      # OffsetReloc
+; CHECK-NEXT:   .long   27                      # Offset reloc section string offset=27
+; CHECK-NEXT:   .long   1
+; CHECK-NEXT:   .long   .Ltmp{{[0-9]+}}
+; CHECK-NEXT:   .long   2
+; CHECK-NEXT:   .long   33
+; CHECK-NEXT:   .long   37
+
+; Function Attrs: nounwind readnone
+declare i16* @llvm.preserve.struct.access.index.p0i16.p0s_struct.ss(%struct.s*, i32, i32, i32) #1
+
+; Function Attrs: nounwind readnone speculatable willreturn
+declare void @llvm.dbg.value(metadata, metadata, metadata) #2
+
+attributes #0 = { nounwind readonly "correctly-rounded-divide-sqrt-fp-math"="false" "disable-tail-calls"="false" "frame-pointer"="all" "less-precise-fpmad"="false" "min-legal-vector-width"="0" "no-infs-fp-math"="false" "no-jump-tables"="false" "no-nans-fp-math"="false" "no-signed-zeros-fp-math"="false" "no-trapping-math"="false" "stack-protector-buffer-size"="8" "unsafe-fp-math"="false" "use-soft-float"="false" }
+attributes #1 = { nounwind readnone }
+attributes #2 = { nounwind readnone speculatable willreturn }
+
+!llvm.dbg.cu = !{!0}
+!llvm.module.flags = !{!3, !4, !5}
+!llvm.ident = !{!6}
+
+!0 = distinct !DICompileUnit(language: DW_LANG_C99, file: !1, producer: "clang version 10.0.0 (https://github.com/llvm/llvm-project.git d07e49be1b25894d97cc5e6f73068ac80a55a977)", isOptimized: true, runtimeVersion: 0, emissionKind: FullDebug, enums: !2, nameTableKind: None)
+!1 = !DIFile(filename: "test.c", directory: "/tmp/home/yhs/work/tests/core")
+!2 = !{}
+!3 = !{i32 2, !"Dwarf Version", i32 4}
+!4 = !{i32 2, !"Debug Info Version", i32 3}
+!5 = !{i32 1, !"wchar_size", i32 4}
+!6 = !{!"clang version 10.0.0 (https://github.com/llvm/llvm-project.git d07e49be1b25894d97cc5e6f73068ac80a55a977)"}
+!7 = distinct !DISubprogram(name: "test", scope: !1, file: !1, line: 8, type: !8, scopeLine: 8, flags: DIFlagPrototyped, isDefinition: true, isOptimized: true, unit: !0, retainedNodes: !18)
+!8 = !DISubroutineType(types: !9)
+!9 = !{!10, !11}
+!10 = !DIBasicType(name: "int", size: 32, encoding: DW_ATE_signed)
+!11 = !DIDerivedType(tag: DW_TAG_pointer_type, baseType: !12, size: 64)
+!12 = distinct !DICompositeType(tag: DW_TAG_structure_type, name: "s", file: !1, line: 1, size: 64, elements: !13)
+!13 = !{!14, !15, !16, !17}
+!14 = !DIDerivedType(tag: DW_TAG_member, name: "a", scope: !12, file: !1, line: 2, baseType: !10, size: 32)
+!15 = !DIDerivedType(tag: DW_TAG_member, name: "b1", scope: !12, file: !1, line: 3, baseType: !10, size: 8, offset: 32, flags: DIFlagBitField, extraData: i64 32)
+!16 = !DIDerivedType(tag: DW_TAG_member, name: "b2", scope: !12, file: !1, line: 4, baseType: !10, size: 3, offset: 40, flags: DIFlagBitField, extraData: i64 32)
+!17 = !DIDerivedType(tag: DW_TAG_member, name: "b3", scope: !12, file: !1, line: 5, baseType: !10, size: 4, offset: 43, flags: DIFlagBitField, extraData: i64 32)
+!18 = !{!19}
+!19 = !DILocalVariable(name: "arg", arg: 1, scope: !7, file: !1, line: 8, type: !11)
+!20 = !DILocation(line: 0, scope: !7)
+!21 = !DILocation(line: 8, column: 34, scope: !7)
+!22 = !DILocation(line: 8, column: 27, scope: !7)

llvm/test/CodeGen/BPF/CORE/intrinsic-struct.ll

 ; RUN: llc -march=bpfel -filetype=asm -o - %s | FileCheck %s
 ;
 ; Source code:
 ;   struct s { int a; int b; };
 ;   #define _(x) (__builtin_preserve_access_index(x))
 ;   int get_value(const void *addr);
 ;   int test(struct s *arg) { return get_value(_(&arg->b)); }
 ; Compiler flag to generate IR:
 ;   clang -target bpf -S -O2 -g -emit-llvm test.c
 
 %struct.s = type { i32, i32 }
 
 ; Function Attrs: nounwind
 define dso_local i32 @test(%struct.s* %arg) local_unnamed_addr #0 !dbg !7 {
 entry:
   call void @llvm.dbg.value(metadata %struct.s* %arg, metadata !17, metadata !DIExpression()), !dbg !18
-  %0 = tail call i32* @llvm.preserve.struct.access.index.p0i32.p0s_struct.ss(%struct.s* %arg, i32 1, i32 1), !dbg !19, !llvm.preserve.access.index !12
+  %0 = tail call i32* @llvm.preserve.struct.access.index.p0i32.p0s_struct.ss(%struct.s* %arg, i32 1, i32 1, i32 1), !dbg !19, !llvm.preserve.access.index !12
   %1 = bitcast i32* %0 to i8*, !dbg !19
   %call = tail call i32 @get_value(i8* %1) #4, !dbg !20
   ret i32 %call, !dbg !21
 }
 
 ; CHECK-LABEL: test
 ; CHECK:       [[RELOC:.Ltmp[0-9]+]]
 ; CHECK:       r2 = 4
 ; CHECK:       r1 += r2
 ; CHECK:       call get_value
 ; CHECK:       exit
 ;
 ; CHECK:      .section        .BTF.ext,"",@progbits
-; CHECK:      .long   12                      # OffsetReloc
+; CHECK:      .long   16                      # OffsetReloc
 ; CHECK-NEXT: .long   20                      # Offset reloc section string offset=20
 ; CHECK-NEXT: .long   1
 ; CHECK-NEXT: .long   [[RELOC]]
 ; CHECK-NEXT: .long   2
 ; CHECK-NEXT: .long   26
+; CHECK-NEXT: .long   0
 
 declare dso_local i32 @get_value(i8*) local_unnamed_addr #1
 
 ; Function Attrs: nounwind readnone
-declare i32* @llvm.preserve.struct.access.index.p0i32.p0s_struct.ss(%struct.s*, i32 immarg, i32 immarg) #2
+declare i32* @llvm.preserve.struct.access.index.p0i32.p0s_struct.ss(%struct.s*, i32 immarg, i32 immarg, i32 immarg) #2
 
 ; Function Attrs: nounwind readnone speculatable
 declare void @llvm.dbg.value(metadata, metadata, metadata) #3
 
 attributes #0 = { nounwind "correctly-rounded-divide-sqrt-fp-math"="false" "disable-tail-calls"="false" "less-precise-fpmad"="false" "min-legal-vector-width"="0" "no-frame-pointer-elim"="true" "no-frame-pointer-elim-non-leaf" "no-infs-fp-math"="false" "no-jump-tables"="false" "no-nans-fp-math"="false" "no-signed-zeros-fp-math"="false" "no-trapping-math"="false" "stack-protector-buffer-size"="8" "unsafe-fp-math"="false" "use-soft-float"="false" }
 attributes #1 = { "correctly-rounded-divide-sqrt-fp-math"="false" "disable-tail-calls"="false" "less-precise-fpmad"="false" "no-frame-pointer-elim"="true" "no-frame-pointer-elim-non-leaf" "no-infs-fp-math"="false" "no-nans-fp-math"="false" "no-signed-zeros-fp-math"="false" "no-trapping-math"="false" "stack-protector-buffer-size"="8" "unsafe-fp-math"="false" "use-soft-float"="false" }
 attributes #2 = { nounwind readnone }
 attributes #3 = { nounwind readnone speculatable }

llvm/test/CodeGen/BPF/CORE/offset-reloc-bitfield.ll

+; RUN: llc -march=bpfel -filetype=asm -o - %s | FileCheck -check-prefixes=CHECK %s
+; RUN: llc -march=bpfeb -filetype=asm -o - %s | FileCheck -check-prefixes=CHECK %s
+; Source code:
+;   struct s {
+;     int a;
+;     int b1:8;
+;     int b2:3;
+;     int b3:4;
+;   };
+;
+;   #define _(x, y) __builtin_preserve_bitfield_info((x), (y))
+;   void process(void *);
+;   unsigned test(struct s *arg) {
+;     unsigned info1, info2;
+;     void *p1, *p2;
+;
+;     p1 = _(arg->b2, &info1);
+;     p2 = _(arg->b3, &info2);
+;     process(p1);
+;     process(p2);
+;     // the last 16 bit is the offset, so
+;     // return value should be 8 + 11 = 19
+;     // At -O2, compiler should do this optimization.
+;     return (info1 & 0xffff) + (info2 & 0xffff);
+;   }
+
+%struct.s = type { i32, i16 }
+
+; Function Attrs: nounwind
+define dso_local i32 @test(%struct.s* %arg) local_unnamed_addr #0 !dbg !7 {
+entry:
+  call void @llvm.dbg.value(metadata %struct.s* %arg, metadata !20, metadata !DIExpression()), !dbg !26
+  %0 = tail call i16* @llvm.preserve.struct.access.index.p0i16.p0s_struct.ss(%struct.s* %arg, i32 1, i32 2, i32 1), !dbg !27, !llvm.preserve.access.index !12
+  %1 = bitcast i16* %0 to i8*, !dbg !27
+  call void @llvm.dbg.value(metadata i32 -2147287032, metadata !21, metadata !DIExpression()), !dbg !26
+  call void @llvm.dbg.value(metadata i8* %1, metadata !23, metadata !DIExpression()), !dbg !26
+  %2 = tail call i16* @llvm.preserve.struct.access.index.p0i16.p0s_struct.ss(%struct.s* %arg, i32 1, i32 3, i32 1), !dbg !28, !llvm.preserve.access.index !12
+  %3 = bitcast i16* %2 to i8*, !dbg !28
+  call void @llvm.dbg.value(metadata i32 -2147221493, metadata !22, metadata !DIExpression()), !dbg !26
+  call void @llvm.dbg.value(metadata i8* %3, metadata !25, metadata !DIExpression()), !dbg !26
+  tail call void @process(i8* %1) #4, !dbg !29
+  tail call void @process(i8* %3) #4, !dbg !30
+  ret i32 19, !dbg !31
+}
+
+; CHECK-LABEL:   test
+; CHECK:         r1 = 4
+; CHECK:         r{{[0-9]+}} += r1
+; CHECK:         r0 = 19
+; CHECK:         exit
+;
+; CHECK:         .ascii  ".text"                 # string offset=40
+; CHECK:         .ascii  "0:1"                   # string offset=83
+; CHECK:         .ascii  "2$3"                   # string offset=87
+; CHECK:        .long   16                      # OffsetReloc
+; CHECK-NEXT:   .long   40                      # Offset reloc section string offset=40
+; CHECK-NEXT:   .long   1
+; CHECK-NEXT:   .long   .Ltmp{{[0-9]+}}
+; CHECK-NEXT:   .long   2
+; CHECK-NEXT:   .long   83
+; CHECK-NEXT:   .long   87
+
+; Function Attrs: nounwind readnone
+declare i16* @llvm.preserve.struct.access.index.p0i16.p0s_struct.ss(%struct.s*, i32, i32, i32) #1
+
+declare dso_local void @process(i8*) local_unnamed_addr #2
+
+; Function Attrs: nounwind readnone speculatable willreturn
+declare void @llvm.dbg.value(metadata, metadata, metadata) #3
+
+attributes #0 = { nounwind "correctly-rounded-divide-sqrt-fp-math"="false" "disable-tail-calls"="false" "frame-pointer"="all" "less-precise-fpmad"="false" "min-legal-vector-width"="0" "no-infs-fp-math"="false" "no-jump-tables"="false" "no-nans-fp-math"="false" "no-signed-zeros-fp-math"="false" "no-trapping-math"="false" "stack-protector-buffer-size"="8" "unsafe-fp-math"="false" "use-soft-float"="false" }
+attributes #1 = { nounwind readnone }
+attributes #2 = { "correctly-rounded-divide-sqrt-fp-math"="false" "disable-tail-calls"="false" "frame-pointer"="all" "less-precise-fpmad"="false" "no-infs-fp-math"="false" "no-nans-fp-math"="false" "no-signed-zeros-fp-math"="false" "no-trapping-math"="false" "stack-protector-buffer-size"="8" "unsafe-fp-math"="false" "use-soft-float"="false" }
+attributes #3 = { nounwind readnone speculatable willreturn }
+attributes #4 = { nounwind }
+
+!llvm.dbg.cu = !{!0}
+!llvm.module.flags = !{!3, !4, !5}
+!llvm.ident = !{!6}
+
+!0 = distinct !DICompileUnit(language: DW_LANG_C99, file: !1, producer: "clang version 10.0.0 (https://github.com/llvm/llvm-project.git d07e49be1b25894d97cc5e6f73068ac80a55a977)", isOptimized: true, runtimeVersion: 0, emissionKind: FullDebug, enums: !2, nameTableKind: None)
+!1 = !DIFile(filename: "test.c", directory: "/tmp/home/yhs/work/tests/core")
+!2 = !{}
+!3 = !{i32 2, !"Dwarf Version", i32 4}
+!4 = !{i32 2, !"Debug Info Version", i32 3}
+!5 = !{i32 1, !"wchar_size", i32 4}
+!6 = !{!"clang version 10.0.0 (https://github.com/llvm/llvm-project.git d07e49be1b25894d97cc5e6f73068ac80a55a977)"}
+!7 = distinct !DISubprogram(name: "test", scope: !1, file: !1, line: 10, type: !8, scopeLine: 10, flags: DIFlagPrototyped, isDefinition: true, isOptimized: true, unit: !0, retainedNodes: !19)
+!8 = !DISubroutineType(types: !9)
+!9 = !{!10, !11}
+!10 = !DIBasicType(name: "unsigned int", size: 32, encoding: DW_ATE_unsigned)
+!11 = !DIDerivedType(tag: DW_TAG_pointer_type, baseType: !12, size: 64)
+!12 = distinct !DICompositeType(tag: DW_TAG_structure_type, name: "s", file: !1, line: 1, size: 64, elements: !13)
+!13 = !{!14, !16, !17, !18}
+!14 = !DIDerivedType(tag: DW_TAG_member, name: "a", scope: !12, file: !1, line: 2, baseType: !15, size: 32)
+!15 = !DIBasicType(name: "int", size: 32, encoding: DW_ATE_signed)
+!16 = !DIDerivedType(tag: DW_TAG_member, name: "b1", scope: !12, file: !1, line: 3, baseType: !15, size: 8, offset: 32, flags: DIFlagBitField, extraData: i64 32)
+!17 = !DIDerivedType(tag: DW_TAG_member, name: "b2", scope: !12, file: !1, line: 4, baseType: !15, size: 3, offset: 40, flags: DIFlagBitField, extraData: i64 32)
+!18 = !DIDerivedType(tag: DW_TAG_member, name: "b3", scope: !12, file: !1, line: 5, baseType: !15, size: 4, offset: 43, flags: DIFlagBitField, extraData: i64 32)
+!19 = !{!20, !21, !22, !23, !25}
+!20 = !DILocalVariable(name: "arg", arg: 1, scope: !7, file: !1, line: 10, type: !11)
+!21 = !DILocalVariable(name: "info1", scope: !7, file: !1, line: 11, type: !10)
+!22 = !DILocalVariable(name: "info2", scope: !7, file: !1, line: 11, type: !10)
+!23 = !DILocalVariable(name: "p1", scope: !7, file: !1, line: 12, type: !24)
+!24 = !DIDerivedType(tag: DW_TAG_pointer_type, baseType: null, size: 64)
+!25 = !DILocalVariable(name: "p2", scope: !7, file: !1, line: 12, type: !24)
+!26 = !DILocation(line: 0, scope: !7)
+!27 = !DILocation(line: 14, column: 8, scope: !7)
+!28 = !DILocation(line: 15, column: 8, scope: !7)
+!29 = !DILocation(line: 16, column: 3, scope: !7)
+!30 = !DILocation(line: 17, column: 3, scope: !7)
+!31 = !DILocation(line: 21, column: 3, scope: !7)