Global Named Register Variables Support

[rengolin]

Summary

Non-allocatable Global Named Register

This patch implements global named registers in Clang, lowering to the just
created intrinsics in LLVM (@llvm.read/write_register). A new type of LValue
had to be created (Register), which just adds support to carry the metadata
node containing the name of the register. Two new methods to emit loads and
stores interoperate with another to emit the named metadata node.

No guarantees are being made and only non-allocatable global variable named
registers are being supported. Local named register support is unchanged.

[rengolin]

Simpler tests, better comments, names.

[rnk]

include/clang/AST/Decl.h

830

This fixes "error: reference to local variable 'current_stack_pointer' declared in enclosing context", right?

I think there's more to sort out here. The Doxygen comments here clearly say that this function only works on variables with function scope, but clearly somebody is calling it on variables without checking if it has function scope. We also have a check for isFileVarDecl() above, further supporting this. Probably the right fix is to return false early if (!isLocalVarDecl()).

lib/CodeGen/CGExpr.cpp

1351

Does Doxygen pick these up from source files? I thought it only worked in headers.

Anyway, we're not repeating the name of the function anymore. Instead we use \brief or @brief.

Is "Global Named Register" a proper noun deserving of capitalization?

1369

This condition will essentially never be true because there aren't many non-allocatable registers worth storing to. The only one I can think of is the TLS pointer on some ISAs, and this will only happen in libc or low-level libs. Let's move this into the !Dst.isSimple() check and put it *after* Dst.isBitField().

1603

The comment is copy-pasted and incorrect.

The same Doxygen advice applies.

1783

star-on-the-right here and for M.

[rengolin]

include/clang/AST/Decl.h

830

The function hasGlobalStorage() returns !hasLocalStorage(), and that's how we get here, so not only local scope here, I guess.

Also, I don't know if isLocalVarDecl will catch all "local" context, since there could be other local declarations.

While early return for !isLocalVarDecl() would be correct, !isFileVarDecl() is used only when the storage class is None, which makes me think that it's not as ubiquitous as it might transpire, and why I didn't add !isLocalVarDecl() on a global scope on that function, but inside a very specific case, which I know to be true.

If both have complete semantics, why not add on the first line:

if (!isLocalVarDecl() && !isFileVarDecl())
  return false;

lib/CodeGen/CGExpr.cpp

1351

I was just following the pattern around this function. I'll use \@brief.

1369

I put it as the last line, but it didn't follow that path, probably because of !isSimple(). I'll move it inside, as the last check.

1603

ok

1783

ok. I'll also move the initialization of Str inside the if.

[rengolin]

Response to reviews:

• Changed Doxygen comments
• Pointer declaration star-on-right
• Moved Str variable inside conditional that uses it
• Moved isGlobalReg() inside !isSimple() just before assert(isBitfield)

Not changed yet (undergoing discussion):

• if (getStorageClass() == SC_Register && !isLocalVarDecl())

[rengolin]

Doxygen "brief" is either slash or at, not both.

[rnk]

LGTM

include/clang/AST/Decl.h

830

OK, let's leave it alone.

[rengolin]

Thanks Reid, r209149.

include/clang/AST/Decl.h

 
   /// hasLocalStorage - Returns true if a variable with function scope
   ///  is a non-static local variable.
   bool hasLocalStorage() const {
     if (getStorageClass() == SC_None)
       // Second check is for C++11 [dcl.stc]p4.
       return !isFileVarDecl() && getTSCSpec() == TSCS_unspecified;
 
+    // Global Named Register (GNU extension)
+    if (getStorageClass() == SC_Register && !isLocalVarDecl())
+      return false;
+
     // Return true for:  Auto, Register.
     // Return false for: Extern, Static, PrivateExtern, OpenCLWorkGroupLocal.
 
     return getStorageClass() >= SC_Auto;
   }
 
   /// isStaticLocal - Returns true if a variable with function scope is a
   /// static local variable.

lib/CodeGen/CGExpr.cpp

 #include "CGCall.h"
 #include "CGDebugInfo.h"
 #include "CGObjCRuntime.h"
 #include "CGRecordLayout.h"
 #include "CodeGenModule.h"
 #include "TargetInfo.h"
 #include "clang/AST/ASTContext.h"
 #include "clang/AST/DeclObjC.h"
+#include "clang/AST/Attr.h"
 #include "clang/Frontend/CodeGenOptions.h"
 #include "llvm/ADT/Hashing.h"
 #include "llvm/IR/DataLayout.h"
 #include "llvm/IR/Intrinsics.h"
 #include "llvm/IR/LLVMContext.h"
 #include "llvm/IR/MDBuilder.h"
 #include "llvm/Support/ConvertUTF.h"
 

                                                     "vecext"));
   }
 
   // If this is a reference to a subset of the elements of a vector, either
   // shuffle the input or extract/insert them as appropriate.
   if (LV.isExtVectorElt())
     return EmitLoadOfExtVectorElementLValue(LV);
 
+  // Global Register variables always invoke intrinsics
+  if (LV.isGlobalReg())
+    return EmitLoadOfGlobalRegLValue(LV);
+
   assert(LV.isBitField() && "Unknown LValue type!");
   return EmitLoadOfBitfieldLValue(LV);
 }
 
 RValue CodeGenFunction::EmitLoadOfBitfieldLValue(LValue LV) {
   const CGBitFieldInfo &Info = LV.getBitFieldInfo();
 
   // Get the output type.

     Mask.push_back(Builder.getInt32(getAccessedFieldNo(i, Elts)));
 
   llvm::Value *MaskV = llvm::ConstantVector::get(Mask);
   Vec = Builder.CreateShuffleVector(Vec, llvm::UndefValue::get(Vec->getType()),
                                     MaskV);
   return RValue::get(Vec);
 }
 
+/// @brief Load of global gamed gegisters are always calls to intrinsics.
+RValue CodeGenFunction::EmitLoadOfGlobalRegLValue(LValue LV) {
+  assert(LV.getType()->isIntegerType() && "Bad type for register variable");
+  llvm::MDNode *RegName = dyn_cast<llvm::MDNode>(LV.getGlobalReg());
+  assert(RegName && "Register LValue is not metadata");
+  llvm::Type *Types[] = { CGM.getTypes().ConvertType(LV.getType()) };
+  llvm::Value *F = CGM.getIntrinsic(llvm::Intrinsic::read_register, Types);
+  llvm::Value* Call = Builder.CreateCall(F, RegName);
+  return RValue::get(Call);
+}
 
 
 /// EmitStoreThroughLValue - Store the specified rvalue into the specified
 /// lvalue, where both are guaranteed to the have the same type, and that type
 /// is 'Ty'.
 void CodeGenFunction::EmitStoreThroughLValue(RValue Src, LValue Dst,
                                              bool isInit) {
   if (!Dst.isSimple()) {
     if (Dst.isVectorElt()) {
       // Read/modify/write the vector, inserting the new element.
       llvm::LoadInst *Load = Builder.CreateLoad(Dst.getVectorAddr(),
                                                 Dst.isVolatileQualified());
       Load->setAlignment(Dst.getAlignment().getQuantity());
       llvm::Value *Vec = Load;
       Vec = Builder.CreateInsertElement(Vec, Src.getScalarVal(),
                                         Dst.getVectorIdx(), "vecins");
       llvm::StoreInst *Store = Builder.CreateStore(Vec, Dst.getVectorAddr(),
                                                    Dst.isVolatileQualified());
       Store->setAlignment(Dst.getAlignment().getQuantity());
       return;
     }
 
     // If this is an update of extended vector elements, insert them as
     // appropriate.
     if (Dst.isExtVectorElt())
       return EmitStoreThroughExtVectorComponentLValue(Src, Dst);
 
+    if (Dst.isGlobalReg())
+      return EmitStoreThroughGlobalRegLValue(Src, Dst);
+
     assert(Dst.isBitField() && "Unknown LValue type");
     return EmitStoreThroughBitfieldLValue(Src, Dst);
   }
 
   // There's special magic for assigning into an ARC-qualified l-value.
   if (Qualifiers::ObjCLifetime Lifetime = Dst.getQuals().getObjCLifetime()) {
     switch (Lifetime) {
     case Qualifiers::OCL_None:

     Vec = Builder.CreateInsertElement(Vec, SrcVal, Elt);
   }
 
   llvm::StoreInst *Store = Builder.CreateStore(Vec, Dst.getExtVectorAddr(),
                                                Dst.isVolatileQualified());
   Store->setAlignment(Dst.getAlignment().getQuantity());
 }
 
+/// @brief Store of global named registers are always calls to intrinsics.
+void CodeGenFunction::EmitStoreThroughGlobalRegLValue(RValue Src, LValue Dst) {
+  assert(Dst.getType()->isIntegerType() && "Bad type for register variable");
+  llvm::MDNode *RegName = dyn_cast<llvm::MDNode>(Dst.getGlobalReg());
+  assert(RegName && "Register LValue is not metadata");
+  llvm::Type *Types[] = { CGM.getTypes().ConvertType(Dst.getType()) };
+  llvm::Value *F = CGM.getIntrinsic(llvm::Intrinsic::write_register, Types);
+  llvm::Value *Value = Src.getScalarVal();
+  Builder.CreateCall2(F, RegName, Value);
+}
+
 // setObjCGCLValueClass - sets class of he lvalue for the purpose of
 // generating write-barries API. It is currently a global, ivar,
 // or neither.
 static void setObjCGCLValueClass(const ASTContext &Ctx, const Expr *E,
                                  LValue &LV,
                                  bool IsMemberAccess=false) {
   if (Ctx.getLangOpts().getGC() == LangOptions::NonGC)
     return;

 
 static LValue EmitCapturedFieldLValue(CodeGenFunction &CGF, const FieldDecl *FD,
                                       llvm::Value *ThisValue) {
   QualType TagType = CGF.getContext().getTagDeclType(FD->getParent());
   LValue LV = CGF.MakeNaturalAlignAddrLValue(ThisValue, TagType);
   return CGF.EmitLValueForField(LV, FD);
 }
 
+/// Named Registers are named metadata pointing to the register name
+/// which will be read from/written to as an argument to the intrinsic
+/// @llvm.read/write_register.
+/// So far, only the name is being passed down, but other options such as
+/// register type, allocation type or even optimization options could be
+/// passed down via the metadata node.
+static LValue EmitGlobalNamedRegister(const VarDecl *VD,
+                                      CodeGenModule &CGM,
+                                      CharUnits Alignment) {
+  AsmLabelAttr *Asm = VD->getAttr<AsmLabelAttr>();
+  llvm::Twine Name("llvm.named.register."+Asm->getLabel());
+  llvm::NamedMDNode *M = CGM.getModule().getOrInsertNamedMetadata(Name.str());
+  if (M->getNumOperands() == 0) {
+    llvm::MDString *Str = llvm::MDString::get(CGM.getLLVMContext(),
+                                              Asm->getLabel());
+    llvm::Value *Ops[] = { Str };
+    M->addOperand(llvm::MDNode::get(CGM.getLLVMContext(), Ops));
+  }
+  return LValue::MakeGlobalReg(M->getOperand(0), VD->getType(), Alignment);
+}
+
 LValue CodeGenFunction::EmitDeclRefLValue(const DeclRefExpr *E) {
   const NamedDecl *ND = E->getDecl();
   CharUnits Alignment = getContext().getDeclAlign(ND);
   QualType T = E->getType();
+  const auto *VD = dyn_cast<VarDecl>(ND);
+
+  // Global Named registers access via intrinsics only
+  if (VD && VD->getStorageClass() == SC_Register &&
+       VD->hasAttr<AsmLabelAttr>() && !VD->isLocalVarDecl())
+    return EmitGlobalNamedRegister(VD, CGM, Alignment);
 
   // A DeclRefExpr for a reference initialized by a constant expression can
   // appear without being odr-used. Directly emit the constant initializer.
-  if (const auto *VD = dyn_cast<VarDecl>(ND)) {
+  if (VD) {
     const Expr *Init = VD->getAnyInitializer(VD);
     if (Init && !isa<ParmVarDecl>(VD) && VD->getType()->isReferenceType() &&
         VD->isUsableInConstantExpressions(getContext()) &&
         VD->checkInitIsICE()) {
       llvm::Constant *Val =
         CGM.EmitConstantValue(*VD->evaluateValue(), VD->getType(), this);
       assert(Val && "failed to emit reference constant expression");
       // FIXME: Eventually we will want to emit vector element references.

          "Should not use decl without marking it used!");
 
   if (ND->hasAttr<WeakRefAttr>()) {
     const auto *VD = cast<ValueDecl>(ND);
     llvm::Constant *Aliasee = CGM.GetWeakRefReference(VD);
     return MakeAddrLValue(Aliasee, T, Alignment);
   }
 
-  if (const auto *VD = dyn_cast<VarDecl>(ND)) {
+  if (VD) {
     // Check if this is a global variable.
     if (VD->hasLinkage() || VD->isStaticDataMember())
       return EmitGlobalVarDeclLValue(*this, E, VD);
 
     bool isBlockVariable = VD->hasAttr<BlocksAttr>();
 
     llvm::Value *V = LocalDeclMap.lookup(VD);
     if (!V && VD->isStaticLocal())

lib/CodeGen/CGValue.h

 /// LValue - This represents an lvalue references.  Because C/C++ allow
 /// bitfields, this is not a simple LLVM pointer, it may be a pointer plus a
 /// bitrange.
 class LValue {
   enum {
     Simple,       // This is a normal l-value, use getAddress().
     VectorElt,    // This is a vector element l-value (V[i]), use getVector*
     BitField,     // This is a bitfield l-value, use getBitfield*.
-    ExtVectorElt  // This is an extended vector subset, use getExtVectorComp
+    ExtVectorElt, // This is an extended vector subset, use getExtVectorComp
+    GlobalReg     // This is a register l-value, use getGlobalReg()
   } LVType;
 
   llvm::Value *V;
 
   union {
     // Index into a vector subscript: V[i]
     llvm::Value *VectorIdx;
 

     this->TBAAInfo = TBAAInfo;
   }
 
 public:
   bool isSimple() const { return LVType == Simple; }
   bool isVectorElt() const { return LVType == VectorElt; }
   bool isBitField() const { return LVType == BitField; }
   bool isExtVectorElt() const { return LVType == ExtVectorElt; }
+  bool isGlobalReg() const { return LVType == GlobalReg; }
 
   bool isVolatileQualified() const { return Quals.hasVolatile(); }
   bool isRestrictQualified() const { return Quals.hasRestrict(); }
   unsigned getVRQualifiers() const {
     return Quals.getCVRQualifiers() & ~Qualifiers::Const;
   }
 
   QualType getType() const { return Type; }

     assert(isBitField());
     return V;
   }
   const CGBitFieldInfo &getBitFieldInfo() const {
     assert(isBitField());
     return *BitFieldInfo;
   }
 
+  // global register lvalue
+  llvm::Value *getGlobalReg() const { assert(isGlobalReg()); return V; }
+
   static LValue MakeAddr(llvm::Value *address, QualType type,
                          CharUnits alignment, ASTContext &Context,
                          llvm::MDNode *TBAAInfo = 0) {
     Qualifiers qs = type.getQualifiers();
     qs.setObjCGCAttr(Context.getObjCGCAttrKind(type));
 
     LValue R;
     R.LVType = Simple;

     LValue R;
     R.LVType = BitField;
     R.V = Addr;
     R.BitFieldInfo = &Info;
     R.Initialize(type, type.getQualifiers(), Alignment);
     return R;
   }
 
+  static LValue MakeGlobalReg(llvm::Value *Reg,
+                              QualType type,
+                              CharUnits Alignment) {
+    LValue R;
+    R.LVType = GlobalReg;
+    R.V = Reg;
+    R.Initialize(type, type.getQualifiers(), Alignment);
+    return R;
+  }
+
   RValue asAggregateRValue() const {
     // FIMXE: Alignment
     return RValue::getAggregate(getAddress(), isVolatileQualified());
   }
 };
 
 /// An aggregate value slot.
 class AggValueSlot {

lib/CodeGen/CodeGenFunction.h

   void EmitStoreOfScalar(llvm::Value *value, LValue lvalue, bool isInit=false);
 
   /// EmitLoadOfLValue - Given an expression that represents a value lvalue,
   /// this method emits the address of the lvalue, then loads the result as an
   /// rvalue, returning the rvalue.
   RValue EmitLoadOfLValue(LValue V, SourceLocation Loc);
   RValue EmitLoadOfExtVectorElementLValue(LValue V);
   RValue EmitLoadOfBitfieldLValue(LValue LV);
+  RValue EmitLoadOfGlobalRegLValue(LValue LV);
 
   /// EmitStoreThroughLValue - Store the specified rvalue into the specified
   /// lvalue, where both are guaranteed to the have the same type, and that type
   /// is 'Ty'.
   void EmitStoreThroughLValue(RValue Src, LValue Dst, bool isInit=false);
   void EmitStoreThroughExtVectorComponentLValue(RValue Src, LValue Dst);
+  void EmitStoreThroughGlobalRegLValue(RValue Src, LValue Dst);
 
   /// EmitStoreThroughBitfieldLValue - Store Src into Dst with same constraints
   /// as EmitStoreThroughLValue.
   ///
   /// \param Result [out] - If non-null, this will be set to a Value* for the
   /// bit-field contents after the store, appropriate for use as the result of
   /// an assignment to the bit-field.
   void EmitStoreThroughBitfieldLValue(RValue Src, LValue Dst,
                                       llvm::Value **Result=0);
 
   /// Emit an l-value for an assignment (simple or compound) of complex type.
   LValue EmitComplexAssignmentLValue(const BinaryOperator *E);
   LValue EmitComplexCompoundAssignmentLValue(const CompoundAssignOperator *E);
   LValue EmitScalarCompooundAssignWithComplex(const CompoundAssignOperator *E,
                                               llvm::Value *&Result);
 
   // Note: only available for agg return types
   LValue EmitBinaryOperatorLValue(const BinaryOperator *E);
   LValue EmitCompoundAssignmentLValue(const CompoundAssignOperator *E);
   // Note: only available for agg return types
   LValue EmitCallExprLValue(const CallExpr *E);
   // Note: only available for agg return types
   LValue EmitVAArgExprLValue(const VAArgExpr *E);
   LValue EmitDeclRefLValue(const DeclRefExpr *E);
+  LValue EmitReadRegister(const VarDecl *VD);
   LValue EmitStringLiteralLValue(const StringLiteral *E);
   LValue EmitObjCEncodeExprLValue(const ObjCEncodeExpr *E);
   LValue EmitPredefinedLValue(const PredefinedExpr *E);
   LValue EmitUnaryOpLValue(const UnaryOperator *E);
   LValue EmitArraySubscriptExpr(const ArraySubscriptExpr *E,
                                 bool Accessed = false);
   LValue EmitExtVectorElementExpr(const ExtVectorElementExpr *E);
   LValue EmitMemberExpr(const MemberExpr *E);

lib/Sema/SemaDecl.cpp

     return 0;
   }
 
   DiagnoseFunctionSpecifiers(D.getDeclSpec());
 
   if (!DC->isRecord() && S->getFnParent() == 0) {
     // C99 6.9p2: The storage-class specifiers auto and register shall not
     // appear in the declaration specifiers in an external declaration.
-    if (SC == SC_Auto || SC == SC_Register) {
-      // If this is a register variable with an asm label specified, then this
-      // is a GNU extension.
-      if (SC == SC_Register && D.getAsmLabel())
-        Diag(D.getIdentifierLoc(), diag::err_unsupported_global_register);
-      else
-        Diag(D.getIdentifierLoc(), diag::err_typecheck_sclass_fscope);
+    // Global Register+Asm is a GNU extension we support.
+    if (SC == SC_Auto || (SC == SC_Register && !D.getAsmLabel())) {
+      Diag(D.getIdentifierLoc(), diag::err_typecheck_sclass_fscope);
       D.setInvalidType();
     }
   }
 
   if (getLangOpts().OpenCL) {
     // Set up the special work-group-local storage class for variables in the
     // OpenCL __local address space.
     if (R.getAddressSpace() == LangAS::opencl_local) {

     StringRef Label = SE->getString();
     if (S->getFnParent() != 0) {
       switch (SC) {
       case SC_None:
       case SC_Auto:
         Diag(E->getExprLoc(), diag::warn_asm_label_on_auto_decl) << Label;
         break;
       case SC_Register:
+        // Local Named register
         if (!Context.getTargetInfo().isValidGCCRegisterName(Label))
           Diag(E->getExprLoc(), diag::err_asm_unknown_register_name) << Label;
         break;
       case SC_Static:
       case SC_Extern:
       case SC_PrivateExtern:
       case SC_OpenCLWorkGroupLocal:
         break;
       }
+    } else if (SC == SC_Register) {
+      // Global Named register
+      if (!Context.getTargetInfo().isValidGCCRegisterName(Label))
+        Diag(E->getExprLoc(), diag::err_asm_unknown_register_name) << Label;
     }
 
     NewVD->addAttr(::new (Context) AsmLabelAttr(SE->getStrTokenLoc(0),
                                                 Context, Label, 0));
   } else if (!ExtnameUndeclaredIdentifiers.empty()) {
     llvm::DenseMap<IdentifierInfo*,AsmLabelAttr*>::iterator I =
       ExtnameUndeclaredIdentifiers.find(NewVD->getIdentifier());
     if (I != ExtnameUndeclaredIdentifiers.end()) {

test/CodeGen/named_reg_global.c

+// RUN: %clang_cc1 -triple x86_64-linux-gnu -S -emit-llvm %s -o - | FileCheck %s
+// RUN: %clang_cc1 -triple aarch64-linux-gnu -S -emit-llvm %s -o - | FileCheck %s
+// RUN: %clang_cc1 -triple arm64-linux-gnu -S -emit-llvm %s -o - | FileCheck %s
+// RUN: %clang_cc1 -triple armv7-linux-gnu -S -emit-llvm %s -o - | FileCheck %s
+
+register unsigned long current_stack_pointer asm("sp");
+
+// CHECK: define{{.*}} i[[bits:[0-9]+]] @get_stack_pointer_addr()
+// CHECK: [[ret:%[0-9]+]] = call i[[bits]] @llvm.read_register.i[[bits]](metadata !0)
+// CHECK: ret i[[bits]] [[ret]]
+unsigned long get_stack_pointer_addr() {
+  return current_stack_pointer;
+}
+// CHECK: declare{{.*}} i[[bits]] @llvm.read_register.i[[bits]](metadata)
+
+// CHECK: define{{.*}} void @set_stack_pointer_addr(i[[bits]] %addr) #0 {
+// CHECK: [[sto:%[0-9]+]] = load i[[bits]]* %addr
+// CHECK: call void @llvm.write_register.i[[bits]](metadata !0, i[[bits]] [[sto]])
+// CHECK: ret void
+void set_stack_pointer_addr(unsigned long addr) {
+  current_stack_pointer = addr;
+}
+// CHECK: declare{{.*}} void @llvm.write_register.i[[bits]](metadata, i[[bits]])
+
+// CHECK: !llvm.named.register.sp = !{!0}
+// CHECK: !0 = metadata !{metadata !"sp"}

test/Sema/asm.c

 }
 
 // PR3905
 void test9(int i) {
   asm("" : [foo] "=r" (i), "=r"(i) : "1[foo]"(i)); // expected-error{{invalid input constraint '1[foo]' in asm}}
   asm("" : [foo] "=r" (i), "=r"(i) : "[foo]1"(i)); // expected-error{{invalid input constraint '[foo]1' in asm}}
 }
 
-register int g asm("dx"); // expected-error{{global register variables are not supported}}
-
 void test10(void){
   static int g asm ("g_asm") = 0;
   extern int gg asm ("gg_asm");
   __private_extern__ int ggg asm ("ggg_asm");
 
   int a asm ("a_asm"); // expected-warning{{ignored asm label 'a_asm' on automatic variable}}
   auto int aa asm ("aa_asm"); // expected-warning{{ignored asm label 'aa_asm' on automatic variable}}
 

test/Sema/decl-invalid.c

 struct; // expected-error {{declaration of anonymous struct must be a definition}} // expected-warning {{declaration does not declare anything}}
 typedef int I;
 I; // expected-warning {{declaration does not declare anything}}
 
 
 
 // rdar://6880449
 register int test1;     // expected-error {{illegal storage class on file-scoped variable}}
-register int test2 __asm__("edi");  // expected-error {{global register variables are not supported}}