[OpenCL] Mark pointers to constant address space as invariant

[yaxunl]

Summary

OpenCL constant address space is immutable, therefore pointers to constant address space can
be marked with llvm.invariant.start permanently.

This should allow more optimization opportunities in LLVM passes.

[rjmccall]

clang/lib/CodeGen/CGDeclCXX.cpp

169–171

I know this is a pre-existing code pattern, but there's an overload of CreateCall that just takes two arguments directly; please switch the code to use that.

170

This check is already done by CreateBitCast.

clang/lib/CodeGen/CGExpr.cpp

1255

This is not the way to do this, but fortunately it's unnecessary anyway. You can just put the invariant-load metadata on any loads from the constant address space in the functions called by EmitLoadOfLValue.

[hliao]

invariant checking already takes account of loading from constant address space or memory (AA::pointsToConstantMemory), that's almost equivalent to adding invariant attributes. Why do we mark these constant loads with additional attributes?

[yaxunl]

In D75423#1901206, @hliao wrote:

invariant checking already takes account of loading from constant address space or memory (AA::pointsToConstantMemory), that's almost equivalent to adding invariant attributes. Why do we mark these constant loads with additional attributes?

I got this request from Matt. He wants a consistent representation for invariant load, not relying on alias analysis.

[rjmccall]

In D75423#1901206, @hliao wrote:

invariant checking already takes account of loading from constant address space or memory (AA::pointsToConstantMemory), that's almost equivalent to adding invariant attributes. Why do we mark these constant loads with additional attributes?

If alias analysis already knows that all memory in the constant address space is immutable, I agree that should be sufficient, at least on targets where the constant address space is actually a different address space in LLVM IR. If it only knows that when it can resolve an access down to a specific constant global variable, this patch is still providing value because it applies even for accesses through a pointer.

[yaxunl]

In D75423#1901254, @rjmccall wrote:

In D75423#1901206, @hliao wrote:

invariant checking already takes account of loading from constant address space or memory (AA::pointsToConstantMemory), that's almost equivalent to adding invariant attributes. Why do we mark these constant loads with additional attributes?

If alias analysis already knows that all memory in the constant address space is immutable, I agree that should be sufficient, at least on targets where the constant address space is actually a different address space in LLVM IR. If it only knows that when it can resolve an access down to a specific constant global variable, this patch is still providing value because it applies even for accesses through a pointer.

There is plan for amdgcn target to drop constant address space in IR and use global address space instead.

Also, currently there are targets (e.g. x86_64) supporting OpenCL which does not have constant address space in IR.

[rjmccall]

In D75423#1901348, @yaxunl wrote:

In D75423#1901254, @rjmccall wrote:

In D75423#1901206, @hliao wrote:

invariant checking already takes account of loading from constant address space or memory (AA::pointsToConstantMemory), that's almost equivalent to adding invariant attributes. Why do we mark these constant loads with additional attributes?

If alias analysis already knows that all memory in the constant address space is immutable, I agree that should be sufficient, at least on targets where the constant address space is actually a different address space in LLVM IR. If it only knows that when it can resolve an access down to a specific constant global variable, this patch is still providing value because it applies even for accesses through a pointer.

There is plan for amdgcn target to drop constant address space in IR and use global address space instead.

Also, currently there are targets (e.g. x86_64) supporting OpenCL which does not have constant address space in IR.

Okay, then I have no problem taking a patch for this into IRGen. But I think it should be fine to do this by adding the invariant-load metadata when loading from an l-value instead of injecting invariant-groups into l-value emission.

[yaxunl]

In D75423#1901362, @rjmccall wrote:

Okay, then I have no problem taking a patch for this into IRGen. But I think it should be fine to do this by adding the invariant-load metadata when loading from an l-value instead of injecting invariant-groups into l-value emission.

The pointer to constant may be casted to a generic pointer then loaded. If we only mark load from pointer to constant, we loses information.

[yaxunl]

revised by John's comments.

[rjmccall]

In D75423#1901407, @yaxunl wrote:

In D75423#1901362, @rjmccall wrote:

Okay, then I have no problem taking a patch for this into IRGen. But I think it should be fine to do this by adding the invariant-load metadata when loading from an l-value instead of injecting invariant-groups into l-value emission.

The pointer to constant may be casted to a generic pointer then loaded. If we only mark load from pointer to constant, we loses information.

A lot of those conversions don't go through EmitLValue.

Doing this at this level in EmitLValue both complicates the code and will introduce a lot of redundant intrinsic calls. If you need to do it, you should do it at the root where a pointer is being turned into an LValue.

clang/lib/CodeGen/CGDeclCXX.cpp

 /// Emit code to cause the variable at the given address to be considered as
 /// constant from this point onwards.
 static void EmitDeclInvariant(CodeGenFunction &CGF, const VarDecl &D,
                               llvm::Constant *Addr) {
   return CGF.EmitInvariantStart(
       Addr, CGF.getContext().getTypeSizeInChars(D.getType()));
 }
 
-void CodeGenFunction::EmitInvariantStart(llvm::Constant *Addr, CharUnits Size) {
+void CodeGenFunction::EmitInvariantStart(llvm::Value *Addr, CharUnits Size) {
   // Do not emit the intrinsic if we're not optimizing.
   if (!CGM.getCodeGenOpts().OptimizationLevel)
     return;
 
   // Grab the llvm.invariant.start intrinsic.
   llvm::Intrinsic::ID InvStartID = llvm::Intrinsic::invariant_start;
   // Overloaded address space type.
-  llvm::Type *ObjectPtr[1] = {Int8PtrTy};
+  llvm::Type *ObjectPtr[1] = {
+      Int8Ty->getPointerTo(Addr->getType()->getPointerAddressSpace())};
   llvm::Function *InvariantStart = CGM.getIntrinsic(InvStartID, ObjectPtr);
 
   // Emit a call with the size in bytes of the object.
-  uint64_t Width = Size.getQuantity();
-  llvm::Value *Args[2] = { llvm::ConstantInt::getSigned(Int64Ty, Width),
-                           llvm::ConstantExpr::getBitCast(Addr, Int8PtrTy)};
-  Builder.CreateCall(InvariantStart, Args);
+  Builder.CreateCall(InvariantStart,
+                     {llvm::ConstantInt::getSigned(Int64Ty, Size.getQuantity()),
+                      Builder.CreateBitCast(Addr, ObjectPtr[0])});
 }
 
 void CodeGenFunction::EmitCXXGlobalVarDeclInit(const VarDecl &D,
                                                llvm::Constant *DeclPtr,
                                                bool PerformInit) {
 
   const Expr *Init = D.getInit();
   QualType T = D.getType();

clang/lib/CodeGen/CGExpr.cpp

 ///
 /// If this returns a normal address, and if the lvalue's C type is fixed size,
 /// this method guarantees that the returned pointer type will point to an LLVM
 /// type of the same size of the lvalue's type.  If the lvalue has a variable
 /// length type, this is not possible.
 ///
 LValue CodeGenFunction::EmitLValue(const Expr *E) {
   ApplyDebugLocation DL(*this, E);
+  LValue Ret;
   switch (E->getStmtClass()) {
-  default: return EmitUnsupportedLValue(E, "l-value expression");
+  default:
+    Ret = EmitUnsupportedLValue(E, "l-value expression");
+    break;
 
   case Expr::ObjCPropertyRefExprClass:
     llvm_unreachable("cannot emit a property reference directly");
+    break;
 
   case Expr::ObjCSelectorExprClass:
-    return EmitObjCSelectorLValue(cast<ObjCSelectorExpr>(E));
+    Ret = EmitObjCSelectorLValue(cast<ObjCSelectorExpr>(E));
+    break;
   case Expr::ObjCIsaExprClass:
-    return EmitObjCIsaExpr(cast<ObjCIsaExpr>(E));
+    Ret = EmitObjCIsaExpr(cast<ObjCIsaExpr>(E));
+    break;
   case Expr::BinaryOperatorClass:
-    return EmitBinaryOperatorLValue(cast<BinaryOperator>(E));
+    Ret = EmitBinaryOperatorLValue(cast<BinaryOperator>(E));
+    break;
   case Expr::CompoundAssignOperatorClass: {
     QualType Ty = E->getType();
     if (const AtomicType *AT = Ty->getAs<AtomicType>())
       Ty = AT->getValueType();
     if (!Ty->isAnyComplexType())
-      return EmitCompoundAssignmentLValue(cast<CompoundAssignOperator>(E));
-    return EmitComplexCompoundAssignmentLValue(cast<CompoundAssignOperator>(E));
+      Ret = EmitCompoundAssignmentLValue(cast<CompoundAssignOperator>(E));
+    else
+      Ret =
+          EmitComplexCompoundAssignmentLValue(cast<CompoundAssignOperator>(E));
+    break;
   }
   case Expr::CallExprClass:
   case Expr::CXXMemberCallExprClass:
   case Expr::CXXOperatorCallExprClass:
   case Expr::UserDefinedLiteralClass:
-    return EmitCallExprLValue(cast<CallExpr>(E));
+    Ret = EmitCallExprLValue(cast<CallExpr>(E));
+    break;
   case Expr::CXXRewrittenBinaryOperatorClass:
-    return EmitLValue(cast<CXXRewrittenBinaryOperator>(E)->getSemanticForm());
+    Ret = EmitLValue(cast<CXXRewrittenBinaryOperator>(E)->getSemanticForm());
+    break;
   case Expr::VAArgExprClass:
-    return EmitVAArgExprLValue(cast<VAArgExpr>(E));
+    Ret = EmitVAArgExprLValue(cast<VAArgExpr>(E));
+    break;
   case Expr::DeclRefExprClass:
-    return EmitDeclRefLValue(cast<DeclRefExpr>(E));
+    Ret = EmitDeclRefLValue(cast<DeclRefExpr>(E));
+    break;
   case Expr::ConstantExprClass:
-    return EmitLValue(cast<ConstantExpr>(E)->getSubExpr());
+    Ret = EmitLValue(cast<ConstantExpr>(E)->getSubExpr());
+    break;
   case Expr::ParenExprClass:
-    return EmitLValue(cast<ParenExpr>(E)->getSubExpr());
+    Ret = EmitLValue(cast<ParenExpr>(E)->getSubExpr());
+    break;
   case Expr::GenericSelectionExprClass:
-    return EmitLValue(cast<GenericSelectionExpr>(E)->getResultExpr());
+    Ret = EmitLValue(cast<GenericSelectionExpr>(E)->getResultExpr());
+    break;
   case Expr::PredefinedExprClass:
-    return EmitPredefinedLValue(cast<PredefinedExpr>(E));
+    Ret = EmitPredefinedLValue(cast<PredefinedExpr>(E));
+    break;
   case Expr::StringLiteralClass:
-    return EmitStringLiteralLValue(cast<StringLiteral>(E));
+    Ret = EmitStringLiteralLValue(cast<StringLiteral>(E));
+    break;
   case Expr::ObjCEncodeExprClass:
-    return EmitObjCEncodeExprLValue(cast<ObjCEncodeExpr>(E));
+    Ret = EmitObjCEncodeExprLValue(cast<ObjCEncodeExpr>(E));
+    break;
   case Expr::PseudoObjectExprClass:
-    return EmitPseudoObjectLValue(cast<PseudoObjectExpr>(E));
+    Ret = EmitPseudoObjectLValue(cast<PseudoObjectExpr>(E));
+    break;
   case Expr::InitListExprClass:
-    return EmitInitListLValue(cast<InitListExpr>(E));
+    Ret = EmitInitListLValue(cast<InitListExpr>(E));
+    break;
   case Expr::CXXTemporaryObjectExprClass:
   case Expr::CXXConstructExprClass:
-    return EmitCXXConstructLValue(cast<CXXConstructExpr>(E));
+    Ret = EmitCXXConstructLValue(cast<CXXConstructExpr>(E));
+    break;
   case Expr::CXXBindTemporaryExprClass:
-    return EmitCXXBindTemporaryLValue(cast<CXXBindTemporaryExpr>(E));
+    Ret = EmitCXXBindTemporaryLValue(cast<CXXBindTemporaryExpr>(E));
+    break;
   case Expr::CXXUuidofExprClass:
-    return EmitCXXUuidofLValue(cast<CXXUuidofExpr>(E));
+    Ret = EmitCXXUuidofLValue(cast<CXXUuidofExpr>(E));
+    break;
   case Expr::LambdaExprClass:
-    return EmitAggExprToLValue(E);
+    Ret = EmitAggExprToLValue(E);
+    break;
 
   case Expr::ExprWithCleanupsClass: {
     const auto *cleanups = cast<ExprWithCleanups>(E);
     enterFullExpression(cleanups);
     RunCleanupsScope Scope(*this);
     LValue LV = EmitLValue(cleanups->getSubExpr());
     if (LV.isSimple()) {
       // Defend against branches out of gnu statement expressions surrounded by
       // cleanups.
       llvm::Value *V = LV.getPointer(*this);
       Scope.ForceCleanup({&V});
-      return LValue::MakeAddr(Address(V, LV.getAlignment()), LV.getType(),
-                              getContext(), LV.getBaseInfo(), LV.getTBAAInfo());
+      Ret = LValue::MakeAddr(Address(V, LV.getAlignment()), LV.getType(),
     }
     // FIXME: Is it possible to create an ExprWithCleanups that produces a
     // bitfield lvalue or some other non-simple lvalue?
-    return LV;
+    else
+      Ret = LV;
+    break;
   }
 
   case Expr::CXXDefaultArgExprClass: {
     auto *DAE = cast<CXXDefaultArgExpr>(E);
     CXXDefaultArgExprScope Scope(*this, DAE);
-    return EmitLValue(DAE->getExpr());
+    Ret = EmitLValue(DAE->getExpr());
+    break;
   }
   case Expr::CXXDefaultInitExprClass: {
     auto *DIE = cast<CXXDefaultInitExpr>(E);
     CXXDefaultInitExprScope Scope(*this, DIE);
-    return EmitLValue(DIE->getExpr());
+    Ret = EmitLValue(DIE->getExpr());
+    break;
   }
   case Expr::CXXTypeidExprClass:
-    return EmitCXXTypeidLValue(cast<CXXTypeidExpr>(E));
+    Ret = EmitCXXTypeidLValue(cast<CXXTypeidExpr>(E));
+    break;
 
   case Expr::ObjCMessageExprClass:
-    return EmitObjCMessageExprLValue(cast<ObjCMessageExpr>(E));
+    Ret = EmitObjCMessageExprLValue(cast<ObjCMessageExpr>(E));
+    break;
   case Expr::ObjCIvarRefExprClass:
-    return EmitObjCIvarRefLValue(cast<ObjCIvarRefExpr>(E));
+    Ret = EmitObjCIvarRefLValue(cast<ObjCIvarRefExpr>(E));
+    break;
   case Expr::StmtExprClass:
-    return EmitStmtExprLValue(cast<StmtExpr>(E));
+    Ret = EmitStmtExprLValue(cast<StmtExpr>(E));
+    break;
   case Expr::UnaryOperatorClass:
-    return EmitUnaryOpLValue(cast<UnaryOperator>(E));
+    Ret = EmitUnaryOpLValue(cast<UnaryOperator>(E));
+    break;
   case Expr::ArraySubscriptExprClass:
-    return EmitArraySubscriptExpr(cast<ArraySubscriptExpr>(E));
+    Ret = EmitArraySubscriptExpr(cast<ArraySubscriptExpr>(E));
+    break;
   case Expr::OMPArraySectionExprClass:
-    return EmitOMPArraySectionExpr(cast<OMPArraySectionExpr>(E));
+    Ret = EmitOMPArraySectionExpr(cast<OMPArraySectionExpr>(E));
+    break;
   case Expr::ExtVectorElementExprClass:
-    return EmitExtVectorElementExpr(cast<ExtVectorElementExpr>(E));
+    Ret = EmitExtVectorElementExpr(cast<ExtVectorElementExpr>(E));
+    break;
   case Expr::MemberExprClass:
-    return EmitMemberExpr(cast<MemberExpr>(E));
+    Ret = EmitMemberExpr(cast<MemberExpr>(E));
+    break;
   case Expr::CompoundLiteralExprClass:
-    return EmitCompoundLiteralLValue(cast<CompoundLiteralExpr>(E));
+    Ret = EmitCompoundLiteralLValue(cast<CompoundLiteralExpr>(E));
+    break;
   case Expr::ConditionalOperatorClass:
-    return EmitConditionalOperatorLValue(cast<ConditionalOperator>(E));
+    Ret = EmitConditionalOperatorLValue(cast<ConditionalOperator>(E));
+    break;
   case Expr::BinaryConditionalOperatorClass:
-    return EmitConditionalOperatorLValue(cast<BinaryConditionalOperator>(E));
+    Ret = EmitConditionalOperatorLValue(cast<BinaryConditionalOperator>(E));
+    break;
   case Expr::ChooseExprClass:
-    return EmitLValue(cast<ChooseExpr>(E)->getChosenSubExpr());
+    Ret = EmitLValue(cast<ChooseExpr>(E)->getChosenSubExpr());
+    break;
   case Expr::OpaqueValueExprClass:
-    return EmitOpaqueValueLValue(cast<OpaqueValueExpr>(E));
+    Ret = EmitOpaqueValueLValue(cast<OpaqueValueExpr>(E));
+    break;
   case Expr::SubstNonTypeTemplateParmExprClass:
-    return EmitLValue(cast<SubstNonTypeTemplateParmExpr>(E)->getReplacement());
+    Ret = EmitLValue(cast<SubstNonTypeTemplateParmExpr>(E)->getReplacement());
+    break;
   case Expr::ImplicitCastExprClass:
   case Expr::CStyleCastExprClass:
   case Expr::CXXFunctionalCastExprClass:
   case Expr::CXXStaticCastExprClass:
   case Expr::CXXDynamicCastExprClass:
   case Expr::CXXReinterpretCastExprClass:
   case Expr::CXXConstCastExprClass:
   case Expr::ObjCBridgedCastExprClass:
-    return EmitCastLValue(cast<CastExpr>(E));
+    Ret = EmitCastLValue(cast<CastExpr>(E));
+    break;
 
   case Expr::MaterializeTemporaryExprClass:
-    return EmitMaterializeTemporaryExpr(cast<MaterializeTemporaryExpr>(E));
+    Ret = EmitMaterializeTemporaryExpr(cast<MaterializeTemporaryExpr>(E));
+    break;
 
   case Expr::CoawaitExprClass:
-    return EmitCoawaitLValue(cast<CoawaitExpr>(E));
+    Ret = EmitCoawaitLValue(cast<CoawaitExpr>(E));
+    break;
   case Expr::CoyieldExprClass:
-    return EmitCoyieldLValue(cast<CoyieldExpr>(E));
+    Ret = EmitCoyieldLValue(cast<CoyieldExpr>(E));
+    break;
   }
+
+  // Mark a pointer to OpenCL constant memory as invariant.
+  // ToDo: Currently we only handle simple l-value. We should also handle other
+  // l-values.
+  if (Ret.getAddressSpace() == LangAS::opencl_constant && Ret.isSimple()) {
+    EmitInvariantStart(Ret.getPointer(*this),
+                       getContext().getTypeSizeInChars(Ret.getType()));
+  }
+
+  return Ret;
 }
 
 /// Given an object of the given canonical type, can we safely copy a
 /// value out of it based on its initializer?
 static bool isConstantEmittableObjectType(QualType type) {
   assert(type.isCanonical());
   assert(!type->isReferenceType());
 
if (CGM.getCodeGenOpts().StrictVTablePointers &&
       // fields may leak the real address of dynamic object, which could result
       // in miscompilation when leaked pointer would be compared.
       auto *stripped = Builder.CreateStripInvariantGroup(addr.getPointer());
       addr = Address(stripped, addr.getAlignment());
     }
   }
 
   unsigned RecordCVR = base.getVRQualifiers();
+  auto RecordAddrSpace = base.getAddressSpace();
   if (rec->isUnion()) {
     // For unions, there is no pointer adjustment.
     if (CGM.getCodeGenOpts().StrictVTablePointers &&
         hasAnyVptr(FieldType, getContext()))
       // Because unions can easily skip invariant.barriers, we need to add
       // a barrier every time CXXRecord field with vptr is referenced.
       addr = Address(Builder.CreateLaunderInvariantGroup(addr.getPointer()),
                      addr.getAlignment());
LValue CodeGenFunction::EmitLValueForField(LValue base,
   addr = Builder.CreateElementBitCast(
       addr, CGM.getTypes().ConvertTypeForMem(FieldType), field->getName());
 
   if (field->hasAttr<AnnotateAttr>())
     addr = EmitFieldAnnotations(field, addr);
 
   LValue LV = MakeAddrLValue(addr, FieldType, FieldBaseInfo, FieldTBAAInfo);
   LV.getQuals().addCVRQualifiers(RecordCVR);
+  LV.getQuals().setAddressSpace(RecordAddrSpace);
 
   // __weak attribute on a field is ignored.
   if (LV.getQuals().getObjCGCAttr() == Qualifiers::Weak)
     LV.getQuals().removeObjCGCAttr();
 
   return LV;
 }
 

clang/lib/CodeGen/CGExprScalar.cpp

public:
   LValue EmitCheckedLValue(const Expr *E, CodeGenFunction::TypeCheckKind TCK) {
     return CGF.EmitCheckedLValue(E, TCK);
   }
 
   void EmitBinOpCheck(ArrayRef<std::pair<Value *, SanitizerMask>> Checks,
                       const BinOpInfo &Info);
 
   Value *EmitLoadOfLValue(LValue LV, SourceLocation Loc) {
-    return CGF.EmitLoadOfLValue(LV, Loc).getScalarVal();
+    auto *V = CGF.EmitLoadOfLValue(LV, Loc).getScalarVal();
+    // Mark a pointer to OpenCL constant address space as invariant.
+    auto QT = LV.getType();
+    if (QT->isPointerType()) {
+      auto PointeeTy = QT->getPointeeType();
+      if (PointeeTy.getAddressSpace() == LangAS::opencl_constant) {
+        CGF.EmitInvariantStart(V,
+                               CGF.getContext().getTypeSizeInChars(PointeeTy));
+      }
+    }
+    return V;
   }
 
   void EmitLValueAlignmentAssumption(const Expr *E, Value *V) {
     const AlignValueAttr *AVAttr = nullptr;
     if (const auto *DRE = dyn_cast<DeclRefExpr>(E)) {
       const ValueDecl *VD = DRE->getDecl();
 
       if (VD->getType()->isReferenceType()) {

clang/lib/CodeGen/CodeGenFunction.h

public:
   /// global variable that has already been created for it.  If the initializer
   /// has a different type than GV does, this may free GV and return a different
   /// one.  Otherwise it just returns GV.
   llvm::GlobalVariable *
   AddInitializerToStaticVarDecl(const VarDecl &D,
                                 llvm::GlobalVariable *GV);
 
   // Emit an @llvm.invariant.start call for the given memory region.
-  void EmitInvariantStart(llvm::Constant *Addr, CharUnits Size);
+  void EmitInvariantStart(llvm::Value *Addr, CharUnits Size);
 
   /// EmitCXXGlobalVarDeclInit - Create the initializer for a C++
   /// variable with global storage.
   void EmitCXXGlobalVarDeclInit(const VarDecl &D, llvm::Constant *DeclPtr,
                                 bool PerformInit);
 
   llvm::Function *createAtExitStub(const VarDecl &VD, llvm::FunctionCallee Dtor,
                                    llvm::Constant *Addr);

clang/test/CodeGenOpenCL/invariant.cl

+// RUN: %clang_cc1 -triple amdgcn-amd-amdhsa -O3 -emit-llvm -o - %s | FileCheck %s
+// RUN: %clang_cc1 -triple spir -O3 -emit-llvm -o - %s | FileCheck %s
+
+typedef struct {
+  int a;
+  char b;
+} X;
+
+constant X x = {0, 'a'};
+
+constant char* constant p = &(x.b);
+
+constant X* foo();
+
+// CHECK-LABEL: test1
+// CHECK: llvm.invariant.start
+char test1() {
+  return x.b;
+}
+
+// CHECK-LABEL: test2
+// CHECK: llvm.invariant.start
+char test2() {
+  return *p;
+}
+
+// CHECK-LABEL: test3
+// CHECK: llvm.invariant.start
+char test3(constant X *x) {
+  constant char *p = &(x->b);
+  return *p;
+}
+
+// CHECK-LABEL: test4
+// CHECK: llvm.invariant.start
+char test4() {
+  return foo()->b;
+}

clang/test/CodeGenOpenCL/printf.cl

 // RUN: %clang_cc1 -cl-std=CL1.2 -cl-ext=-+cl_khr_fp64 -triple spir-unknown-unknown -disable-llvm-passes -emit-llvm -o - %s | FileCheck -check-prefixes=FP64,ALL %s
 // RUN: %clang_cc1 -cl-std=CL1.2 -cl-ext=-cl_khr_fp64 -triple spir-unknown-unknown -disable-llvm-passes -emit-llvm -o - %s | FileCheck -check-prefixes=NOFP64,ALL %s
 
 typedef __attribute__((ext_vector_type(2))) float float2;
 typedef __attribute__((ext_vector_type(2))) half half2;
 
 #ifdef cl_khr_fp64
 typedef __attribute__((ext_vector_type(2))) double double2;
 #endif
 
 int printf(__constant const char* st, ...) __attribute__((format(printf, 1, 2)));
 
 
 // ALL-LABEL: @test_printf_float2(
-// FP64: %call = call spir_func i32 (i8 addrspace(2)*, ...) @printf(i8 addrspace(2)* getelementptr inbounds ([7 x i8], [7 x i8] addrspace(2)* @.str, i32 0, i32 0), <2 x float> %0)
+// FP64: %call = call spir_func i32 (i8 addrspace(2)*, ...) @printf(i8 addrspace(2)* getelementptr inbounds ([7 x i8], [7 x i8] addrspace(2)* @.str, i32 0, i32 0), <2 x float>
 
 
-// NOFP64:  call spir_func i32 (i8 addrspace(2)*, ...) @printf(i8 addrspace(2)* getelementptr inbounds ([7 x i8], [7 x i8] addrspace(2)* @.str, i32 0, i32 0), <2 x float> %0)
+// NOFP64:  call spir_func i32 (i8 addrspace(2)*, ...) @printf(i8 addrspace(2)* getelementptr inbounds ([7 x i8], [7 x i8] addrspace(2)* @.str, i32 0, i32 0), <2 x float>
 kernel void test_printf_float2(float2 arg) {
   printf("%v2hlf", arg);
 }
 
 // ALL-LABEL: @test_printf_half2(
-// FP64:  %call = call spir_func i32 (i8 addrspace(2)*, ...) @printf(i8 addrspace(2)* getelementptr inbounds ([6 x i8], [6 x i8] addrspace(2)* @.str.1, i32 0, i32 0), <2 x half> %0)
+// FP64:  %call = call spir_func i32 (i8 addrspace(2)*, ...) @printf(i8 addrspace(2)* getelementptr inbounds ([6 x i8], [6 x i8] addrspace(2)* @.str.1, i32 0, i32 0), <2 x half>
 
-// NOFP64:  %call = call spir_func i32 (i8 addrspace(2)*, ...) @printf(i8 addrspace(2)* getelementptr inbounds ([6 x i8], [6 x i8] addrspace(2)* @.str.1, i32 0, i32 0), <2 x half> %0)
+// NOFP64:  %call = call spir_func i32 (i8 addrspace(2)*, ...) @printf(i8 addrspace(2)* getelementptr inbounds ([6 x i8], [6 x i8] addrspace(2)* @.str.1, i32 0, i32 0), <2 x half>
 kernel void test_printf_half2(half2 arg) {
   printf("%v2hf", arg);
 }
 
 #ifdef cl_khr_fp64
 // FP64-LABEL: @test_printf_double2(
-// FP64: call spir_func i32 (i8 addrspace(2)*, ...) @printf(i8 addrspace(2)* getelementptr inbounds ([6 x i8], [6 x i8] addrspace(2)* @.str.2, i32 0, i32 0), <2 x double> %0)
+// FP64: call spir_func i32 (i8 addrspace(2)*, ...) @printf(i8 addrspace(2)* getelementptr inbounds ([6 x i8], [6 x i8] addrspace(2)* @.str.2, i32 0, i32 0), <2 x double>
 kernel void test_printf_double2(double2 arg) {
   printf("%v2lf", arg);
 }
 #endif