[Coroutines] Add coro_maychange intrinsic to solve TLS problem (2/5)

[ChuanqiXu]

Summary

This is intended to fix the TLS problem in coroutine described in https://github.com/llvm/llvm-project/issues/47179

Simply, we would assume the address of a TLS variable is same in one function. Since a function should be executed in one thread only. However, it is not true for unlowered coroutine. This patch tries to fix the problem by adding a wrapper for every TLS variable to block the alias analysis. Note that we couldn't do this for unlowered coroutine only due to inlining. Also the compiler is still available to optimize TLS variables for lowered coroutine.

[efriedma]

llvm/lib/Transforms/Coroutines/CoroEarly.cpp

265

UserInst might not be a legal insertion point for a call if it's a PHI node.

[ChuanqiXu]

llvm/lib/Transforms/Coroutines/CoroEarly.cpp

265

Oh, good Catcha!

[ChuanqiXu]

Emit llvm.coro.maychange in the frontend according to the discussion in https://discourse.llvm.org/t/address-thread-identification-problems-with-coroutine/62015

[rjmccall]

clang/lib/CodeGen/CGExpr.cpp

2623 ↗

(On Diff #425470)

I guess this is unnecessary under OpenMP because the privatization logic will already anchor this appropriately in the function. That's worth mentioning in a comment so that readers don't think the combination is somehow busted.

Same question as the other patch: is there any way to safely only do this in code that's actually going to be part of a coroutine? Because getLangOpts().Coroutines is true for everyone using -std=c++20, and most of that code is probably not using coroutines. It seems like we have two options:

• Use this pattern everywhere in the translation unit, and then eliminate it from all functions after we've split all coroutines
• Use this pattern in the frontend when directly emitting unsplit coroutine bodies, and also change LLVM to introduce this pattern when cloning code into an unsplit coroutine body (most importantly, in the inliner)

I think the second option makes more sense. In the first option, the mere possibility of having coroutines in the module could significantly impede optimization. The second option also means you'll have to find and remove this pattern in *all* functions, not just when transforming coroutines.

You could also use the second option for your readnone problem: you can have your early pass make readnone calls directly from coroutine bodies coro_readnone instead, and the inliner can do the same for readnone calls being inlined into unsplit coroutines.

[efriedma]

clang/lib/CodeGen/CGExpr.cpp

2623 ↗

(On Diff #425470)

I'm not really happy with the idea that IR semantics are different in the body of a coroutine; that implies that every interprocedural analysis or optimization author has to think about the possible interactions with coroutine semantics. It's much simpler to just pretend a coroutine is a thread. I mean, you mention inlining, but there's also IPSCCP, and Attributor, maybe GlobalOpt, and other interactions I'm probably not thinking of.

And really, as long as alias analysis understands llvm.coro.maychange, we can do almost all the optimizations we want to do on thread-local variables anyway.

But maybe I'm leaning too far towards conceptual purity, as opposed to practicality.

[rjmccall]

clang/lib/CodeGen/CGExpr.cpp

2623 ↗

(On Diff #425470)

Like I said in Discourse, I think our hands are pretty forced here for TLS, because LLVM IR is assuming in its basic representation that functions are pinned to a single thread, which simply is not true in coroutine bodies. We have three options:

1. We can globally change the structure of TLS access. You would not be able to simply use thread_local GVs as constants; instead, you would have to use some instruction which yields the address of the TLS for the current thread.
2. We can require different structure for TLS access in unlowered coroutines.
3. We can decline to support coroutines.

I don't think #3 is an option, and #1 isn't really a fight I want to fight, so we're left with #2. Maybe there's an option I'm not thinking of, though.

[efriedma]

clang/lib/CodeGen/CGExpr.cpp

2623 ↗

(On Diff #425470)

I think that's basically the three options, yes.

I guess strictly speaking, there is a fourth option: we could mess with the semantics of coroutines. We can say, for example, that the address of thread-local variables is always the address on entry to the coroutine. Then we can just hoist the computation to the entry block of the coroutine. But the spec probably doesn't allow that.

My intuition is that #1 is not actually that terrible. We don't really optimize thread-local variables very much anyway; as long as alias analysis doesn't explode, we're probably don't miss many optimizations. And I'm not sure #2 is actually significantly less work to implement. But I guess it's easier to prove the changes involved in #2 don't have any impact on code that doesn't use coroutines.

[ChuanqiXu]

Address comments:

• Don't add filter for OpenMP
• Add readonly and inaccessiblememonly attribute to llvm.coro.may_change intrinsics.

---

I think it is better to discuss whether or not to insert llvm.coro.may_change in non-coroutines in discourse: https://discourse.llvm.org/t/address-thread-identification-problems-with-coroutine/62015

[ChuanqiXu]

Abandon this one since we prefer the proposal from James in https://discourse.llvm.org/t/address-thread-identification-problems-with-coroutine/62015/21

llvm/include/llvm/IR/Intrinsics.td

 
 // Coroutine Lowering Intrinsics. Used internally by coroutine passes.
 
 def int_coro_subfn_addr : Intrinsic<[llvm_ptr_ty], [llvm_ptr_ty, llvm_i8_ty],
                                     [IntrReadMem, IntrArgMemOnly,
                                      ReadOnly<ArgIndex<0>>,
                                      NoCapture<ArgIndex<0>>]>;
 
+// Coroutine Lowering Intrinsics to block optimizations. Used internally by 
+// coroutine passes.
+def int_coro_maychange : Intrinsic<[llvm_ptr_ty], [llvm_ptr_ty]>;
+
 ///===-------------------------- Other Intrinsics --------------------------===//
 //
 def int_trap : Intrinsic<[], [], [IntrNoReturn, IntrCold]>,
                GCCBuiltin<"__builtin_trap">;
 def int_debugtrap : Intrinsic<[]>,
                     GCCBuiltin<"__builtin_debugtrap">;
 def int_ubsantrap : Intrinsic<[], [llvm_i8_ty],
                               [IntrNoReturn, IntrCold, ImmArg<ArgIndex<0>>]>;

llvm/lib/Transforms/Coroutines/CoroCleanup.cpp

if (auto *II = dyn_cast<IntrinsicInst>(&I)) {
         break;
       case Intrinsic::coro_end:
       case Intrinsic::coro_suspend_retcon:
         if (IsPrivateAndUnprocessed) {
           II->replaceAllUsesWith(UndefValue::get(II->getType()));
         } else
           continue;
         break;
+      case Intrinsic::coro_maychange:
+        II->replaceAllUsesWith(II->getOperand(0));
+        break;
       case Intrinsic::coro_async_size_replace:
         auto *Target = cast<ConstantStruct>(
             cast<GlobalVariable>(II->getArgOperand(0)->stripPointerCasts())
                 ->getInitializer());
         auto *Source = cast<ConstantStruct>(
             cast<GlobalVariable>(II->getArgOperand(1)->stripPointerCasts())
                 ->getInitializer());
         auto *TargetSize = Target->getOperand(1);
         auto *SourceSize = Source->getOperand(1);
         if (TargetSize->isElementWiseEqual(SourceSize)) {
           break;
         }
         auto *TargetRelativeFunOffset = Target->getOperand(0);
         auto *NewFuncPtrStruct = ConstantStruct::get(
             Target->getType(), TargetRelativeFunOffset, SourceSize);
         Target->replaceAllUsesWith(NewFuncPtrStruct);
         break;
       }
+
       II->eraseFromParent();
       Changed = true;
     }
   }
 
   if (Changed) {
     // After replacement were made we can cleanup the function body a little.
     simplifyCFG(F);
   }
 
   return Changed;
 }
 
 static bool declaresCoroCleanupIntrinsics(const Module &M) {
   return coro::declaresIntrinsics(
       M, {"llvm.coro.alloc", "llvm.coro.begin", "llvm.coro.subfn.addr",
           "llvm.coro.free", "llvm.coro.id", "llvm.coro.id.retcon",
           "llvm.coro.id.retcon.once", "llvm.coro.async.size.replace",
-          "llvm.coro.async.resume"});
+          "llvm.coro.async.resume", "llvm.coro.maychange"});
 }
 
 PreservedAnalyses CoroCleanupPass::run(Function &F,
                                        FunctionAnalysisManager &AM) {
   auto &M = *F.getParent();
   if (!declaresCoroCleanupIntrinsics(M) ||
       !Lowerer(M).lowerRemainingCoroIntrinsics(F))
     return PreservedAnalyses::all();
 
   return PreservedAnalyses::none();
 }

llvm/lib/Transforms/Coroutines/CoroEarly.cpp

 PreservedAnalyses CoroEarlyPass::run(Module &M, ModuleAnalysisManager &) {
   if (!declaresCoroEarlyIntrinsics(M))
     return PreservedAnalyses::all();
 
   Lowerer L(M);
   for (auto &F : M)
     L.lowerEarlyIntrinsics(F);
 
+  // Add llvm.coro.maychange to TLS variable to avoid misoptimization.
+  //
+  // Previously, the optimizer could assume the address of a TLS variable is
+  // same in the same function. The assumption is broken now after we introduced
+  // coroutines.
+  //
+  // Note that we couldn't mark the use in coroutines only due to potentially
+  // inlining. It is too expensive to do CFG analysis here to mark the use for
+  // reachable function only. And the TLS variable should be able to be optmized
+  // finally after we removed llvm.coro.maychange intrinsic in CoroCleanup.
+  auto *MaychangeFn = Intrinsic::getDeclaration(&M, Intrinsic::coro_maychange);
+  for (auto &GV : M.getGlobalList())
+    if (GV.isThreadLocal())
+      for (auto &Use : llvm::make_early_inc_range(GV.uses())) {
+        auto *UserInst = dyn_cast<Instruction>(Use.getUser());
+        if (!UserInst)
+          continue;
+
+        Use.set(CallInst::Create(MaychangeFn, {&GV}, GV.getName(), UserInst));
+      }
+
   PreservedAnalyses PA;
   PA.preserveSet<CFGAnalyses>();
   return PA;
 }

llvm/lib/Transforms/Coroutines/Coroutines.cpp

static const char *const CoroIntrinsics[] = {
     "llvm.coro.end",
     "llvm.coro.end.async",
     "llvm.coro.frame",
     "llvm.coro.free",
     "llvm.coro.id",
     "llvm.coro.id.async",
     "llvm.coro.id.retcon",
     "llvm.coro.id.retcon.once",
+    "llvm.coro.maychange",
     "llvm.coro.noop",
     "llvm.coro.prepare.async",
     "llvm.coro.prepare.retcon",
     "llvm.coro.promise",
     "llvm.coro.resume",
     "llvm.coro.save",
     "llvm.coro.size",
     "llvm.coro.subfn.addr",

llvm/test/Transforms/Coroutines/coro-TLS-01.ll

+; Tests that the TLS variables which cross suspend points wouldn't be misoptimized.
+; RUN: opt < %s -S -passes=coro-early,sroa,early-cse,coro-split,coro-cleanup,simplifycfg -opaque-pointers | FileCheck %s
+
+@tls_variable = thread_local global i32 0
+
+define ptr @f() "coroutine.presplit"="0" {
+entry:
+  %id = call token @llvm.coro.id(i32 0, i8* null, i8* null, i8* null)
+  %size = call i32 @llvm.coro.size.i32()
+  %alloc = call i8* @malloc(i32 %size)
+  %i = alloca ptr
+  %j = alloca ptr
+  %hdl = call i8* @llvm.coro.begin(token %id, i8* %alloc)
+  store ptr @tls_variable, ptr %i
+  %sus_result = call i8 @llvm.coro.suspend(token none, i1 false)
+  switch i8 %sus_result, label %suspend [i8 0, label %resume
+                                         i8 1, label %cleanup]
+resume:
+  store ptr @tls_variable, ptr %j
+  %i_value = load ptr, ptr %i
+  %j_value = load ptr, ptr %j
+  %cmp = icmp eq ptr %i_value, %j_value
+  br i1 %cmp, label %same, label %diff
+
+same:
+  call void @print_same()
+  br label %cleanup
+
+diff:
+  call void @print_diff()
+  br label %cleanup
+
+cleanup:
+  %mem = call i8* @llvm.coro.free(token %id, i8* %hdl)
+  call void @free(i8* %mem)
+  br label %suspend
+
+suspend:
+  call i1 @llvm.coro.end(i8* %hdl, i1 0)
+  ret i8* %hdl
+}
+
+; CHECK-LABEL: f.resume(
+; CHECK: br i1 %cmp, label %same, label %diff
+; CHECK-EMPTY:
+; CHECK-NEXT: same:
+; CHECK-NEXT:   call void @print_same()
+; CHECK-NEXT:   br label %cleanup
+; CHECK-EMPTY:
+; CHECK-NEXT: diff:
+; CHECK-NEXT:   call void @print_diff()
+; CHECK-NEXT:   br label %cleanup
+
+declare void @print_same()
+declare void @print_diff()
+declare i8* @llvm.coro.free(token, i8*)
+declare i32 @llvm.coro.size.i32()
+declare i8  @llvm.coro.suspend(token, i1)
+
+declare token @llvm.coro.id(i32, i8*, i8*, i8*)
+declare i1 @llvm.coro.alloc(token)
+declare i8* @llvm.coro.begin(token, i8*)
+declare i1 @llvm.coro.end(i8*, i1)
+
+declare noalias i8* @malloc(i32)
+declare void @free(i8*)

llvm/test/Transforms/Coroutines/coro-TLS-02.ll

+; Tests that the TLS variables which don't cross suspend points would be optimized correctly.
+; RUN: opt < %s -S -passes=coro-early,coro-split,coro-cleanup,sroa,early-cse,simplifycfg -opaque-pointers | FileCheck %s
+
+@tls_variable = thread_local global i32 0
+
+define ptr @f() "coroutine.presplit"="0" {
+entry:
+  %id = call token @llvm.coro.id(i32 0, i8* null, i8* null, i8* null)
+  %size = call i32 @llvm.coro.size.i32()
+  %alloc = call i8* @malloc(i32 %size)
+  %i = alloca ptr
+  %j = alloca ptr
+  %hdl = call i8* @llvm.coro.begin(token %id, i8* %alloc)
+  %sus_result = call i8 @llvm.coro.suspend(token none, i1 false)
+  switch i8 %sus_result, label %suspend [i8 0, label %resume
+                                         i8 1, label %cleanup]
+resume:
+  store ptr @tls_variable, ptr %i
+  store ptr @tls_variable, ptr %j
+  %i_value = load ptr, ptr %i
+  %j_value = load ptr, ptr %j
+  %cmp = icmp eq ptr %i_value, %j_value
+  br i1 %cmp, label %same, label %diff
+
+same:
+  call void @print_same()
+  br label %cleanup
+
+diff:
+  call void @print_diff()
+  br label %cleanup
+
+cleanup:
+  %mem = call i8* @llvm.coro.free(token %id, i8* %hdl)
+  call void @free(i8* %mem)
+  br label %suspend
+
+suspend:
+  call i1 @llvm.coro.end(i8* %hdl, i1 0)
+  ret i8* %hdl
+}
+
+; CHECK: void @f.resume
+; CHECK-NEXT: entry.resume:
+; CHECK-NEXT: call void @print_same()
+; CHECK-NEXT: call void @free(ptr %hdl)
+; CHECK-NEXT: ret void 
+
+declare void @print_same()
+declare void @print_diff()
+declare i8* @llvm.coro.free(token, i8*)
+declare i32 @llvm.coro.size.i32()
+declare i8  @llvm.coro.suspend(token, i1)
+
+declare token @llvm.coro.id(i32, i8*, i8*, i8*)
+declare i1 @llvm.coro.alloc(token)
+declare i8* @llvm.coro.begin(token, i8*)
+declare i1 @llvm.coro.end(i8*, i1)
+
+declare noalias i8* @malloc(i32)
+declare void @free(i8*)

llvm/test/Transforms/Coroutines/coro-TLS-03.ll

+; Tests that the TLS variables which cross suspend points wouldn't be misoptimized during O2 pipeline.
+; RUN: opt < %s -S -passes='default<O2>' -opaque-pointers | FileCheck %s
+
+@tls_variable = thread_local global i32 0
+
+define ptr @f() "coroutine.presplit"="0" {
+entry:
+  %id = call token @llvm.coro.id(i32 0, i8* null, i8* null, i8* null)
+  %size = call i32 @llvm.coro.size.i32()
+  %alloc = call i8* @malloc(i32 %size)
+  %i = alloca ptr
+  %j = alloca ptr
+  %hdl = call i8* @llvm.coro.begin(token %id, i8* %alloc)
+  store ptr @tls_variable, ptr %i
+  %sus_result = call i8 @llvm.coro.suspend(token none, i1 false)
+  switch i8 %sus_result, label %suspend [i8 0, label %resume
+                                         i8 1, label %cleanup]
+resume:
+  store ptr @tls_variable, ptr %j
+  %i_value = load ptr, ptr %i
+  %j_value = load ptr, ptr %j
+  %cmp = icmp eq ptr %i_value, %j_value
+  br i1 %cmp, label %same, label %diff
+
+same:
+  call void @print_same()
+  br label %cleanup
+
+diff:
+  call void @print_diff()
+  br label %cleanup
+
+cleanup:
+  %mem = call i8* @llvm.coro.free(token %id, i8* %hdl)
+  call void @free(i8* %mem)
+  br label %suspend
+
+suspend:
+  call i1 @llvm.coro.end(i8* %hdl, i1 0)
+  ret i8* %hdl
+}
+
+; CHECK-LABEL: f.resume(
+; CHECK: br i1 %cmp, label %same, label %diff
+; CHECK-EMPTY:
+; CHECK-NEXT: same:
+; CHECK-NEXT:   call void @print_same()
+; CHECK-NEXT:   br label
+; CHECK-EMPTY:
+; CHECK-NEXT: diff:
+; CHECK-NEXT:   call void @print_diff()
+; CHECK-NEXT:   br label
+
+declare void @print_same()
+declare void @print_diff()
+declare i8* @llvm.coro.free(token, i8*)
+declare i32 @llvm.coro.size.i32()
+declare i8  @llvm.coro.suspend(token, i1)
+
+declare token @llvm.coro.id(i32, i8*, i8*, i8*)
+declare i1 @llvm.coro.alloc(token)
+declare i8* @llvm.coro.begin(token, i8*)
+declare i1 @llvm.coro.end(i8*, i1)
+
+declare noalias i8* @malloc(i32)
+declare void @free(i8*)

llvm/test/Transforms/Coroutines/coro-TLS-04.ll

+; Tests that the TLS variables which don't cross suspend points would be optimized correctly during O2 pipelines.
+; RUN: opt < %s -S -passes='default<O2>' -opaque-pointers | FileCheck %s
+
+@tls_variable = thread_local global i32 0
+
+define ptr @f() "coroutine.presplit"="0" {
+entry:
+  %id = call token @llvm.coro.id(i32 0, i8* null, i8* null, i8* null)
+  %size = call i32 @llvm.coro.size.i32()
+  %alloc = call i8* @malloc(i32 %size)
+  %i = alloca ptr
+  %j = alloca ptr
+  %hdl = call i8* @llvm.coro.begin(token %id, i8* %alloc)
+  %sus_result = call i8 @llvm.coro.suspend(token none, i1 false)
+  switch i8 %sus_result, label %suspend [i8 0, label %resume
+                                         i8 1, label %cleanup]
+resume:
+  store ptr @tls_variable, ptr %i
+  store ptr @tls_variable, ptr %j
+  %i_value = load ptr, ptr %i
+  %j_value = load ptr, ptr %j
+  %cmp = icmp eq ptr %i_value, %j_value
+  br i1 %cmp, label %same, label %diff
+
+same:
+  call void @print_same()
+  br label %cleanup
+
+diff:
+  call void @print_diff()
+  br label %cleanup
+
+cleanup:
+  %mem = call i8* @llvm.coro.free(token %id, i8* %hdl)
+  call void @free(i8* %mem)
+  br label %suspend
+
+suspend:
+  call i1 @llvm.coro.end(i8* %hdl, i1 0)
+  ret i8* %hdl
+}
+
+; CHECK: void @f.resume
+; CHECK-NEXT: resume:
+; CHECK-NEXT: call void @print_same(
+; CHECK-NEXT: call void @free(
+; CHECK-NEXT: ret void 
+
+declare void @print_same()
+declare void @print_diff()
+declare i8* @llvm.coro.free(token, i8*)
+declare i32 @llvm.coro.size.i32()
+declare i8  @llvm.coro.suspend(token, i1)
+
+declare token @llvm.coro.id(i32, i8*, i8*, i8*)
+declare i1 @llvm.coro.alloc(token)
+declare i8* @llvm.coro.begin(token, i8*)
+declare i1 @llvm.coro.end(i8*, i1)
+
+declare noalias i8* @malloc(i32)
+declare void @free(i8*)

llvm/test/Transforms/Coroutines/coro-cleanup-maychange.ll

+; Test that coro-cleanup would convert llvm.coro.maychange intrinsics
+; correctly.
+; RUN: opt < %s -S -passes=coro-cleanup -opaque-pointers | FileCheck %s
+
+%f.Frame = type { ptr, ptr, ptr, ptr, i1 }
+
+@tls_variable = thread_local global i32 0
[e-mail] = private constant [3 x ptr] [ptr @f.resume, ptr @f.destroy, ptr @f.cleanup]
+
+define ptr @f() {
+entry:
+  %id = call token @llvm.coro.id(i32 0, ptr null, ptr @f, ptr @f.resumers)
+  %alloc = call ptr @malloc(i32 40)
+  %i = alloca ptr, align 8
+  %j = alloca ptr, align 8
+  %hdl = call noalias nonnull ptr @llvm.coro.begin(token %id, ptr %alloc)
+  %resume.addr = getelementptr inbounds %f.Frame, ptr %hdl, i32 0, i32 0
+  store ptr @f.resume, ptr %resume.addr, align 8
+  %destroy.addr = getelementptr inbounds %f.Frame, ptr %hdl, i32 0, i32 1
+  store ptr @f.destroy, ptr %destroy.addr, align 8
+  %i.reload.addr = getelementptr inbounds %f.Frame, ptr %hdl, i32 0, i32 2
+  %j.reload.addr = getelementptr inbounds %f.Frame, ptr %hdl, i32 0, i32 3
+  ; CHECK: store ptr @tls_variable, ptr %i.reload.addr, align 8
+  %tls_variable1 = call ptr @llvm.coro.maychange(ptr @tls_variable)
+  store ptr %tls_variable1, ptr %i.reload.addr, align 8
+  %index.addr2 = getelementptr inbounds %f.Frame, ptr %hdl, i32 0, i32 4
+  store i1 false, ptr %index.addr2, align 1
+  ret ptr %hdl
+}
+
+define internal fastcc void @f.resume(ptr noalias nonnull align 8 dereferenceable(40) %hdl) {
+entry.resume:
+  %i.reload.addr = getelementptr inbounds %f.Frame, ptr %hdl, i32 0, i32 2
+  %j.reload.addr = getelementptr inbounds %f.Frame, ptr %hdl, i32 0, i32 3
+  %tls_variable = call ptr @llvm.coro.maychange(ptr @tls_variable)
+  ; CHECK: store ptr @tls_variable, ptr %j.reload.addr, align 8
+  store ptr %tls_variable, ptr %j.reload.addr, align 8
+  call void @consume(ptr %i.reload.addr)
+  call void @consume(ptr %j.reload.addr)
+  call void @free(ptr %hdl)
+  ret void
+}
+
+define internal fastcc void @f.destroy(ptr noalias nonnull align 8 dereferenceable(40) %hdl) {
+entry.destroy:
+  %i.reload.addr = getelementptr inbounds %f.Frame, ptr %hdl, i32 0, i32 2
+  %j.reload.addr = getelementptr inbounds %f.Frame, ptr %hdl, i32 0, i32 3
+  call void @free(ptr %hdl)
+  ret void
+}
+
+define internal fastcc void @f.cleanup(ptr noalias nonnull align 8 dereferenceable(40) %hdl) {
+entry.cleanup:
+  %i.reload.addr = getelementptr inbounds %f.Frame, ptr %hdl, i32 0, i32 2
+  %j.reload.addr = getelementptr inbounds %f.Frame, ptr %hdl, i32 0, i32 3
+  call void @free(ptr null)
+  ret void
+}
+
+declare void @consume(ptr)
+declare i8* @llvm.coro.free(token, i8*)
+declare i32 @llvm.coro.size.i32()
+declare i8  @llvm.coro.suspend(token, i1)
+
+declare token @llvm.coro.id(i32, i8*, i8*, i8*)
+declare i1 @llvm.coro.alloc(token)
+declare i8* @llvm.coro.begin(token, i8*)
+declare i1 @llvm.coro.end(i8*, i1)
+
+declare noalias i8* @malloc(i32)
+declare void @free(i8*)
+
+declare ptr @llvm.coro.maychange(ptr)

llvm/test/Transforms/Coroutines/coro-early-maychange.ll

+; Tests that coro-early would generate llvm.coro.maychange intrinsics for TLS
+; variable correctly.
+; RUN: opt < %s -S -passes=coro-early -opaque-pointers | FileCheck %s
+
+@tls_variable = thread_local global i32 0
+
+define i8* @f() "coroutine.presplit"="0" {
+entry:
+  %id = call token @llvm.coro.id(i32 0, i8* null, i8* null, i8* null)
+  %size = call i32 @llvm.coro.size.i32()
+  %alloc = call i8* @malloc(i32 %size)
+  %i = alloca ptr
+  %j = alloca ptr
+  %hdl = call i8* @llvm.coro.begin(token %id, i8* %alloc)
+  ; CHECK: %[[TLS_VARIABLE:.*]] = call ptr @llvm.coro.maychange(ptr @tls_variable)
+  ; CHECK-NEXT: store ptr %[[TLS_VARIABLE]], ptr %i
+  store ptr @tls_variable, ptr %i
+  %sus_result = call i8 @llvm.coro.suspend(token none, i1 false)
+  switch i8 %sus_result, label %suspend [i8 0, label %resume
+                                         i8 1, label %cleanup]
+resume:
+  ; CHECK: %[[TLS_VARIABLE2:.*]] = call ptr @llvm.coro.maychange(ptr @tls_variable)
+  ; CHECK-NEXT: store ptr %[[TLS_VARIABLE2]], ptr %j
+  store ptr @tls_variable, ptr %j
+  call void @consume(ptr %i)
+  call void @consume(ptr %j)
+  br label %cleanup
+
+cleanup:
+  %mem = call i8* @llvm.coro.free(token %id, i8* %hdl)
+  call void @free(i8* %mem)
+  br label %suspend
+
+suspend:
+  call i1 @llvm.coro.end(i8* %hdl, i1 0)
+  ret i8* %hdl
+}
+
+declare void @consume(ptr)
+declare i8* @llvm.coro.free(token, i8*)
+declare i32 @llvm.coro.size.i32()
+declare i8  @llvm.coro.suspend(token, i1)
+
+declare token @llvm.coro.id(i32, i8*, i8*, i8*)
+declare i1 @llvm.coro.alloc(token)
+declare i8* @llvm.coro.begin(token, i8*)
+declare i1 @llvm.coro.end(i8*, i1)
+
+declare noalias i8* @malloc(i32)
+declare void @free(i8*)