[coroutines] Add codegen for await and yield expressions

[GorNishanov]

Summary

Emit suspend expression which roughly looks like:

auto && x = CommonExpr();
if (!x.await_ready()) {
   llvm_coro_save();
   x.await_suspend(...);     (*)
   llvm_coro_suspend(); (**)
}
x.await_resume();

where the result of the entire expression is the result of x.await_resume()

(*) If x.await_suspend return type is bool, it allows to veto a suspend:

if (x.await_suspend(...)) 
   llvm_coro_suspend();

(**) llvm_coro_suspend() encodes three possible continuations as a switch instruction:

%where-to = call i8 @llvm.coro.suspend(...)
switch i8 %where-to, label %coro.ret [ ; jump to epilogue to suspend
  i8 0, label %yield.ready   ; go here when resumed
  i8 1, label %yield.cleanup ; go here when destroyed
]

[GorNishanov]

+ @majnemer who is most familiar with LLVM Coroutines representation.

[majnemer]

lib/CodeGen/CGCoroutine.cpp

26

I'd move this into buildSuspendSuffixStr.

32

Shouldn't this struct be in an anonymous namespace?

86–88

I'd just make this fully covered, it's just two more cases.

lib/CodeGen/CGExprScalar.cpp

279–282

Pointers should lean right.

285

Formatting looks off.

[GorNishanov]

Addressed review feedback

• clang-format three functions in CGExprScalar.cpp
• got rid of AwaitKindStr array and select appropriate string in the switch statement in buildAwaitKindPrefix

[GorNishanov]

lib/CodeGen/CGCoroutine.cpp

32

It is forward declared in CodeGenFunction.h and CodeGenFunction class holds a unique_ptr to CGCoroData as a member.
Here is the snippet from CodeGenFunction.h:

// Holds coroutine data if the current function is a coroutine. We use a
// wrapper to manage its lifetime, so that we don't have to define CGCoroData
// in this header.
struct CGCoroInfo {
  std::unique_ptr<CGCoroData> Data;
  CGCoroInfo();
  ~CGCoroInfo();
};
CGCoroInfo CurCoro;

[majnemer]

lib/CodeGen/CGCoroutine.cpp

86

I'd use a StringRef here.

[GorNishanov]

Implemented review feedback: const char * => StringRef in buildSuspendSuffixStr

[GorNishanov]

lib/CodeGen/CGCoroutine.cpp

86

StringRef it is.

[majnemer]

lib/CodeGen/CGCoroutine.cpp

86

I'd just let the default constructor do its thing.

[rjmccall]

lib/CodeGen/CGCoroutine.cpp

32

If it's already forward-declared, you should check here that you're matching it by doing:

struct CodeGen::CGCoroData {
  ...
};

instead of making namespace declarations.

43

Comments about what these mean would be useful.

86

Agreed. Assigning 0 to a StringRef reads very wrong.

204

This function should return an RValue and take an AggValueSlot, just like every other generically-typed expression emitter. That way, you can just use EmitAnyExpr here instead of inlining two of three cases.

And you should implement it in CGExprComplex, of course, and add that as a test case.

[GorNishanov]

• reworked EmitAwait/Yield to has the signature similar to EmitAnyExpr
• await expresions now support _Complex types.
• s/Suffix/Prefix in buildCoroutineSuffix, since it was actually building a prefix
• buildCoroutineSuffix uses literal array for names, as opposed to getting names from the switch stmt.
• added tests to test for Aggr, Scalar and Complex
• added tests for codegen for operator co_await
• sprinkled more comments

Thank you very much for your feedback, John and David!

[GorNishanov]

lib/CodeGen/CGCoroutine.cpp

26

I prefer to keep AwaitKindStr here, so it is trivial to visually inspect that the order in the array matches the order of the enum.

86

I got rid of this one and now use a llvm::StringLiteral array to get names.

204

Thank you! That was very helpful. The code shrunk and looks nicer now. Not sure, if ignoreResult is needed, but, I added it as well to match EmitAnyExpr.

[GorNishanov]

Looks good now?

[rjmccall]

Looks great, thanks.

[GorNishanov]

Thank you very much for the review!

Merged with the top of the trunk and preparing to land

[GorNishanov]

Commit: r298784

lib/CodeGen/CGCoroutine.cpp

 //===----- CGCoroutine.cpp - Emit LLVM Code for C++ coroutines ------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This contains code dealing with C++ code generation of coroutines.
 //
 //===----------------------------------------------------------------------===//
 
 #include "CodeGenFunction.h"
+#include "llvm/ADT/ScopeExit.h"
 #include "clang/AST/StmtCXX.h"
 
 using namespace clang;
 using namespace CodeGen;
 
+using llvm::Value;
+using llvm::BasicBlock;
+
+namespace {
+enum class AwaitKind { Init, Normal, Yield, Final };
+char const *AwaitKindStr[] = {"init", "await", "yield", "final"};
+}
+
 namespace clang {
 namespace CodeGen {
 
 struct CGCoroData {
+  AwaitKind CurrentAwaitKind = AwaitKind::Init;
+  llvm::BasicBlock *SuspendBB = nullptr;
 
+  CodeGenFunction::JumpDest CleanupJD;
   // Stores the jump destination just before the final suspend. Coreturn
   // statements jumps to this point after calling return_xxx promise member.
   CodeGenFunction::JumpDest FinalJD;
 
+  unsigned AwaitNum = 0;
+  unsigned YieldNum = 0;
   unsigned CoreturnCount = 0;
 
   // Stores the llvm.coro.id emitted in the function so that we can supply it
   // as the first argument to coro.begin, coro.alloc and coro.free intrinsics.
   // Note: llvm.coro.id returns a token that cannot be directly expressed in a
   // builtin.
   llvm::CallInst *CoroId = nullptr;
   // If coro.id came from the builtin, remember the expression to give better
   // diagnostic. If CoroIdExpr is nullptr, the coro.id was created by
if (CurCoro.Data) {
     return;
   }
 
   CurCoro.Data = std::unique_ptr<CGCoroData>(new CGCoroData);
   CurCoro.Data->CoroId = CoroId;
   CurCoro.Data->CoroIdExpr = CoroIdExpr;
 }
 
+// Synthesize a pretty name for a suspend point.
+static SmallString<32> buildSuspendSuffixStr(CGCoroData &Coro, AwaitKind Kind) {
+  unsigned No = 0;
+  switch (Kind) {
+  default:
+    break;
+  case AwaitKind::Normal:
+    No = ++Coro.AwaitNum;
+    break;
+  case AwaitKind::Yield:
+    No = ++Coro.YieldNum;
+    break;
+  }
+  SmallString<32> Suffix(AwaitKindStr[static_cast<int>(Kind)]);
+  if (No > 1) {
+    Twine(No).toVector(Suffix);
+  }
+  return Suffix;
+}
+
+// Emit suspend expression which roughly looks like:
+//
+//   auto && x = CommonExpr();
+//   if (!x.await_ready()) {
+//      llvm_coro_save();
+//      x.await_suspend(...);     (*)
+//      llvm_coro_suspend(); (**)
+//   }
+//   x.await_resume();
+//
+// where the result of the entire expression is the result of x.await_resume()
+//
+//   (*) If x.await_suspend return type is bool, it allows to veto a suspend:
+//      if (x.await_suspend(...))
+//        llvm_coro_suspend();
+//
+//  (**) llvm_coro_suspend() encodes three possible continuation passes as
+//       a switch instruction:
+//
+//  %where-to = call i8 @llvm.coro.suspend(...)
+//  switch i8 %where-to, label %coro.ret [ ; jump to epilogue to suspend
+//    i8 0, label %yield.ready   ; go here when resumed
+//    i8 1, label %yield.cleanup ; go here when destroyed
+//  ]
+//
+//  See llvm's docs/Coroutines.rst for more details.
+//
+static Value *emitSuspendExpression(CodeGenFunction &CGF, CGCoroData &Coro,
+                                    CoroutineSuspendExpr const &S,
+                                    AwaitKind Kind,
+                                    ReturnValueSlot ReturnValue) {
+  auto &Builder = CGF.Builder;
+  const bool IsFinalSuspend = Kind == AwaitKind::Final;
+  auto Suffix = buildSuspendSuffixStr(Coro, Kind);
+
+  auto *E = S.getCommonExpr();
+  if (auto *UO = dyn_cast<UnaryOperator>(E))
+    if (UO->getOpcode() == UO_Coawait)
+      E = UO->getSubExpr();
+
+  auto Binder =
+      CodeGenFunction::OpaqueValueMappingData::bind(CGF, S.getOpaqueValue(), E);
+  auto UnbindOnExit = llvm::make_scope_exit([&] { Binder.unbind(CGF); });
+
+  BasicBlock *ReadyBlock = CGF.createBasicBlock(Suffix + Twine(".ready"));
+  BasicBlock *SuspendBlock = CGF.createBasicBlock(Suffix + Twine(".suspend"));
+  BasicBlock *CleanupBlock = CGF.createBasicBlock(Suffix + Twine(".cleanup"));
+
+  CGF.EmitBranchOnBoolExpr(S.getReadyExpr(), ReadyBlock, SuspendBlock, 0);
+  CGF.EmitBlock(SuspendBlock);
+
+  llvm::Function *CoroSave = CGF.CGM.getIntrinsic(llvm::Intrinsic::coro_save);
+  auto *NullPtr = llvm::ConstantPointerNull::get(CGF.CGM.Int8PtrTy);
+  auto *SaveCall = Builder.CreateCall(CoroSave, {NullPtr});
+
+  auto *SuspendRet = CGF.EmitScalarExpr(S.getSuspendExpr());
+  if (SuspendRet != nullptr) {
+    assert(SuspendRet->getType()->isIntegerTy(1) &&
+      "Sema should have already checked that it is void or bool");
+    BasicBlock *RealSuspendBlock =
+        CGF.createBasicBlock(Suffix + Twine(".suspend.bool"));
+    CGF.Builder.CreateCondBr(SuspendRet, RealSuspendBlock, ReadyBlock);
+    SuspendBlock = RealSuspendBlock;
+    CGF.EmitBlock(RealSuspendBlock);
+  }
+
+  llvm::Function *CoroSuspend =
+      CGF.CGM.getIntrinsic(llvm::Intrinsic::coro_suspend);
+  auto *SuspendResult = Builder.CreateCall(
+      CoroSuspend, {SaveCall, Builder.getInt1(IsFinalSuspend)});
+  auto *Switch = Builder.CreateSwitch(SuspendResult, Coro.SuspendBB, 2);
+  Switch->addCase(Builder.getInt8(0), ReadyBlock);
+  Switch->addCase(Builder.getInt8(1), CleanupBlock);
+
+  CGF.EmitBlock(CleanupBlock);
+
+  CGF.EmitBranchThroughCleanup(Coro.CleanupJD);
+
+  // Emit await_resume expression.
+  CGF.EmitBlock(ReadyBlock);
+  QualType Type = S.getResumeExpr()->getType();
+  switch (CGF.getEvaluationKind(Type)) {
+  case TEK_Scalar:
+    return CGF.EmitScalarExpr(S.getResumeExpr());
+  case TEK_Aggregate:
+    CGF.EmitAggExpr(S.getResumeExpr(),
+                    AggValueSlot::forAddr(ReturnValue.getValue(), Qualifiers(),
+                                          AggValueSlot::IsDestructed,
+                                          AggValueSlot::DoesNotNeedGCBarriers,
+                                          AggValueSlot::IsNotAliased));
+    break;
+  case TEK_Complex:
+    CGF.CGM.ErrorUnsupported(S.getResumeExpr(), "_Complex await expression");
+    break;
+  }
+  return nullptr;
+}
+
+llvm::Value *CodeGenFunction::EmitCoawaitExpr(const CoawaitExpr &E,
+                                              ReturnValueSlot ReturnValue) {
+  return emitSuspendExpression(*this, *CurCoro.Data, E,
+                               CurCoro.Data->CurrentAwaitKind, ReturnValue);
+}
+llvm::Value *CodeGenFunction::EmitCoyieldExpr(const CoyieldExpr &E,
+                                              ReturnValueSlot ReturnValue) {
+  return emitSuspendExpression(*this, *CurCoro.Data, E, AwaitKind::Yield,
+                               ReturnValue);
+}
+
 void CodeGenFunction::EmitCoreturnStmt(CoreturnStmt const &S) {
   ++CurCoro.Data->CoreturnCount;
   EmitStmt(S.getPromiseCall());
   EmitBranchThroughCleanup(CurCoro.Data->FinalJD);
 }
 
 void CodeGenFunction::EmitCoroutineBody(const CoroutineBodyStmt &S) {
   auto *NullPtr = llvm::ConstantPointerNull::get(Builder.getInt8PtrTy());
   auto &TI = CGM.getContext().getTargetInfo();
   unsigned NewAlign = TI.getNewAlign() / TI.getCharWidth();
 
   auto *FinalBB = createBasicBlock("coro.final");
+  auto *RetBB = createBasicBlock("coro.ret");
 
   auto *CoroId = Builder.CreateCall(
       CGM.getIntrinsic(llvm::Intrinsic::coro_id),
       {Builder.getInt32(NewAlign), NullPtr, NullPtr, NullPtr});
   createCoroData(*this, CurCoro, CoroId);
+  CurCoro.Data->SuspendBB = RetBB;
 
   EmitScalarExpr(S.getAllocate());
 
   // FIXME: Setup cleanup scopes.
 
   EmitStmt(S.getPromiseDeclStmt());
 
+  CurCoro.Data->CleanupJD = getJumpDestInCurrentScope(RetBB);
   CurCoro.Data->FinalJD = getJumpDestInCurrentScope(FinalBB);
 
   // FIXME: Emit initial suspend and more before the body.
 
   EmitStmt(S.getBody());
 
   // See if we need to generate final suspend.
   const bool CanFallthrough = Builder.GetInsertBlock();
   const bool HasCoreturns = CurCoro.Data->CoreturnCount > 0;
   if (CanFallthrough || HasCoreturns) {
     EmitBlock(FinalBB);
     // FIXME: Emit final suspend.
   }
   EmitStmt(S.getDeallocate());
 
+  EmitBlock(RetBB);
+
   // FIXME: Emit return for the coroutine return object.
 }
 
 // Emit coroutine intrinsic and patch up arguments of the token type.
 RValue CodeGenFunction::EmitCoroutineIntrinsic(const CallExpr *E,
                                                unsigned int IID) {
   SmallVector<llvm::Value *, 8> Args;
   switch (IID) {

lib/CodeGen/CGExprAgg.cpp

public:
 
   void VisitStmt(Stmt *S) {
     CGF.ErrorUnsupported(S, "aggregate expression");
   }
   void VisitParenExpr(ParenExpr *PE) { Visit(PE->getSubExpr()); }
   void VisitGenericSelectionExpr(GenericSelectionExpr *GE) {
     Visit(GE->getResultExpr());
   }
+  void VisitCoawaitExpr(CoawaitExpr *E) {
+    CGF.EmitCoawaitExpr(*E, getReturnValueSlot());
+  }
+  void VisitCoyieldExpr(CoyieldExpr *E) {
+    CGF.EmitCoyieldExpr(*E, getReturnValueSlot());
+  }
+  void VisitUnaryCoawait(UnaryOperator *E) { Visit(E->getSubExpr()); }
   void VisitUnaryExtension(UnaryOperator *E) { Visit(E->getSubExpr()); }
   void VisitSubstNonTypeTemplateParmExpr(SubstNonTypeTemplateParmExpr *E) {
     return Visit(E->getReplacement());
   }
 
   // l-values.
   void VisitDeclRefExpr(DeclRefExpr *E) {
     // For aggregates, we should always be able to emit the variable

lib/CodeGen/CGExprScalar.cpp

Value *VisitParenExpr(ParenExpr *PE) {
     return Visit(PE->getSubExpr());
   }
   Value *VisitSubstNonTypeTemplateParmExpr(SubstNonTypeTemplateParmExpr *E) {
     return Visit(E->getReplacement());
   }
   Value *VisitGenericSelectionExpr(GenericSelectionExpr *GE) {
     return Visit(GE->getResultExpr());
   }
+  Value *VisitCoawaitExpr(CoawaitExpr* S) {
+    return CGF.EmitCoawaitExpr(*S);
+  }
+  Value *VisitCoyieldExpr(CoyieldExpr* S) {
+    return CGF.EmitCoyieldExpr(*S);
+  }
+  Value *VisitUnaryCoawait  (const UnaryOperator *E) {
+    return Visit(E->getSubExpr());
+  }
 
   // Leaves.
   Value *VisitIntegerLiteral(const IntegerLiteral *E) {
     return Builder.getInt(E->getValue());
   }
   Value *VisitFloatingLiteral(const FloatingLiteral *E) {
     return llvm::ConstantFP::get(VMContext, E->getValue());
   }
Value *ScalarExprEmitter::VisitCastExpr(CastExpr *CE) {
   }
 
   case CK_ZeroToOCLQueue: {
     assert(DestTy->isQueueT() && "CK_ZeroToOCLQueue cast on non queue_t type");
     return llvm::Constant::getNullValue(ConvertType(DestTy));
   }
 
   case CK_IntToOCLSampler:
-    return CGF.CGM.createOpenCLIntToSamplerConversion(E, CGF);
+    return CGF.CGM.createOpenCLIntToSamplerConversion(E, CGF);
 
   } // end of switch
 
   llvm_unreachable("unknown scalar cast");
 }
 
 Value *ScalarExprEmitter::VisitStmtExpr(const StmtExpr *E) {
   CodeGenFunction::StmtExprEvaluation eval(CGF);

lib/CodeGen/CodeGenFunction.h

#endif
   void EmitObjCForCollectionStmt(const ObjCForCollectionStmt &S);
   void EmitObjCAtTryStmt(const ObjCAtTryStmt &S);
   void EmitObjCAtThrowStmt(const ObjCAtThrowStmt &S);
   void EmitObjCAtSynchronizedStmt(const ObjCAtSynchronizedStmt &S);
   void EmitObjCAutoreleasePoolStmt(const ObjCAutoreleasePoolStmt &S);
 
   void EmitCoroutineBody(const CoroutineBodyStmt &S);
   void EmitCoreturnStmt(const CoreturnStmt &S);
+  llvm::Value *EmitCoawaitExpr(const CoawaitExpr &E,
+                               ReturnValueSlot ReturnValue = ReturnValueSlot());
+  llvm::Value *EmitCoyieldExpr(const CoyieldExpr &E,
+                               ReturnValueSlot ReturnValue = ReturnValueSlot());
   RValue EmitCoroutineIntrinsic(const CallExpr *E, unsigned int IID);
 
   void EnterCXXTryStmt(const CXXTryStmt &S, bool IsFnTryBlock = false);
   void ExitCXXTryStmt(const CXXTryStmt &S, bool IsFnTryBlock = false);
 
   void EmitCXXTryStmt(const CXXTryStmt &S);
   void EmitSEHTryStmt(const SEHTryStmt &S);
   void EmitSEHLeaveStmt(const SEHLeaveStmt &S);

test/CodeGenCoroutines/coro-await.cpp

+// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu -fcoroutines-ts -std=c++14 -emit-llvm %s -o - -disable-llvm-passes | FileCheck %s
+
+namespace std {
+namespace experimental {
+template <typename... T>
+struct coroutine_traits;
+
+template <typename Promise = void> struct coroutine_handle;
+
+template <>
+struct coroutine_handle<void> {
+  void *ptr;
+  static coroutine_handle from_address(void *);
+};
+
+template <typename Promise>
+struct coroutine_handle : coroutine_handle<> {
+  static coroutine_handle from_address(void *);
+};
+
+}
+}
+
+struct suspend_always {
+  int stuff;
+  bool await_ready();
+  void await_suspend(std::experimental::coroutine_handle<>);
+  void await_resume();
+};
+
+template<>
+struct std::experimental::coroutine_traits<void> {
+  struct promise_type {
+    void get_return_object();
+    suspend_always initial_suspend();
+    suspend_always final_suspend();
+    void return_void();
+  };
+};
+
+// CHECK-LABEL: f0(
+extern "C" void f0() {
+
+  co_await suspend_always{};
+  // See if we need to suspend:
+  // --------------------------
+  // CHECK: %[[READY:.+]] = call zeroext i1 @_ZN14suspend_always11await_readyEv(%struct.suspend_always* %[[AWAITABLE:.+]])
+  // CHECK: br i1 %[[READY]], label %[[READY_BB:.+]], label %[[SUSPEND_BB:.+]]
+
+  // If we are suspending:
+  // ---------------------
+  // CHECK: [[SUSPEND_BB]]:
+  // CHECK: %[[SUSPEND_ID:.+]] = call token @llvm.coro.save(
+  // ---------------------------
+  // Build the coroutine handle and pass it to await_suspend
+  // ---------------------------
+  // CHECK: %[[FRAME:.+]] = call i8* @llvm.coro.frame()
+  // CHECK: call i8* @_ZNSt12experimental16coroutine_handleINS_16coroutine_traitsIJvEE12promise_typeEE12from_addressEPv(i8* %[[FRAME]])
+  //   ... many lines of code to coerce coroutine_handle into an i8* scalar
+  // CHECK: %[[CH:.+]] = load i8*, i8** %{{.+}}
+  // CHECK: call void @_ZN14suspend_always13await_suspendENSt12experimental16coroutine_handleIvEE(%struct.suspend_always* %[[AWAITABLE]], i8* %[[CH]])
+  // -------------------------
+  // Generate a suspend point:
+  // -------------------------
+  // CHECK: %[[OUTCOME:.+]] = call i8 @llvm.coro.suspend(token %[[SUSPEND_ID]], i1 false)
+  // CHECK: switch i8 %[[OUTCOME]], label %[[RET_BB:.+]] [
+  // CHECK:   i8 0, label %[[READY_BB]]
+  // CHECK:   i8 1, label %[[CLEANUP_BB:.+]]
+  // CHECK: ]
+
+  // Cleanup code goes here:
+  // -----------------------
+  // CHECK: [[CLEANUP_BB]]:
+  
+  // When coroutine is resumed, call await_resume
+  // --------------------------
+  // CHECK: [[READY_BB]]:
+  // CHECK:  call void @_ZN14suspend_always12await_resumeEv(%struct.suspend_always* %[[AWAITABLE]])
+}
+
+struct suspend_maybe {
+  float stuff;
+  ~suspend_maybe();
+  bool await_ready();
+  bool await_suspend(std::experimental::coroutine_handle<>);
+  void await_resume();
+};
+
+
+template<>
+struct std::experimental::coroutine_traits<void,int> {
+  struct promise_type {
+    void get_return_object();
+    suspend_always initial_suspend();
+    suspend_always final_suspend();
+    void return_void();
+    suspend_maybe yield_value(int);
+  };
+};
+
+// CHECK-LABEL: f1(
+extern "C" void f1(int) {
+  co_yield 42;
+  // CHECK: %[[PROMISE:.+]] = alloca %"struct.std::experimental::coroutine_traits<void, int>::promise_type"
+  // CHECK: call void @_ZNSt12experimental16coroutine_traitsIJviEE12promise_type11yield_valueEi(%struct.suspend_maybe* sret %[[AWAITER:.+]], %"struct.std::experimental::coroutine_traits<void, int>::promise_type"* %[[PROMISE]], i32 42)
+
+  // See if we need to suspend:
+  // --------------------------
+  // CHECK: %[[READY:.+]] = call zeroext i1 @_ZN13suspend_maybe11await_readyEv(%struct.suspend_maybe* %[[AWAITABLE]])
+  // CHECK: br i1 %[[READY]], label %[[READY_BB:.+]], label %[[SUSPEND_BB:.+]]
+
+  // If we are suspending:
+  // ---------------------
+  // CHECK: [[SUSPEND_BB]]:
+  // CHECK: %[[SUSPEND_ID:.+]] = call token @llvm.coro.save(
+  // ---------------------------
+  // Build the coroutine handle and pass it to await_suspend
+  // ---------------------------
+  // CHECK: %[[FRAME:.+]] = call i8* @llvm.coro.frame()
+  // CHECK: call i8* @_ZNSt12experimental16coroutine_handleINS_16coroutine_traitsIJviEE12promise_typeEE12from_addressEPv(i8* %[[FRAME]])
+  //   ... many lines of code to coerce coroutine_handle into an i8* scalar
+  // CHECK: %[[CH:.+]] = load i8*, i8** %{{.+}}
+  // CHECK: %[[YES:.+]] = call zeroext i1 @_ZN13suspend_maybe13await_suspendENSt12experimental16coroutine_handleIvEE(%struct.suspend_maybe* %[[AWAITABLE]], i8* %[[CH]])
+  // -------------------------------------------
+  // See if await_suspend decided not to suspend
+  // -------------------------------------------
+  // CHECK: br i1 %[[YES]], label %[[SUSPEND_PLEASE:.+]], label %[[READY_BB]]
+
+  // CHECK: [[SUSPEND_PLEASE]]:
+  // CHECK:    call i8 @llvm.coro.suspend(token %[[SUSPEND_ID]], i1 false)
+
+  // CHECK: [[READY_BB]]:
+  // CHECK:     call void @_ZN13suspend_maybe12await_resumeEv(%struct.suspend_maybe* %[[AWAITABLE]])
+}