[CodeGen][ObjC] Make block copy/dispose helper function exception-safe.

[ahatanak]

Summary

Block copy/dispose helper functions currently aren’t exception-safe. When an exception is thrown in a copy function, the function exits without destroying or releasing C++ objects, ObjC objects, and __block objects that have already been copied. Similarly, dispose functions don't release ObjC objects and __block objects that are yet to be released when an exception is thrown.

[ahatanak]

Note that in order to destruct C++ objects, I'm using pushDestroy which pushes a NormalCleanup when exception handling isn't enabled. This is different from PushDestructorCleanup which always pushes a NormalAndEHCleanup, but I think using pushDestroy is more correct since I don't think we need to do a cleanup on the EH path when exception isn't enabled.

[rjmccall]

In D49718#1173038, @ahatanak wrote:

Note that in order to destruct C++ objects, I'm using pushDestroy which pushes a NormalCleanup when exception handling isn't enabled. This is different from PushDestructorCleanup which always pushes a NormalAndEHCleanup, but I think using pushDestroy is more correct since I don't think we need to do a cleanup on the EH path when exception isn't enabled.

I don't think it makes any difference because we shouldn't be emitting cleanup paths when exceptions are disabled. I don't think there's an intended difference in semantics between those two destructor-cleanup paths, at least.

lib/CodeGen/CGBlocks.cpp

1603

There's already a CallBlockRelease cleanup that can presumably be unified with this.

1918

These two switches differ only in (1) what kind of cleanup they push and (2) a small optimization that can easily be conditionalized by the request to push an EH-only cleanup.

[ahatanak]

Address review comments.

I think I should be able to merge pushBlockObjectRelease and enterByrefCleanup, but the BlockFieldFlags that are passed are different. We set BLOCK_FIELD_IS_WEAK in addition to BLOCK_FIELD_IS_BYREF when isObjCGCWeak() returns true and we are emitting the copy/dispose helper functions, but when enterByrefCleanup is called from EmitAutoVarCleanups, only BLOCK_FIELD_IS_BYREF is set.

In D49718#1173068, @rjmccall wrote:

I don't think it makes any difference because we shouldn't be emitting cleanup paths when exceptions are disabled. I don't think there's an intended difference in semantics between those two destructor-cleanup paths, at least.

OK, I see.

If it doesn't make any difference, I think I can push NormalAndEHCleanup unconditionally when copying a BlockObject capture instead of pushing NormalCleanup when EH is disabled. I made that change in the updated patch.

lib/CodeGen/CGBlocks.cpp

1918

Merged the two switch statements into helper function pushCaptureCleanup.

[rjmccall]

In D49718#1173883, @ahatanak wrote:

Address review comments.

I think I should be able to merge pushBlockObjectRelease and enterByrefCleanup, but the BlockFieldFlags that are passed are different. We set BLOCK_FIELD_IS_WEAK in addition to BLOCK_FIELD_IS_BYREF when isObjCGCWeak() returns true and we are emitting the copy/dispose helper functions, but when enterByrefCleanup is called from EmitAutoVarCleanups, only BLOCK_FIELD_IS_BYREF is set.

Heh, okay. It looks like the runtime doesn't do anything different, but I think it's probably more correct to always pass BLOCK_FIELD_IS_WEAK when destroying a __weak block in GC modes.

In D49718#1173068, @rjmccall wrote:

I don't think it makes any difference because we shouldn't be emitting cleanup paths when exceptions are disabled. I don't think there's an intended difference in semantics between those two destructor-cleanup paths, at least.

OK, I see.

If it doesn't make any difference, I think I can push NormalAndEHCleanup unconditionally when copying a BlockObject capture instead of pushing NormalCleanup when EH is disabled. I made that change in the updated patch.

Ok. Thanks, I like how it looks now.

[ahatanak]

Merge pushBlockObjectRelease and enterByrefCleanup.

In D49718#1174113, @rjmccall wrote:

Heh, okay. It looks like the runtime doesn't do anything different, but I think it's probably more correct to always pass BLOCK_FIELD_IS_WEAK when destroying a __weak block in GC modes.

I tried passing BLOCK_FIELD_IS_WEAK | BLOCK_FIELD_IS_BYREF to the call to enterByrefCleanup in EmitAutoVarCleanups, but it looks like test/CodeGenObjC/blocks.m fails if I do so. The test was committed in r125823 and there is a comment that says " We're not supposed to pass BLOCK_FIELD_IS_WEAK here".

http://lists.llvm.org/pipermail/cfe-commits/Week-of-Mon-20110214/038996.html

Should we change the check in the test?

[rjmccall]

In D49718#1174281, @ahatanak wrote:

Merge pushBlockObjectRelease and enterByrefCleanup.

In D49718#1174113, @rjmccall wrote:

Heh, okay. It looks like the runtime doesn't do anything different, but I think it's probably more correct to always pass BLOCK_FIELD_IS_WEAK when destroying a __weak block in GC modes.

I tried passing BLOCK_FIELD_IS_WEAK | BLOCK_FIELD_IS_BYREF to the call to enterByrefCleanup in EmitAutoVarCleanups, but it looks like test/CodeGenObjC/blocks.m fails if I do so. The test was committed in r125823 and there is a comment that says " We're not supposed to pass BLOCK_FIELD_IS_WEAK here".

http://lists.llvm.org/pipermail/cfe-commits/Week-of-Mon-20110214/038996.html

Should we change the check in the test?

Thanks for the archaeology; that's a very good question. It looks to me like that comment was unrelated to the functionality change made by that patch, and nothing in the radar history explains it, either. I don't see anything in the runtime source which suggests that it's wrong to pass BYREF|WEAK to Block_object_dispose. Going way back, it does seem to have been necessary under GC to pass BYREF|WEAK for Block_object_assign, but the runtime never did anything special with the flag for Block_object_dispose; it just always treated it exactly the same as BYREF alone.

I think it's fine to just change the test.

[ahatanak]

Set the BLOCK_FIELD_IS_WEAK bit of the flag passed to the call to enterByrefCleanup in EmitAutoVarCleanups and fix test case test/CodeGenObjC/blocks.m.

[rjmccall]

lib/CodeGen/CGBlocks.cpp

2578–2579

Please document the meaning of the parameters here.

[ahatanak]

Document the meaning of enterByrefCleanup's parameters.

[rjmccall]

Thanks, LGTM.

[ahatanak]

lib/CodeGen/CGBlocks.cpp

2578–2579

One more question: I think documentation comments for public APIs are normally in the header file. Should we move these comments to CodeGenFunction.h?

[rjmccall]

lib/CodeGen/CGBlocks.cpp

2578–2579

It'd be better, yeah. Having it be on a unique implementation is also okay because it's not hard to find, but yeah, header is better.

lib/CodeGen/CGBlocks.cpp

if (Capture.isConstant())
       continue;
 
     auto Info = Predicate(CI, Variable->getType(), LangOpts);
     if (Info.first != BlockCaptureEntityKind::None)
       ManagedCaptures.emplace_back(Info.first, Info.second, CI, Capture);
   }
 }
 
+namespace {
+/// Release a __block variable.
+struct CallBlockRelease final : EHScopeStack::Cleanup {
+  Address Addr;
+  BlockFieldFlags FieldFlags;
+  bool LoadBlockVarAddr;
+
+  CallBlockRelease(Address Addr, BlockFieldFlags Flags, bool LoadValue)
+      : Addr(Addr), FieldFlags(Flags), LoadBlockVarAddr(LoadValue) {}
+
+  void Emit(CodeGenFunction &CGF, Flags flags) override {
+    llvm::Value *BlockVarAddr;
+    if (LoadBlockVarAddr) {
+      BlockVarAddr = CGF.Builder.CreateLoad(Addr);
+      BlockVarAddr = CGF.Builder.CreateBitCast(BlockVarAddr, CGF.VoidPtrTy);
+    } else {
+      BlockVarAddr = Addr.getPointer();
+    }
+
+    CGF.BuildBlockRelease(BlockVarAddr, FieldFlags);
+  }
+};
+} // end anonymous namespace
+
+static void pushCaptureCleanup(BlockCaptureEntityKind CaptureKind,
+                               Address Field, QualType CaptureType,
+                               BlockFieldFlags Flags, bool EHOnly,
+                               CodeGenFunction &CGF) {
+  switch (CaptureKind) {
+  case BlockCaptureEntityKind::CXXRecord:
+  case BlockCaptureEntityKind::ARCWeak:
+  case BlockCaptureEntityKind::NonTrivialCStruct:
+  case BlockCaptureEntityKind::ARCStrong: {
+    if (CaptureType.isDestructedType() &&
+        (!EHOnly || CGF.needsEHCleanup(CaptureType.isDestructedType()))) {
+      CodeGenFunction::Destroyer *Destroyer =
+          CaptureKind == BlockCaptureEntityKind::ARCStrong
+              ? CodeGenFunction::destroyARCStrongImprecise
+              : CGF.getDestroyer(CaptureType.isDestructedType());
+      CleanupKind Kind =
+          EHOnly ? EHCleanup
+                 : CGF.getCleanupKind(CaptureType.isDestructedType());
+      CGF.pushDestroy(Kind, Field, CaptureType, Destroyer, Kind & EHCleanup);
+    }
+    break;
+  }
+  case BlockCaptureEntityKind::BlockObject: {
+    if (!EHOnly || CGF.getLangOpts().Exceptions) {
+      CleanupKind Kind = EHOnly ? EHCleanup : NormalAndEHCleanup;
+      CGF.enterByrefCleanup(Kind, Field, Flags, /*LoadBlockVarAddr*/ true);
+    }
+    break;
+  }
+  case BlockCaptureEntityKind::None:
+    llvm_unreachable("unexpected BlockCaptureEntityKind");
+  }
+}
+
 /// Generate the copy-helper function for a block closure object:
 ///   static void block_copy_helper(block_t *dst, block_t *src);
 /// The runtime will have previously initialized 'dst' by doing a
 /// bit-copy of 'src'.
 ///
 /// Note that this copies an entire block closure object to the heap;
 /// it should not be confused with a 'byref copy helper', which moves
 /// the contents of an individual __block variable to the heap.
CodeGenFunction::GenerateCopyHelperFunction(const CGBlockInfo &blockInfo) {
 
   SmallVector<BlockCaptureManagedEntity, 4> CopiedCaptures;
   findBlockCapturedManagedEntities(blockInfo, getLangOpts(), CopiedCaptures,
                                    computeCopyInfoForBlockCapture);
 
   for (const auto &CopiedCapture : CopiedCaptures) {
     const BlockDecl::Capture &CI = CopiedCapture.CI;
     const CGBlockInfo::Capture &capture = CopiedCapture.Capture;
+    QualType captureType = CI.getVariable()->getType();
     BlockFieldFlags flags = CopiedCapture.Flags;
 
     unsigned index = capture.getIndex();
     Address srcField = Builder.CreateStructGEP(src, index, capture.getOffset());
     Address dstField = Builder.CreateStructGEP(dst, index, capture.getOffset());
 
     // If there's an explicit copy expression, we do that.
     if (CI.getCopyExpr()) {
// call to its copy constructor.
           EmitARCStoreStrongCall(dstField, srcValue, true);
 
         // With optimization enabled, take advantage of the fact that
         // the blocks runtime guarantees a memcpy of the block data, and
         // just emit a retain of the src field.
         } else {
           EmitARCRetainNonBlock(srcValue);
 
-          // We don't need this anymore, so kill it.  It's not quite
-          // worth the annoyance to avoid creating it in the first place.
+          // Unless EH cleanup is required, we don't need this anymore, so kill
+          // it. It's not quite worth the annoyance to avoid creating it in the
+          // first place.
+          if (!needsEHCleanup(captureType.isDestructedType()))
             cast<llvm::Instruction>(dstField.getPointer())->eraseFromParent();
         }
       } else {
         assert(CopiedCapture.Kind == BlockCaptureEntityKind::BlockObject);
         srcValue = Builder.CreateBitCast(srcValue, VoidPtrTy);
         llvm::Value *dstAddr =
           Builder.CreateBitCast(dstField.getPointer(), VoidPtrTy);
         llvm::Value *args[] = {
           dstAddr, srcValue, llvm::ConstantInt::get(Int32Ty, flags.getBitMask())
// call to its copy constructor.
 
         if (copyCanThrow) {
           EmitRuntimeCallOrInvoke(CGM.getBlockObjectAssign(), args);
         } else {
           EmitNounwindRuntimeCall(CGM.getBlockObjectAssign(), args);
         }
       }
     }
+
+    // Ensure that we destroy the copied object if an exception is thrown later
+    // in the helper function.
+    pushCaptureCleanup(CopiedCapture.Kind, dstField, captureType, flags, /*EHOnly*/ true,
+                       *this);
   }
 
   FinishFunction();
 
   return llvm::ConstantExpr::getBitCast(Fn, VoidPtrTy);
 }
 
 static BlockFieldFlags
CodeGenFunction::GenerateDestroyHelperFunction(const CGBlockInfo &blockInfo) {
   for (const auto &DestroyedCapture : DestroyedCaptures) {
     const BlockDecl::Capture &CI = DestroyedCapture.CI;
     const CGBlockInfo::Capture &capture = DestroyedCapture.Capture;
     BlockFieldFlags flags = DestroyedCapture.Flags;
 
     Address srcField =
       Builder.CreateStructGEP(src, capture.getIndex(), capture.getOffset());
 
-    // If the captured record has a destructor then call it.
-    if (DestroyedCapture.Kind == BlockCaptureEntityKind::CXXRecord) {
-      const auto *Dtor =
-          CI.getVariable()->getType()->getAsCXXRecordDecl()->getDestructor();
-      PushDestructorCleanup(Dtor, srcField);
-
-      // If this is a __weak capture, emit the release directly.
-    } else if (DestroyedCapture.Kind == BlockCaptureEntityKind::ARCWeak) {
-      EmitARCDestroyWeak(srcField);
-
-    // Destroy strong objects with a call if requested.
-    } else if (DestroyedCapture.Kind == BlockCaptureEntityKind::ARCStrong) {
-      EmitARCDestroyStrong(srcField, ARCImpreciseLifetime);
-
-    // If this is a C struct that requires non-trivial destruction, emit a call
-    // to its destructor.
-    } else if (DestroyedCapture.Kind ==
-               BlockCaptureEntityKind::NonTrivialCStruct) {
-      QualType varType = CI.getVariable()->getType();
-      pushDestroy(varType.isDestructedType(), srcField, varType);
-
-    // Otherwise we call _Block_object_dispose.  It wouldn't be too
-    // hard to just emit this as a cleanup if we wanted to make sure
-    // that things were done in reverse.
-    } else {
-      assert(DestroyedCapture.Kind == BlockCaptureEntityKind::BlockObject);
-      llvm::Value *value = Builder.CreateLoad(srcField);
-      value = Builder.CreateBitCast(value, VoidPtrTy);
-      BuildBlockRelease(value, flags);
-    }
+    pushCaptureCleanup(DestroyedCapture.Kind, srcField,
+                       CI.getVariable()->getType(), flags, /*EHOnly*/ false, *this);
   }
 
   cleanups.ForceCleanup();
 
   FinishFunction();
 
   return llvm::ConstantExpr::getBitCast(Fn, VoidPtrTy);
 }
 
 namespace {
 
 /// Emits the copy/dispose helper functions for a __block object of id type.
 class ObjectByrefHelpers final : public BlockByrefHelpers {
   BlockFieldFlags Flags;
 
 public:
   ObjectByrefHelpers(CharUnits alignment, BlockFieldFlags flags)
     : BlockByrefHelpers(alignment), Flags(flags) {}
 
   void emitCopy(CodeGenFunction &CGF, Address destField,
                 Address srcField) override {
     destField = CGF.Builder.CreateBitCast(destField, CGF.VoidPtrTy);
 
     srcField = CGF.Builder.CreateBitCast(srcField, CGF.VoidPtrPtrTy);
     llvm::Value *srcValue = CGF.Builder.CreateLoad(srcField);
 

 void CodeGenFunction::BuildBlockRelease(llvm::Value *V, BlockFieldFlags flags) {
   llvm::Value *F = CGM.getBlockObjectDispose();
   llvm::Value *args[] = {
     Builder.CreateBitCast(V, Int8PtrTy),
     llvm::ConstantInt::get(Int32Ty, flags.getBitMask())
   };
   EmitNounwindRuntimeCall(F, args); // FIXME: throwing destructors?
 }
 
-namespace {
-  /// Release a __block variable.
-  struct CallBlockRelease final : EHScopeStack::Cleanup {
-    llvm::Value *Addr;
-    CallBlockRelease(llvm::Value *Addr) : Addr(Addr) {}
-
-    void Emit(CodeGenFunction &CGF, Flags flags) override {
-      // Should we be passing FIELD_IS_WEAK here?
-      CGF.BuildBlockRelease(Addr, BLOCK_FIELD_IS_BYREF);
-    }
-  };
-} // end anonymous namespace
-
-/// Enter a cleanup to destroy a __block variable.  Note that this
-/// cleanup should be a no-op if the variable hasn't left the stack
-/// yet; if a cleanup is required for the variable itself, that needs
-/// to be done externally.
-void CodeGenFunction::enterByrefCleanup(const AutoVarEmission &emission) {
-  // We don't enter this cleanup if we're in pure-GC mode.
-  if (CGM.getLangOpts().getGC() == LangOptions::GCOnly)
-    return;
-
-  EHStack.pushCleanup<CallBlockRelease>(NormalAndEHCleanup,
-                                        emission.Addr.getPointer());
+void CodeGenFunction::enterByrefCleanup(CleanupKind Kind, Address Addr,
+                                        BlockFieldFlags Flags,
+                                        bool LoadBlockVarAddr) {
+  EHStack.pushCleanup<CallBlockRelease>(Kind, Addr, Flags, LoadBlockVarAddr);
 }
 
 /// Adjust the declaration of something from the blocks API.
 static void configureBlocksRuntimeObject(CodeGenModule &CGM,
                                          llvm::Constant *C) {
   auto *GV = cast<llvm::GlobalValue>(C->stripPointerCasts());
 
   if (CGM.getTarget().getTriple().isOSBinFormatCOFF()) {

lib/CodeGen/CGDecl.cpp

if (const CleanupAttr *CA = D.getAttr<CleanupAttr>()) {
     llvm::Constant *F = CGM.GetAddrOfFunction(FD);
     assert(F && "Could not find function!");
 
     const CGFunctionInfo &Info = CGM.getTypes().arrangeFunctionDeclaration(FD);
     EHStack.pushCleanup<CallCleanupFunction>(NormalAndEHCleanup, F, &Info, &D);
   }
 
   // If this is a block variable, call _Block_object_destroy
-  // (on the unforwarded address).
-  if (emission.IsByRef)
-    enterByrefCleanup(emission);
+  // (on the unforwarded address). Don't enter this cleanup if we're in pure-GC
+  // mode.
+  if (emission.IsByRef && CGM.getLangOpts().getGC() != LangOptions::GCOnly) {
+    BlockFieldFlags Flags = BLOCK_FIELD_IS_BYREF;
+    if (emission.Variable->getType().isObjCGCWeak())
+      Flags |= BLOCK_FIELD_IS_WEAK;
+    enterByrefCleanup(NormalAndEHCleanup, emission.Addr, Flags,
+                      /*LoadBlockVarAddr*/ false);
+  }
 }
 
 CodeGenFunction::Destroyer *
 CodeGenFunction::getDestroyer(QualType::DestructionKind kind) {
   switch (kind) {
   case QualType::DK_none: llvm_unreachable("no destroyer for trivial dtor");
   case QualType::DK_cxx_destructor:
     return destroyCXXObject;

lib/CodeGen/CodeGenFunction.h

llvm::Constant *GenerateObjCAtomicGetterCopyHelperFunction(
                                              const ObjCPropertyImplDecl *PID);
   llvm::Value *EmitBlockCopyAndAutorelease(llvm::Value *Block, QualType Ty);
 
   void BuildBlockRelease(llvm::Value *DeclPtr, BlockFieldFlags flags);
 
   class AutoVarEmission;
 
   void emitByrefStructureInit(const AutoVarEmission &emission);
-  void enterByrefCleanup(const AutoVarEmission &emission);
+
+  /// Enter a cleanup to destroy a __block variable.  Note that this
+  /// cleanup should be a no-op if the variable hasn't left the stack
+  /// yet; if a cleanup is required for the variable itself, that needs
+  /// to be done externally.
+  ///
+  /// \param Kind Cleanup kind.
+  ///
+  /// \param Addr When \p LoadBlockVarAddr is false, the address of the __block
+  /// structure that will be passed to _Block_object_dispose. When
+  /// \p LoadBlockVarAddr is true, the address of the field of the block
+  /// structure that holds the address of the __block structure.
+  ///
+  /// \param Flags The flag that will be passed to _Block_object_dispose.
+  ///
+  /// \param LoadBlockVarAddr Indicates whether we need to emit a load from
+  /// \p Addr to get the address of the __block structure.
+  void enterByrefCleanup(CleanupKind Kind, Address Addr, BlockFieldFlags Flags,
+                         bool LoadBlockVarAddr);
 
   void setBlockContextParameter(const ImplicitParamDecl *D, unsigned argNum,
                                 llvm::Value *ptr);
 
   Address LoadBlockStruct();
   Address GetAddrOfBlockDecl(const VarDecl *var, bool ByRef);
 
   /// BuildBlockByrefAddress - Computes the location of the

test/CodeGenObjC/blocks.m

void test2(Test2 *x) {
   // Actually capture the value.
   // CHECK-NEXT: [[T6:%.*]] = getelementptr inbounds [[WEAK_T]], [[WEAK_T]]* [[WEAKX]], i32 0, i32 6
   // CHECK-NEXT: [[CAPTURE:%.*]] = load [[TEST2]]*, [[TEST2]]** [[X]]
   // CHECK-NEXT: store [[TEST2]]* [[CAPTURE]], [[TEST2]]** [[T6]]
 
   // Then we initialize the block, blah blah blah.
   // CHECK:      call void @test2_helper(
 
-  // Finally, kill the variable with BLOCK_FIELD_IS_BYREF.  We're not
-  // supposed to pass BLOCK_FIELD_IS_WEAK here.
+  // Finally, kill the variable with BLOCK_FIELD_IS_BYREF.
   // CHECK:      [[T0:%.*]] = bitcast [[WEAK_T]]* [[WEAKX]] to i8*
-  // CHECK:      call void @_Block_object_dispose(i8* [[T0]], i32 8)
+  // CHECK:      call void @_Block_object_dispose(i8* [[T0]], i32 24)
 
   __attribute__((objc_gc(weak))) __block Test2 *weakX = x;
   test2_helper(^{ [weakX destroy]; });
 }
 
 // rdar://problem/9124263
 // In the test above, check that the use in the invocation function
 // doesn't require a read barrier.

test/CodeGenObjCXX/arc-blocks.mm

-// RUN: %clang_cc1 -std=gnu++98 -triple x86_64-apple-darwin10 -emit-llvm -fobjc-runtime-has-weak -fblocks -fobjc-arc -o - %s | FileCheck %s
+// RUN: %clang_cc1 -std=gnu++98 -triple x86_64-apple-darwin10 -emit-llvm -fobjc-runtime-has-weak -fblocks -fobjc-arc -fexceptions -fobjc-arc-exceptions -o - %s | FileCheck -check-prefix CHECK %s
+// RUN: %clang_cc1 -std=gnu++98 -triple x86_64-apple-darwin10 -emit-llvm -fobjc-runtime-has-weak -fblocks -fobjc-arc -fexceptions -fobjc-arc-exceptions -O1 -o - %s | FileCheck -check-prefix CHECK-O1 %s
 
 // CHECK: [[A:.*]] = type { i64, [10 x i8*] }
+// CHECK: %[[STRUCT_TEST1_S0:.*]] = type { i32 }
+// CHECK: %[[STRUCT_BLOCK_DESCRIPTOR:.*]] = type { i64, i64 }
 
 // CHECK: [[LAYOUT0:@.*]] = private unnamed_addr constant [3 x i8] c" 9\00"
 
 // rdar://13045269
 // If a __block variable requires extended layout information *and*
 // a copy/dispose helper, be sure to adjust the offsets used in copy/dispose.
 namespace test0 {
   struct A {
namespace test0 {
   // CHECK-NEXT: ret void
 
   // CHECK:    define internal void [[DISPOSE_HELPER]](
   // CHECK:      [[T0:%.*]] = bitcast i8* {{.*}} to [[BYREF_A]]*
   // CHECK-NEXT: [[T1:%.*]] = getelementptr inbounds [[BYREF_A]], [[BYREF_A]]* [[T0]], i32 0, i32 7
   // CHECK-NEXT: call void @_ZN5test01AD1Ev([[A]]* [[T1]])
   // CHECK-NEXT: ret void
 }
+
+namespace test1 {
+
+// Check that copy/dispose helper functions are exception safe.
+
+// CHECK-LABEL: define internal void @__copy_helper_block_(
+// CHECK: %[[BLOCK_SOURCE:.*]] = bitcast i8* %{{.*}} to <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>*
+// CHECK: %[[BLOCK_DEST:.*]] = bitcast i8* %{{.*}} to <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>*
+
+// CHECK: %[[V4:.*]] = getelementptr inbounds <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>, <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>* %[[BLOCK_SOURCE]], i32 0, i32 6
+// CHECK: %[[V5:.*]] = getelementptr inbounds <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>, <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>* %[[BLOCK_DEST]], i32 0, i32 6
+// CHECK: %[[BLOCKCOPY_SRC:.*]] = load i8*, i8** %[[V4]], align 8
+// CHECK: %[[V6:.*]] = bitcast i8** %[[V5]] to i8*
+// CHECK: call void @_Block_object_assign(i8* %[[V6]], i8* %[[BLOCKCOPY_SRC]], i32 8)
+
+// CHECK: %[[V7:.*]] = getelementptr inbounds <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>, <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>* %[[BLOCK_SOURCE]], i32 0, i32 7
+// CHECK: %[[V8:.*]] = getelementptr inbounds <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>, <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>* %[[BLOCK_DEST]], i32 0, i32 7
+// CHECK: call void @objc_copyWeak(i8** %[[V8]], i8** %[[V7]])
+
+// CHECK: %[[V9:.*]] = getelementptr inbounds <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>, <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>* %[[BLOCK_SOURCE]], i32 0, i32 5
+// CHECK: %[[V10:.*]] = getelementptr inbounds <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>, <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>* %[[BLOCK_DEST]], i32 0, i32 5
+// CHECK: %[[BLOCKCOPY_SRC2:.*]] = load i8*, i8** %[[V9]], align 8
+// CHECK: store i8* null, i8** %[[V10]], align 8
+// CHECK: call void @objc_storeStrong(i8** %[[V10]], i8* %[[BLOCKCOPY_SRC2]])
+
+// CHECK: %[[V11:.*]] = getelementptr inbounds <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>, <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>* %[[BLOCK_SOURCE]], i32 0, i32 8
+// CHECK: %[[V12:.*]] = getelementptr inbounds <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>, <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>* %[[BLOCK_DEST]], i32 0, i32 8
+// CHECK: invoke void @_ZN5test12S0C1ERKS0_(%[[STRUCT_TEST1_S0]]* %[[V12]], %[[STRUCT_TEST1_S0]]* dereferenceable(4) %[[V11]])
+// CHECK: to label %[[INVOKE_CONT:.*]] unwind label %[[LPAD:.*]]
+
+// CHECK: [[INVOKE_CONT]]:
+// CHECK: %[[V13:.*]] = getelementptr inbounds <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>, <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>* %[[BLOCK_SOURCE]], i32 0, i32 9
+// CHECK: %[[V14:.*]] = getelementptr inbounds <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>, <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>* %[[BLOCK_DEST]], i32 0, i32 9
+// CHECK: invoke void @_ZN5test12S0C1ERKS0_(%[[STRUCT_TEST1_S0]]* %[[V14]], %[[STRUCT_TEST1_S0]]* dereferenceable(4) %[[V13]])
+// CHECK: to label %[[INVOKE_CONT4:.*]] unwind label %[[LPAD3:.*]]
+
+// CHECK: [[INVOKE_CONT4]]:
+// CHECK: ret void
+
+// CHECK: [[LPAD]]:
+// CHECK: br label %[[EHCLEANUP:.*]]
+
+// CHECK: [[LPAD3]]:
+// CHECK: invoke void @_ZN5test12S0D1Ev(%[[STRUCT_TEST1_S0]]* %[[V12]])
+// CHECK: to label %[[INVOKE_CONT5:.*]] unwind label %[[TERMINATE_LPAD:.*]]
+
+// CHECK: [[INVOKE_CONT5]]:
+// CHECK: br label %[[EHCLEANUP]]
+
+// CHECK: [[EHCLEANUP]]:
+// CHECK: call void @objc_storeStrong(i8** %[[V10]], i8* null)
+// CHECK: call void @objc_destroyWeak(i8** %[[V8]])
+// CHECK: %[[V21:.*]] = load i8*, i8** %[[V5]], align 8
+// CHECK: call void @_Block_object_dispose(i8* %[[V21]], i32 8)
+// CHECK: br label %[[EH_RESUME:.*]]
+
+// CHECK: [[EH_RESUME]]:
+// CHECK: resume { i8*, i32 }
+
+// CHECK: [[TERMINATE_LPAD]]:
+// CHECK: call void @__clang_call_terminate(
+
+// CHECK-O1-LABEL: define internal void @__copy_helper_block_(
+// CHECK-O1: tail call void @objc_release({{.*}}) {{.*}} !clang.imprecise_release
+
+// CHECK: define internal void @__destroy_helper_block_(
+// CHECK: %[[BLOCK:.*]] = bitcast i8* %{{.*}} to <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>*
+// CHECK: %[[V2:.*]] = getelementptr inbounds <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>, <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>* %[[BLOCK]], i32 0, i32 6
+// CHECK: %[[V3:.*]] = getelementptr inbounds <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>, <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>* %[[BLOCK]], i32 0, i32 7
+// CHECK: %[[V4:.*]] = getelementptr inbounds <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>, <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>* %[[BLOCK]], i32 0, i32 5
+// CHECK: %[[V5:.*]] = getelementptr inbounds <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>, <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>* %[[BLOCK]], i32 0, i32 8
+// CHECK: %[[V6:.*]] = getelementptr inbounds <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>, <{ i8*, i32, i32, i8*, %[[STRUCT_BLOCK_DESCRIPTOR]]*, i8*, i8*, i8*, %[[STRUCT_TEST1_S0]], %[[STRUCT_TEST1_S0]] }>* %[[BLOCK]], i32 0, i32 9
+// CHECK: invoke void @_ZN5test12S0D1Ev(%[[STRUCT_TEST1_S0]]* %[[V6]])
+// CHECK: to label %[[INVOKE_CONT:.*]] unwind label %[[LPAD:.*]]
+
+// CHECK: [[INVOKE_CONT]]:
+// CHECK: invoke void @_ZN5test12S0D1Ev(%[[STRUCT_TEST1_S0]]* %[[V5]])
+// CHECK: to label %[[INVOKE_CONT2:.*]] unwind label %[[LPAD1:.*]]
+
+// CHECK: [[INVOKE_CONT2]]:
+// CHECK: call void @objc_storeStrong(i8** %[[V4]], i8* null)
+// CHECK: call void @objc_destroyWeak(i8** %[[V3]])
+// CHECK: %[[V7:.*]] = load i8*, i8** %[[V2]], align 8
+// CHECK: call void @_Block_object_dispose(i8* %[[V7]], i32 8)
+// CHECK: ret void
+
+// CHECK: [[LPAD]]:
+// CHECK: invoke void @_ZN5test12S0D1Ev(%[[STRUCT_TEST1_S0]]* %[[V5]])
+// CHECK: to label %[[INVOKE_CONT3:.*]] unwind label %[[TERMINATE_LPAD]]
+
+// CHECK: [[LPAD1]]
+// CHECK: br label %[[EHCLEANUP:.*]]
+
+// CHECK: [[INVOKE_CONT3]]:
+// CHECK: br label %[[EHCLEANUP]]
+
+// CHECK: [[EHCLEANUP]]:
+// CHECK: call void @objc_storeStrong(i8** %[[V4]], i8* null)
+// CHECK: call void @objc_destroyWeak(i8** %[[V3]])
+// CHECK: %[[V14:.*]] = load i8*, i8** %[[V2]], align 8
+// CHECK: call void @_Block_object_dispose(i8* %[[V14]], i32 8)
+// CHECK: br label %[[EH_RESUME]]
+
+// CHECK: [[EH_RESUME]]:
+// CHECK: resume { i8*, i32 }
+
+// CHECK: [[TERMINATE_LPAD]]:
+// CHECK: call void @__clang_call_terminate(
+
+// CHECK-O1-LABEL: define internal void @__destroy_helper_block_(
+// CHECK-O1: tail call void @objc_release({{.*}}) {{.*}} !clang.imprecise_release
+// CHECK-O1: tail call void @objc_release({{.*}}) {{.*}} !clang.imprecise_release
+
+struct S0 {
+  S0();
+  S0(const S0 &);
+  ~S0();
+  int f0;
+};
+
+id getObj();
+
+void foo1() {
+  __block id t0 = getObj();
+  __weak id t1 = getObj();
+  id t2 = getObj();
+  S0 t3, t4;
+  ^{ (void)t0; (void)t1; (void)t2; (void)t3; (void)t4; };
+}
+}