[ObjC] Enable __strong pointers in structs under ARC

[ahatanak]

Summary

ObjectiveC ARC in C mode currently disallows having strong and weak pointers in structs. This patch takes the first step towards lifting that restriction. In order to properly manage the lifetimes of the objects owned by the structs, this patch modifies IRGen to synthesize special functions for structs with __strong pointers and call them when those structs have to be initialized, copied, and destructed similarly to what C++ special member functions of non-trivial classes do.

I plan to send a patch that allows __weak pointers in structs after this patch is committed.

[ahatanak]

I just found that the code that creates the mangled name for a special function is not correct. Two structs with different record layouts can end up having functions that have the same name.

I'll fix the bug and update the patch today.

[ahatanak]

Fixed a bug where the code gets stuck in an infinite loop when a struct with an array field is passed to FieldInfoToString. Encoded the offset of an array in a struct into the function name. Also, added comments that describe the structure of the mangled function name.

[ahatanak]

Insert an underscore before src-alignment in the BNF rule for alignment-info.

<alignment-info> ::= <dst-alignment> ["_" <src-alignment>]

[rjmccall]

You should add a has_feature check for this (has_feature(objc_arc_fields)?), as well as documentation. The documentation would go in the ARC spec.

include/clang/AST/Decl.h

3575 ↗

(On Diff #127233)

Naming style would be "isNonTrivialToDefualtInitialize" and so on for all of these.

If these aren't guaranteed to be consistent with the C++ definitions (and the copying one almost certainly isn't, since there isn't a single definition of copying in C++), I think you probably need to name these in a way that makes it clear that it's the C bit that's being queried/changed. Perhaps adding "Primitive" to each one, e.g. isNonTrivialToPrimitiveCopy? Richard is probably opinionated about this.

lib/AST/ASTContext.cpp

5780 ↗

(On Diff #127233)

"Return true" is just a description of the code that follows. I would suggest something like "The block needs copy/destroy helpers if...".

lib/CodeGen/CGBlocks.cpp

2244 ↗

(On Diff #127233)

You've updated the code for __block captures, but I think you missed computeBlockInfo for direct captures.

lib/CodeGen/CGDecl.cpp

1421 ↗

(On Diff #127233)

This can only be getARCCleanupKind() if it's non-trivial to destroy solely because of strong members. Since the setup here is significantly more general than ARC, I think you need to default to the more-correct behavior; if you want to add a special-case check for only strong members, you ought to do that explicitly.

lib/CodeGen/CGExprAgg.cpp

315 ↗

(On Diff #127233)

Do these functions have a memcpy as a precondition? I would expect them to do the full copy (for code size if nothing else).

lib/CodeGen/CMakeLists.txt

73 ↗

(On Diff #127233)

I think CGNonTrivialStruct.cpp would be a better name.

[rjmccall]

I accidentally hit 'send' too early on my review, so here's part two. I still haven't fully reviewed the new IRGen file.

lib/CodeGen/CodeGenCStruct.cpp

27 ↗

(On Diff #127233)

There's no need to put this in an anonymous namespace.

This enum makes me wonder if it would make more sense to create equivalents to QualType::DestructionKind for each of these operations and all of the different primitive types. That would let you e.g. implement your only-strong-members check above much more easily, and it would also make it simpler to guarantee that all the places that need to exhaustively enumerate the reasons why something might need special initialization/copying logic don't miss an important new case.

lib/CodeGen/CodeGenFunction.cpp

1144 ↗

(On Diff #127233)

You're not actually checking the "is not passed indirectly" part of this, but I think that's okay, because I don't think we actually want the ownership aspects of the ABI to vary based on how the argument is passed. So you should just strike that part from the comment.

Also, this should really be done in EmitParmDecl.

lib/CodeGen/CodeGenFunction.h

3347 ↗

(On Diff #127233)

These methods should *definitely* be somehow namespaced for your new feature.

[ahatanak]

Address review comments.

lib/CodeGen/CGDecl.cpp

1421 ↗

(On Diff #127233)

I added a DestructionKind (DK_c_struct_strong_field) that is used just for structs that have strong fields (but doesn't have weak fields).

lib/CodeGen/CGExprAgg.cpp

315 ↗

(On Diff #127233)

Yes, there should always be a call to memcpy before the copy/move special functions are called. I don't think we want to fold the call to memcpy into the body of the special functions since another special function can be called from the body if the non-trivial C struct has a field whose type is a non-trivial C struct too (in which case, there will be a redundant memcpy to copy the C struct field).

For example, in the following code, there shouldn't be a call to memcpy to copy field "f0" of StrongOuter if there is already a memcpy that copies struct StrongOuter:

typedef struct {
  int f0;
  id f1;
} Strong;

typedef struct {
  Strong f0;
  id f1;
} StrongOuter;

lib/CodeGen/CodeGenCStruct.cpp

27 ↗

(On Diff #127233)

I'm not sure if I understood your point here. Do you mean there should be DestructionKinds for arrays or structs too or merge these enums into DestructionKind?

lib/CodeGen/CodeGenFunction.cpp

1144 ↗

(On Diff #127233)

To make ARC and ARC++ compatible, I think we'll have to change the ABI of C++ functions that are passed structs containing weak fields if we want to destruct non-trivial C arguments destructed in the callee. I guess that's OK since we have to break the ABI for structs containing strong fields too.

lib/CodeGen/CodeGenFunction.h

3347 ↗

(On Diff #127233)

I renamed these methods to make it clear that they are calling special functions for C structs. Is that what you mean by "namespaced"?

[ahatanak]

Update patch.

[rjmccall]

lib/CodeGen/CGDecl.cpp

1421 ↗

(On Diff #127233)

Sure, that's a simple enough way to do it, and I think it's fairly warranted.

lib/CodeGen/CGExprAgg.cpp

315 ↗

(On Diff #127233)

Well, I guess I was imagining something more C++-ish where you don't necessarily have a struct-wide memcpy, and instead you just memcpy the parts where that's profitable and otherwise do something type-specific, which would mean recursing for a struct. Your approach is reasonable if the non-trivial copying is relatively sparse and the structure is large; on the other hand, if the non-trivial copying is dense, the memcpy itself might be mostly redundant. And it does mean a bigger code-size hit in the original place that kicks off the copy.

IIRC we do already have some code in copy-constructor emission that's capable of emitting sequences of field copies with a memcpy, if you wanted to try the C++ style, you could probably take advantage of that pretty easily. But I won't hold up the patch if you think the memcpy precondition is the right way to go.

lib/CodeGen/CodeGenCStruct.cpp

27 ↗

(On Diff #127233)

I was suggesting there should be equivalents to DestructionKind for some of the other operations, like a NonTrivialCopyKind, and queries on QualType analogous to isDestructedType(), e.g. isNonTrivialToCopy(). That way you can directly put this stuff in the AST, and then here you can just walk the struct fields and switch on the enum, at which point I think you don't need FieldKind at all. Remember that some types are non-trivial in only a subset of these dimensions.

lib/CodeGen/CodeGenFunction.cpp

1144 ↗

(On Diff #127233)

I hadn't been thinking of weak, just the difference between small and large structs, but this is a good point. I think I agree with you, we should also change the destruction ABI for structs containing weak fields; basically, the rule will be that only C++ destruction (under the Itanium ABI) requires destruction in the caller.

lib/CodeGen/CodeGenFunction.h

3347 ↗

(On Diff #127233)

Yes, thank you.

[ahatanak]

Address review comments. Changed the mangling of special functions to make it a bit easier to read.

[ahatanak]

lib/CodeGen/CGExprAgg.cpp

315 ↗

(On Diff #127233)

I made changes so that either a load/store pair or a call to memcpy is emitted in the copy/move special functions to copy fields that are not non-trivial.

lib/CodeGen/CodeGenCStruct.cpp

27 ↗

(On Diff #127233)

I added enum NonTrivialCopyKind and function isNonTrivialToCopy to QualType and removed FieldKind from CGNonTrivialStruct.cpp. Since there are special functions other than copy constructor/assignment operator, should the enum's name be NonTrivialStructKind or something instead of NonTrivialCopyKind?

[ahatanak]

lib/CodeGen/CGExprAgg.cpp

315 ↗

(On Diff #127233)

I haven't implemented the optimization you suggested which merges a sequence of field copies into a single memcpy. I'll look into it later.

[ahatanak]

• Improved IRGen for copying trivial fields in a non-trivial C struct. IRGen in CGNonTrivialStruct.cpp now calls a single memcpy if there are consecutive trivial fields in a struct (it does something similar to what FieldMemcpyizer does)

• IRGen for structs containing bitfields was not correct, so I fixed that too.

[rjmccall]

include/clang/AST/Type.h

1152 ↗

(On Diff #127818)

We don't actually distinguish arrays in DestructionKind. Is it important here? You can't actually do anything with that information.

Regarding your naming question, I wonder if it would be useful to just invent a new term here, something meaning "non-trivial" but without the C++ baggage. Maybe just "PrimitiveCopyKind", with the understanding that C++ can never do a "primitive" copy? And for consistency with DestructionKind, maybe you should lowercase the second words.

I'm not sure how isNonTrivialToCopy is supposed to be used. Where does the function come from? And why is a context required when it isn't required for isDestructedType?

[ahatanak]

include/clang/AST/Type.h

1152 ↗

(On Diff #127818)

Enum NonTrivialCopyKind and isNonTrivialToCopy() are used just to distinguish the different types of fields in a C struct. Currently, there are four types that are handled by visitField functions in CGNonTrivialStruct.cpp:

• struct field
• array field
• __strong field
• trivial field that can be copied using memcpy

I thought using enums would make the code easier to read, but of course it's possible to remove them and instead just check the field types calling functions like getAsArrayType or getAs<RecordType>. ASTContext is passed to isNonTrivialToCopy so that it can call ASTContext::getAsArrayType to determine whether the type is an array. Maybe there is another way to detect an array that doesn't require ASTContext?

As for the naming, PrimitiveCopyKind seems find if we want to distinguish it from C++ non-trivial classes (I don't have a better name). If we are going to call non-trivial C structs primitiveCopyKind, what should we call the C structs that are trivial (those that can be copied using memcpy)?

[rjmccall]

include/clang/AST/Type.h

1152 ↗

(On Diff #127818)

I think "trivial" is a reasonable kind of primitive-copy semantics. Basically, we're saying that all complete object types other than non-trivial C++ types have some sort of primitive-copy semantics, and this enum tells us what those semantics are.

DestructionKind has to deal with arrays as well, but it does so by just reporting the appropriate value for the underlying element type without mentioning that it's specifically an *array* of the type. I think that's a reasonable design, since most places do not need to distinguish arrays from non-arrays. IRGen does, but only when you're actually finally emitting code; prior to that (when e.g. deciding that a field is non-trivial to copy) you can just use the enum value.

You can do without getAsArrayType the same way that isDestructedType() does: the more complex operation is only necessary in order to preserve qualifiers on the element type, but that's unnecessary for this operation because the array type itself is considered to have the same qualifiers as its element type. That is, you can ask the original type whether it's strong/weak-qualified without having to specifically handle array types, and once you've done all of those queries, you can use the "unsafe" operations to drill down to the element type because you no longer care about qualification.

[ahatanak]

include/clang/AST/Type.h

1152 ↗

(On Diff #127818)

So QualType::hasNonTrivialToDefaultInitializeStruct should be renamed to QualType::hasPrimitiveDefaultInitializeStruct and RecordDecl::isNonTrivialToPrimitiveDefaultInitialize to RecordDecl::isPrimitiveDefaultInitialize, for example?

Also, I realized that, after I made the changes that removed the NonTrivial flags from RecordDecl, CGNonTrivialStruct.cpp is the only user of enum NonTrivialCopyKind and the following methods of QualType (Sema::isValidVarArgType calls nonTrivialToDestroy, but it should call methods like hasNonTrivialToDestroyStruct that detect non-trivial C structs instead):

nonTrivialToDefaultInitialize
nonTrivialToCopy
nonTrivialToDestructiveMove
nonTrivialToDestroy

Maybe we should make these enums and functions local to CGNonTrivialStruct.cpp to avoid adding something that is not used outside CGNonTrivialStruct.cpp?

[rjmccall]

include/clang/AST/Type.h

1152 ↗

(On Diff #127818)

I think the right name would be isNonTrivialToPrimitiveDefaultInitialize(), and like isDestructedType() it would return an enum that happens to have a zero value (and is thus false) for the trivial case.

I think we are likely to add more places that use these methods and enums. For example, if we want to add a diagnostic that warns about memcpy'ing non-trivial struct types, that diagnostic should probably include a note about which field is non-trivial, and that will still basically want to do this same investigation.

[ahatanak]

Rename functions. Check whether a non-trivial type passed to Sema::isValidVarArgType is a struct.

[ahatanak]

include/clang/AST/Type.h

1152 ↗

(On Diff #127818)

I renamed the functions to names that start with isNonTrivialToPrimitive and removed the enumerator for "Array" from the enum too.

[ahatanak]

In EmitCallArg and EmitParmDecl, use the existing code for Microsoft C++ ABI to determine whether the argument should be destructed in the callee. Also, add a test case that checks the destructor for a non-trivial C struct is called on the EH path.

[rjmccall]

include/clang/AST/Decl.h

3529 ↗

(On Diff #129359)

Is it interesting to track all the individual reasons that a struct is non-trivial at the struct level, as opposed to (like we do with CXXDefinitionData) just tracking the four aggregate properties you've described below?

include/clang/AST/Type.h

1098 ↗

(On Diff #129359)

I think this one should probably get its own enum. It's not too hard to imagine types (maybe relative references, or something else that's address-sensitive?) that require non-trivial copy semantics but isn't non-trivial to default-initialize.

1108 ↗

(On Diff #129359)

You need to define what you mean by "destructive move":

• A C++-style move leaves the source in a still-initialized state, just potentially with a different value (such as nil for a strong reference). This is what we would do when e.g. loading from an r-value reference in C++. It's also what we have to do when moving a block variable to the heap, since there may still be pointers to the stack copy of the variable, and since the compiler will unconditionally attempt to destroy that stack copy when it goes out of scope. Since the source still needs to be destroyed, a non-trivially-copyable type will probably always have to do non-trivial work to do this. Because of that, a function for this query could reasonably just return PrimitiveCopyKind.

• A truly primitive destructive move is something that actually leaves the source uninitialized. This is generally only possible in narrow circumstances where the compiler can guarantee that the previous value is never again going to be referenced, including to destroy it. I don't know if you have a reason to track this at all (i.e. whether there are copies you want to perform with a destructive move in IRGen), but if you do, you should use a different return type, because many types that are non-trivial to "C++-style move" are trivial to "destructively move": for example, __strong references are trivial to destructively move.

1113 ↗

(On Diff #129359)

Is there a reason we need this in addition to isDestructedType()? It seems to me that it's exactly the same except ruling out the possibility of a C++ destructor.

1137 ↗

(On Diff #129359)

Are these four queries really important enough to provide API for them on QualType?

1148 ↗

(On Diff #129359)

I don't think you want to refer to the fact that the C struct specifically contains a __strong field here. As we add more reasons, would we create a new enumerator for each? What if a struct is non-trivial for multiple reasons? Just say that it's a non-trivial C struct.

lib/AST/ASTContext.cpp

5778 ↗

(On Diff #129359)

I think you should use the non-struct-specific checks here. In addition to being cleaner, it would also let you completely short-circuit the rest of this function in ARC mode. In non-ARC mode, you do still have to check lifetime (only to see if it's unsafe_unretained; the strong and __weak cases would be unreachable), and you need the type-specific special cases.

[ahatanak]

include/clang/AST/Type.h

1148 ↗

(On Diff #129359)

I added an enumerator for DK_c_struct_strong_field since CodeGenFunction::needsEHCleanup distinguishes between __weak and __strong types. Is it not necessary to distinguish between a struct that has a __weak field and a struct that has a __strong field but not a __weak field?

[rjmccall]

include/clang/AST/Type.h

1148 ↗

(On Diff #129359)

Oh, that's right.

I... hmm. The problem is that that's really a special-case behavior, and we're designing a more general feature. If we try to handle the special case in the first patch, we'll end up trivializing the general case, and that will permeate the design in unfortunate ways.

So I recommend that we *not* treat them separately right now; just emit cleanups for them unconditionally. You're still planning to add support for __weak properties in a follow-up patch, right? Once that's in, it will make more sense to carve out the __strong-only behavior as a special case.

[ahatanak]

Address review comments and feedback I got from John offline.

The main changes are in CGNonTrivialStruct.cpp. I cleaned up the class hierarchy and used variadic template functions in CGNonTrivialStruct.cpp to enable sharing code between the class creating the function name and the class emitting IR for the special functions. I also made changes so that memset is used instead of a loop to default-initialize a large array (see test_constructor_destructor_IDArray in test/CodeGenObjC/strong-in-c-struct.m).

[rjmccall]

include/clang/AST/Type.h

1102 ↗

(On Diff #133994)

This is unused.

lib/CodeGen/CGBlocks.cpp

1565 ↗

(On Diff #133994)

Please restructure this code to use the result of isNonTrivialToPrimitiveCopy(), preferably in a way that involves an exhaustive switch. That should be as easy as breaking out in the PCK_Strong case and in the PCK_None case when !isObjCRetainableType().

1773 ↗

(On Diff #133994)

Same as above with using the result of isDestructedType().

2070 ↗

(On Diff #133994)

Just isDestructedType() should be fine.

lib/CodeGen/CGDecl.cpp

1299 ↗

(On Diff #133994)

Again, it would be nice to restructure this code to use the result of isNonTrivialToPrimitiveDefaultInitialize() exhaustively.

lib/CodeGen/CGNonTrivialStruct.cpp

38 ↗

(On Diff #133994)

These visitors would be re-usable components outside of IRGen if they just took a QualType and then forwarded an arbitrary set of other arguments. I would suggest structuring them like this:

template <class Derived, class RetTy> struct DestructedTypeVisitor {
  Derived &asDerived() { return static_cast<Derived&>(*this); }

  template <class... Ts>
  RetTy visit(QualType Ty, Ts &&...Args) {
    return asDerived().visit(Ty.isDestructedType(), Ty, std::forward<Ts>(Args)...);
  }

  template <class... Ts>
  RetTy visit(QualType::DestructionKind DK, QualType Ty, Ts &&...Args) {
    switch (DK) {
    case QualType::DK_none: return asDerived().visitTrivial(Ty, std::forward<Ts>(Args)...);
    case QualType::DK_cxx_destructor: return asDerived().visitCXXDestructor(Ty, std::forward<Ts>(Args)...);
    ...
    }
  }
};

You can then pretty easily add the special behavior for IsVolatile and arrays (when DK is not DK_none) in an IRGen-specific subclass. visitArray would take a DK and then recurse by calling back into visit with that same DK.

55 ↗

(On Diff #133994)

Better not to have a default case here. You can add an assert for the C++ case that it should never get here.

92 ↗

(On Diff #133994)

This entire template is dead.

110 ↗

(On Diff #133994)

Please pass around offsets as CharUnits until you actually need to interact with LLVM.

131 ↗

(On Diff #133994)

I think you should just call this CopyStructVisitor.

145 ↗

(On Diff #133994)

This would also be a good place to check for arrays so you don't have to do it in every non-trivial case below.

There's a couple other places in this file where I would suggest the same thing.

289 ↗

(On Diff #133994)

Offset should definitely be a CharUnits here.

432 ↗

(On Diff #133994)

You should at least make sure this has the right function type; it's possible, even if not really allowed, to declare a C function that collides with one of these names.

lib/CodeGen/CodeGenFunction.h

3407 ↗

(On Diff #133994)

I think we're moving towards generally taking LValues instead of Addresses unless you're talking about a very low-level routine.

[ahatanak]

Address review comments.

include/clang/AST/Type.h

1137 ↗

(On Diff #129359)

I removed these functions.

lib/CodeGen/CGDecl.cpp

1299 ↗

(On Diff #133994)

It looks like this code needs restructuring, but I didn't understand how I can use isNonTrivialToPrimitiveDefaultInitialize() here. The only change I made here is that drillIntoBlockVariable is called if the variable is byref. This fixes a bug where a non-trivial struct variable declared as __block wasn't initialized correctly.

Also, in the process, I found that when a c++ struct declared as a __block is captured by its initializer, an assertion fails (see test case in test/CodeGenObjCXX/arc-blocks.mm). I fixed the bug in CodeGenFunction::EmitExprAsInit, but I'm not sure that's the right way to fix it. I'm not sure what the comment "how can we delay here if D is captured by its initializer" means.

lib/CodeGen/CGNonTrivialStruct.cpp

38 ↗

(On Diff #133994)

I created three visitor classes that can be used outside of IRGen,

55 ↗

(On Diff #133994)

I added llvm_unreachable in StructVisitor's methods visitWeak and visitCXXDestructor.

[rjmccall]

include/clang/AST/Decl.h

3619 ↗

(On Diff #134430)

Please document that this means a C++-style destructive move that leaves the source object in a validly-initialized state.

include/clang/AST/Type.h

1094 ↗

(On Diff #134430)

Please call this ARCStrong, it'll be much clearer (and easier to visually distinguish from "struct').

1096 ↗

(On Diff #134430)

Nit-pick: please declare this with the methods that use it.

lib/AST/ASTContext.cpp

5795 ↗

(On Diff #134430)

Nit-pick: period.

lib/CodeGen/CGBlocks.cpp

1770 ↗

(On Diff #134430)

Should we just have a helper function like getBlockFieldFlagsForObjCObjectPointer(QualType);? You could just call it from all the object-specific places instead of having this Flags variable that's not valid in half the cases.

lib/CodeGen/CGDecl.cpp

1423 ↗

(On Diff #134430)

The "delay" is that we generally handle __block captures within initializers by delaying the drill-into until we're ready to actually store into the variable. Not delaying has the advantage of allowing us to just project out the stack storage instead of actually chasing the forwarding pointer, but this isn't safe if the forwarding pointer might no longer point to the stack storage, which is what happens when a block capturing a __block variable is copied to the heap.

Delaying the drill-into is easy when the stores can be completely separated from the initializer, as is true for scalar and complex types. It's not easy for aggregates because we need to emit the initializer in place, but it's possible that a block capturing the __block variable will be copied during the emission of that initializer (e.g. during a later element of a braced-init-list or within a C++ constructor call), meaning that we might be copying an object that's only partly initialized. This is particularly dangerous if there are destructors involved.

Probably the best solution is to forbid capturing __block variables of aggregate type within their own initializer. It's possible that that might break existing code that's not doing the specific dangerous things, though.

The more complex solution is to make IRGen force the __block variable to the heap before actually initializing it. (It can do this by directly calling _Block_object_assign.) We would then initialize the heap copy in-place, safely knowing that there is no possibility that it will move again. Effectively, the stack copy of the variable would never exist as an object; only its header would matter, and only enough to allow the runtime to "copy" it to the heap. We'd have to hack the __block variable's copy helper to not try to call the move-constructor, and we'd need to suppress the cleanup that destroys the stack copy of the variable. We already push a cleanup to release the __block variable when it goes out of scope, and that would stay.

1421 ↗

(On Diff #127233)

Okay, since you don't have a DK_c_struct_strong_field anymore, you either need to stop using getARCCleanupKind() or do the test for just __strong fields.

lib/CodeGen/CGExprAgg.cpp

82 ↗

(On Diff #134430)

Please use a boolean enum for this instead of a raw bool.

255 ↗

(On Diff #134430)

Hrm. Okay, I guess. This is specific to ignored results because otherwise we assume that the caller is going to manage the lifetime of the object?

lib/CodeGen/CGNonTrivialStruct.cpp

63 ↗

(On Diff #134430)

DefaultInitialize*d*TypeVisitor would be better, I think.

77 ↗

(On Diff #134430)

Use returns, please. And you can add an llvm_unreachable after the switch (in all of the visitors).

92 ↗

(On Diff #134430)

CopiedTypeVisitor, I guess.

138 ↗

(On Diff #134430)

These computations should be in uint64_t.

143 ↗

(On Diff #134430)

ASTContext has a toCharUnitsFromBits method.

184 ↗

(On Diff #134430)

You can add using StructVisitor<Derived>::asDerived; to the class to avoid having to write this a bunch.

193 ↗

(On Diff #134430)

I feel like maybe volatile fields should be individually copied instead of being aggregated into a multi-field memcpy. This is a more natural interpretation of the C volatile rules than we currently do. In fact, arguably we should really add a PrimitiveCopyKind enumerator for volatile fields (that are otherwise trivially-copyable) and force all copies of structs with volatile fields into this path precisely so that we can make a point of copying the volatile fields this way. (Obviously that part is not something that's your responsibility to do.)

To get that right with bit-fields, you'll need to propagate the actual FieldDecl down. On the plus side, that should let you use EmitLValueForField to do the field projection in the common case.

341 ↗

(On Diff #134430)

"N-ary" is the usual spelling.

[ahatanak]

Address review comments.

lib/CodeGen/CGDecl.cpp

1423 ↗

(On Diff #134430)

Ah I see. I'll see if I can implement the first solution you suggested in a separate patch. If it turns out that doing so breaks too much code, we can consider implementing the second solution.

Is this something users do commonly?

lib/CodeGen/CGExprAgg.cpp

255 ↗

(On Diff #134430)

Yes, this is specific to ignored results. This is needed because otherwise function return values that are ignored won't get destructed by the caller.

lib/CodeGen/CGNonTrivialStruct.cpp

193 ↗

(On Diff #134430)

I added method visitVolatileTrivial that copies volatile fields individually. Please see test case test_copy_constructor_Bitfield1 in test/CodeGenObjC/strong-in-c-struct.m.

[rjmccall]

include/clang/AST/Type.h

1108 ↗

(On Diff #135309)

These should all be /// documentation comments, and they mostly shouldn't talk about fields since this is a query on QualType, not FieldDecl. I would suggest something like:

for Trivial - The type does not fall into any of the following categories. Note that this case is zero-valued so that values of this enum can be used as a boolean condition for non-triviality.

for VolatileTrivial - The type would be trivial except that it is volatile-qualified. Types that fall into one of the other non-trivial cases may additionally be volatile-qualified.

for ARCStrong - The type is an Objective-C retainable pointer type that is qualified with the ARC __strong qualifier.

for Struct - The type is a struct containing a field whose type is not PCK_Trivial. Note that a C++ struct type does not necessarily match this; C++ copying semantics are too complex to express here, in part because they depend on the exact constructor or assignment operator that is chosen by overload resolution to do the copy.

1121 ↗

(On Diff #135309)

"truly"

Hmm. Now that I'm thinking more about it, I'm not sure there's any point in tracking non-triviality of a C++-style destructive move separately from the non-triviality of a copy. It's hard to imagine that there would ever be a non-C++ type that primitively has non-trivial copies but trivial C++-style moves or vice-versa. Type-based destructors imply that the type represents some kind of resource, and a C++-style move will always be non-trivial for resource types because ownership of the resource needs to be given up by the old location. Otherwise, a type might be non-trivial to copy but not destroy because there's something special about how it's stored (like volatility), but it's hard to imagine what could possibly cause it to be non-trivial to destroy but not copy.

If we were tracking non-triviality of an *unsafe* destructive move, one that leaves the source in an uninitialized state, that's quite different.

I think there are three reasonable options here:

• Ignore the argument I just made about the types that we're *likely* to care about modeling and generalize your tracking to also distinguish construction from assignment. In such an environment, I think you can absolutely make an argument that it's still interesting to track C++-style moves separately from copies.

• Drop the tracking of destructive moves completely. If you want to keep the method around, find, but it can just call isNonTrivialToPrimitiveCopy().

• Change the tracking of *destructive* moves to instead track *deinitializing* moves. The implementation would stop considering __strong types to be non-trivial to move.

But as things stand today, I do not see any point in separately tracking triviality of C++-style destructive moves.

lib/AST/Type.cpp

2231 ↗

(On Diff #135309)

You can re-use this call to getQualifiers() in the check for volatile below.

3945 ↗

(On Diff #135309)

Please combine this check with the getAsCXXRecordDecl() check below. That is, (1) check whether it's a record type once, and if so, (2) it's a CXXRecordDecl, ask about its destructor, and (3) if it's not a CXXRecordDecl, ask whether it's non-trivial to primitive destroy.

lib/CodeGen/CGBlocks.cpp

1779 ↗

(On Diff #135309)

This entire switch can be within an #ifndef NDEBUG if you really feel it's important to assert.

1792 ↗

(On Diff #135309)

I think you should leave the byref case of that function here, but you can make it local to this if-clause.

lib/CodeGen/CGDecl.cpp

1423 ↗

(On Diff #134430)

There's an idiom of copying __block variables of *block pointer* type in their own initializers, as a way of constructing a recursive block. I don't know of any reason why someone would intentionally do it for some random struct type, but that's always a risk when locking down on something retroactively. Anyway, I agree that we can experiment to see if locking down on aggregate __block captures causes any actual problems.

1421 ↗

(On Diff #127233)

Ping about this again.

lib/CodeGen/CGExprAgg.cpp

255 ↗

(On Diff #134430)

Okay. I'm just a little worried about what this implies about when we push destructor cleanups in general. I know it's rather ad-hoc for C++ destructors right now.

lib/CodeGen/CGNonTrivialStruct.cpp

193 ↗

(On Diff #134430)

Okay, great! I like the name.

Does this mean we're now copying all structs that contain volatile fields with one of these helper functions? If so, please add a C test case testing just that. Also, you should retitle this review and stress that we're changing how *all* non-trivial types are copied, and that that includes both volatile and ARC-qualified fields.

110 ↗

(On Diff #135309)

Please call these methods visitARCStrong to go with the new PCK name.

[ahatanak]

include/clang/AST/Type.h

1121 ↗

(On Diff #135309)

The second option seems most reasonable to me. We can always make changes if someone comes up with a type that requires tracking destructive moves separately.

lib/CodeGen/CGNonTrivialStruct.cpp

193 ↗

(On Diff #134430)

No, the current patch doesn't copy volatile fields of a struct individually unless the struct is a non-trivial type (which means its primitive copy kind is PCK_Struct). I'll look into today how I can force structs with volatile fields that are not non-trivial to be copied using the helper functions.

It seems like we would need a boolean flag in RecordDecl that tracks the presence of volatile fields in the struct or one of its subobjects. I assume we want to copy volatile fields individually in C++ too, in which case the flag needs to be set in both C and C++ mode. Is that right?

[rjmccall]

include/clang/AST/Type.h

1121 ↗

(On Diff #135309)

Well, we already have a type that would have a trivial deinitializing move: ARC __strong pointers. What we don't have is anything in IRGen that currently would take advantage of that fact. Anyway, I agree that we can wait to start tracking this until we add such code to the compiler.

lib/CodeGen/CGNonTrivialStruct.cpp

193 ↗

(On Diff #134430)

Oh, I see, there's specific logic to not flag the struct trivial if a field is merely VolatileTrivial. Okay, that works for me; we can think about the right thing to do in a follow-up commit. It probably needs to be raised as an RFC on cfe-dev.

The C++ standard has explicit language about triviality that says that volatile fields don't make defaulted copy-constructors non-trivial. Whatever we do here can't break that. But that doesn't necessarily imply that we should continue to use a simple memcpy to copy the struct; we could still use the C logic to copy "trivial" C++ structs with volatile members.

If we do try to apply this rule to C++, you will need to propagate these bits appropriately for C++ types, including from bases.

lib/Sema/SemaDecl.cpp

15424 ↗

(On Diff #135309)

Please compute isNonTrivialToPrimitiveCopy once here, it's not trivial.

...arguably we should figure out a way to do all these type checks once and then set the bits from that. We can do that as a separate patch, though.

[ahatanak]

Address review comments.

[ahatanak]

include/clang/AST/Type.h

1108 ↗

(On Diff #135309)

Thanks, I copied your comments verbatim except the comment on PCK_Struct: types that are PCK_Struct have a field that is neither PCK_Trivial nor PCK_VolatileTrivial. We can use the original comment once we start distinguishing PCK_Trivial structs that have volatile fields from those that don't.

lib/CodeGen/CGBlocks.cpp

1779 ↗

(On Diff #135309)

It's probably not important to assert here, so I removed the switch.

1792 ↗

(On Diff #135309)

I also simplified the QualType::DK_none case below.

lib/CodeGen/CGDecl.cpp

1421 ↗

(On Diff #127233)

Sorry I missed this one.

[rjmccall]

Thanks! This looks ready; thank you for your patience in working this out.

include/clang/AST/Type.h

1108 ↗

(On Diff #135309)

Yes, of course, that makes sense.

[sberg]

It looks like this caused a clang-cl regression https://bugs.llvm.org/show_bug.cgi?id=37146 "clang-cl emits special functions for non-trivial C-structs ('__destructor_8') introduced for Objective-C".

cfe/trunk/docs/LanguageExtensions.rst

 Use ``__has_feature(objc_instancetype)`` to determine whether the
 ``instancetype`` contextual keyword is available.
 
 Automatic reference counting
 ----------------------------
 
 Clang provides support for :doc:`automated reference counting
 <AutomaticReferenceCounting>` in Objective-C, which eliminates the need
-for manual ``retain``/``release``/``autorelease`` message sends.  There are two
+for manual ``retain``/``release``/``autorelease`` message sends.  There are three
 feature macros associated with automatic reference counting:
 ``__has_feature(objc_arc)`` indicates the availability of automated reference
 counting in general, while ``__has_feature(objc_arc_weak)`` indicates that
 automated reference counting also includes support for ``__weak`` pointers to
-Objective-C objects.
+Objective-C objects. ``__has_feature(objc_arc_fields)`` indicates that C structs
+are allowed to have fields that are pointers to Objective-C objects managed by
+automatic reference counting.
 
 .. _objc-fixed-enum:
 
 Enumerations with a fixed underlying type
 -----------------------------------------
 
 Clang provides support for C++11 enumerations with a fixed underlying type
 within Objective-C.  For example, one can write an enumeration type as:

cfe/trunk/include/clang/AST/Decl.h

class RecordDecl : public TagDecl {
   bool HasVolatileMember : 1;
 
   /// Whether the field declarations of this record have been loaded
   /// from external storage. To avoid unnecessary deserialization of
   /// methods/nested types we allow deserialization of just the fields
   /// when needed.
   mutable bool LoadedFieldsFromExternalStorage : 1;
 
+  /// Basic properties of non-trivial C structs.
+  bool NonTrivialToPrimitiveDefaultInitialize : 1;
+  bool NonTrivialToPrimitiveCopy : 1;
+  bool NonTrivialToPrimitiveDestroy : 1;
+
 protected:
   RecordDecl(Kind DK, TagKind TK, const ASTContext &C, DeclContext *DC,
              SourceLocation StartLoc, SourceLocation IdLoc,
              IdentifierInfo *Id, RecordDecl *PrevDecl);
 
 public:
   static RecordDecl *Create(const ASTContext &C, TagKind TK, DeclContext *DC,
                             SourceLocation StartLoc, SourceLocation IdLoc,
public:
 
   bool hasLoadedFieldsFromExternalStorage() const {
     return LoadedFieldsFromExternalStorage;
   }
   void setHasLoadedFieldsFromExternalStorage(bool val) {
     LoadedFieldsFromExternalStorage = val;
   }
 
+  /// Functions to query basic properties of non-trivial C structs.
+  bool isNonTrivialToPrimitiveDefaultInitialize() const {
+    return NonTrivialToPrimitiveDefaultInitialize;
+  }
+
+  void setNonTrivialToPrimitiveDefaultInitialize() {
+    NonTrivialToPrimitiveDefaultInitialize = true;
+  }
+
+  bool isNonTrivialToPrimitiveCopy() const {
+    return NonTrivialToPrimitiveCopy;
+  }
+
+  void setNonTrivialToPrimitiveCopy() {
+    NonTrivialToPrimitiveCopy = true;
+  }
+
+  bool isNonTrivialToPrimitiveDestroy() const {
+    return NonTrivialToPrimitiveDestroy;
+  }
+
+  void setNonTrivialToPrimitiveDestroy() {
+    NonTrivialToPrimitiveDestroy = true;
+  }
+
   /// \brief Determines whether this declaration represents the
   /// injected class name.
   ///
   /// The injected class name in C++ is the name of the class that
   /// appears inside the class itself. For example:
   ///
   /// \code
   /// struct C {

cfe/trunk/include/clang/AST/Type.h

public:
 
   bool hasStrongOrWeakObjCLifetime() const {
     return getQualifiers().hasStrongOrWeakObjCLifetime();
   }
 
   // true when Type is objc's weak and weak is enabled but ARC isn't.
   bool isNonWeakInMRRWithObjCWeak(const ASTContext &Context) const;
 
+  enum PrimitiveDefaultInitializeKind {
+    /// The type does not fall into any of the following categories. Note that
+    /// this case is zero-valued so that values of this enum can be used as a
+    /// boolean condition for non-triviality.
+    PDIK_Trivial,
+
+    /// The type is an Objective-C retainable pointer type that is qualified
+    /// with the ARC __strong qualifier.
+    PDIK_ARCStrong,
+
+    /// The type is a struct containing a field whose type is not PCK_Trivial.
+    PDIK_Struct
+  };
+
+  /// Functions to query basic properties of non-trivial C struct types.
+
+  /// Check if this is a non-trivial type that would cause a C struct
+  /// transitively containing this type to be non-trivial to default initialize
+  /// and return the kind.
+  PrimitiveDefaultInitializeKind
+  isNonTrivialToPrimitiveDefaultInitialize() const;
+
+  enum PrimitiveCopyKind {
+    /// The type does not fall into any of the following categories. Note that
+    /// this case is zero-valued so that values of this enum can be used as a
+    /// boolean condition for non-triviality.
+    PCK_Trivial,
+
+    /// The type would be trivial except that it is volatile-qualified. Types
+    /// that fall into one of the other non-trivial cases may additionally be
+    /// volatile-qualified.
+    PCK_VolatileTrivial,
+
+    /// The type is an Objective-C retainable pointer type that is qualified
+    /// with the ARC __strong qualifier.
+    PCK_ARCStrong,
+
+    /// The type is a struct containing a field whose type is neither
+    /// PCK_Trivial nor PCK_VolatileTrivial.
+    /// Note that a C++ struct type does not necessarily match this; C++ copying
+    /// semantics are too complex to express here, in part because they depend
+    /// on the exact constructor or assignment operator that is chosen by
+    /// overload resolution to do the copy.
+    PCK_Struct
+  };
+
+  /// Check if this is a non-trivial type that would cause a C struct
+  /// transitively containing this type to be non-trivial to copy and return the
+  /// kind.
+  PrimitiveCopyKind isNonTrivialToPrimitiveCopy() const;
+
+  /// Check if this is a non-trivial type that would cause a C struct
+  /// transitively containing this type to be non-trivial to destructively
+  /// move and return the kind. Destructive move in this context is a C++-style
+  /// move in which the source object is placed in a valid but unspecified state
+  /// after it is moved, as opposed to a truly destructive move in which the
+  /// source object is placed in an uninitialized state.
+  PrimitiveCopyKind isNonTrivialToPrimitiveDestructiveMove() const;
+
   enum DestructionKind {
     DK_none,
     DK_cxx_destructor,
     DK_objc_strong_lifetime,
-    DK_objc_weak_lifetime
+    DK_objc_weak_lifetime,
+    DK_nontrivial_c_struct
   };
 
   /// Returns a nonzero value if objects of this type require
   /// non-trivial work to clean up after.  Non-zero because it's
   /// conceivable that qualifiers (objc_gc(weak)?) could make
   /// something require destruction.
   DestructionKind isDestructedType() const {
     return isDestructedTypeImpl(*this);

cfe/trunk/include/clang/Basic/DiagnosticSemaKinds.td

 def note_protected_by_seh_finally : Note<
   "jump bypasses initialization of __finally block">;
 def note_protected_by___block : Note<
   "jump bypasses setup of __block variable">;
 def note_protected_by_objc_strong_init : Note<
   "jump bypasses initialization of __strong variable">;
 def note_protected_by_objc_weak_init : Note<
   "jump bypasses initialization of __weak variable">;
+def note_protected_by_non_trivial_c_struct_init : Note<
+  "jump bypasses initialization of variable of non-trivial C struct type">;
 def note_enters_block_captures_cxx_obj : Note<
   "jump enters lifetime of block which captures a destructible C++ object">;
 def note_enters_block_captures_strong : Note<
   "jump enters lifetime of block which strongly captures a variable">;
 def note_enters_block_captures_weak : Note<
   "jump enters lifetime of block which weakly captures a variable">;
+def note_enters_block_captures_non_trivial_c_struct : Note<
+  "jump enters lifetime of block which captures a C struct that is non-trivial "
+  "to destroy">;
 
 def note_exits_cleanup : Note<
   "jump exits scope of variable with __attribute__((cleanup))">;
 def note_exits_dtor : Note<
   "jump exits scope of variable with non-trivial destructor">;
 def note_exits_temporary_dtor : Note<
   "jump exits scope of lifetime-extended temporary with non-trivial "
   "destructor">;

 def note_exits_objc_weak : Note<
   "jump exits scope of __weak variable">;
 def note_exits_block_captures_cxx_obj : Note<
   "jump exits lifetime of block which captures a destructible C++ object">;
 def note_exits_block_captures_strong : Note<
   "jump exits lifetime of block which strongly captures a variable">;
 def note_exits_block_captures_weak : Note<
   "jump exits lifetime of block which weakly captures a variable">;
+def note_exits_block_captures_non_trivial_c_struct : Note<
+  "jump exits lifetime of block which captures a C struct that is non-trivial "
+  "to destroy">;
 
 def err_func_returning_qualified_void : ExtWarn<
   "function cannot return qualified void type %0">,
   InGroup<DiagGroup<"qualified-void-return-type">>;
 def err_func_returning_array_function : Error<
   "function cannot return %select{array|function}0 type %1">;
 def err_field_declared_as_function : Error<"field %0 declared as a function">;
 def err_field_incomplete : Error<"field has incomplete type %0">;
def warn_non_pod_vararg_with_format_string : Warning<
   "cannot pass %select{non-POD|non-trivial}0 object of type %1 to variadic "
   "%select{function|block|method|constructor}2; expected type from format "
   "string was %3">, InGroup<NonPODVarargs>, DefaultError;
 // The arguments to this diagnostic should match the warning above.
 def err_cannot_pass_objc_interface_to_vararg_format : Error<
   "cannot pass object with interface type %1 by value to variadic "
   "%select{function|block|method|constructor}2; expected type from format "
   "string was %3">;
+def err_cannot_pass_non_trivial_c_struct_to_vararg : Error<
+  "cannot pass non-trivial C object of type %0 by value to variadic "
+  "%select{function|block|method|constructor}1">;
+
 
 def err_cannot_pass_objc_interface_to_vararg : Error<
   "cannot pass object with interface type %0 by value through variadic "
   "%select{function|block|method|constructor}1">;
 def warn_cannot_pass_non_pod_arg_to_vararg : Warning<
   "cannot pass object of %select{non-POD|non-trivial}0 type %1 through variadic"
   " %select{function|block|method|constructor}2; call will abort at runtime">,
   InGroup<NonPODVarargs>, DefaultError;

cfe/trunk/lib/AST/ASTContext.cpp

assert(TypeLoc::getFullDataSizeForType(Updated) ==
                TypeLoc::getFullDataSizeForType(TSInfo->getType()) &&
            "TypeLoc size mismatch from updating exception specification");
     TSInfo->overrideType(Updated);
   }
 }
 
 bool ASTContext::isParamDestroyedInCallee(QualType T) const {
   return getTargetInfo().getCXXABI().areArgsDestroyedLeftToRightInCallee() ||
-         T.hasTrivialABIOverride();
+         T.hasTrivialABIOverride() ||
+         T.isDestructedType() == QualType::DK_nontrivial_c_struct;
 }
 
 /// getComplexType - Return the uniqued reference to the type for a complex
 /// number with the specified element type.
 QualType ASTContext::getComplexType(QualType T) const {
   // Unique pointers, to guarantee there is only one pointer of a particular
   // structure.
   llvm::FoldingSetNodeID ID;
bool ASTContext::BlockRequiresCopying(QualType Ty,
                                       const VarDecl *D) {
   if (const CXXRecordDecl *record = Ty->getAsCXXRecordDecl()) {
     const Expr *copyExpr = getBlockVarCopyInits(D);
     if (!copyExpr && record->hasTrivialDestructor()) return false;
     
     return true;
   }
   
+  // The block needs copy/destroy helpers if Ty is non-trivial to destructively
+  // move or destroy.
+  if (Ty.isNonTrivialToPrimitiveDestructiveMove() || Ty.isDestructedType())
+    return true;
+
   if (!Ty->isObjCRetainableType()) return false;
   
   Qualifiers qs = Ty.getQualifiers();
   
   // If we have lifetime, that dominates.
   if (Qualifiers::ObjCLifetime lifetime = qs.getObjCLifetime()) {
     switch (lifetime) {
       case Qualifiers::OCL_None: llvm_unreachable("impossible");
         
       // These are just bits as far as the runtime is concerned.
       case Qualifiers::OCL_ExplicitNone:
       case Qualifiers::OCL_Autoreleasing:
         return false;
-        
-      // Tell the runtime that this is ARC __weak, called by the
-      // byref routines.
+      // These cases should have been taken care of when checking the type's
+      // non-triviality.
       case Qualifiers::OCL_Weak:
-      // ARC __strong __block variables need to be retained.
       case Qualifiers::OCL_Strong:
-        return true;
+        llvm_unreachable("impossible");
     }
     llvm_unreachable("fell out of lifetime switch!");
   }
   return (Ty->isBlockPointerType() || isObjCNSObjectType(Ty) ||
           Ty->isObjCObjectPointerType());
 }
 
 bool ASTContext::getByrefLifetime(QualType Ty,

cfe/trunk/lib/AST/Decl.cpp

 
 RecordDecl::RecordDecl(Kind DK, TagKind TK, const ASTContext &C,
                        DeclContext *DC, SourceLocation StartLoc,
                        SourceLocation IdLoc, IdentifierInfo *Id,
                        RecordDecl *PrevDecl)
     : TagDecl(DK, TK, C, DC, IdLoc, Id, PrevDecl, StartLoc),
       HasFlexibleArrayMember(false), AnonymousStructOrUnion(false),
       HasObjectMember(false), HasVolatileMember(false),
-      LoadedFieldsFromExternalStorage(false) {
+      LoadedFieldsFromExternalStorage(false),
+      NonTrivialToPrimitiveDefaultInitialize(false),
+      NonTrivialToPrimitiveCopy(false), NonTrivialToPrimitiveDestroy(false) {
   assert(classof(static_cast<Decl*>(this)) && "Invalid Kind!");
 }
 
 RecordDecl *RecordDecl::Create(const ASTContext &C, TagKind TK, DeclContext *DC,
                                SourceLocation StartLoc, SourceLocation IdLoc,
                                IdentifierInfo *Id, RecordDecl* PrevDecl) {
   RecordDecl *R = new (C, DC) RecordDecl(Record, TK, C, DC,
                                          StartLoc, IdLoc, Id, PrevDecl);

cfe/trunk/lib/AST/Type.cpp

 }
 
 bool QualType::isNonWeakInMRRWithObjCWeak(const ASTContext &Context) const {
   return !Context.getLangOpts().ObjCAutoRefCount &&
          Context.getLangOpts().ObjCWeak &&
          getObjCLifetime() != Qualifiers::OCL_Weak;
 }
 
+QualType::PrimitiveDefaultInitializeKind
+QualType::isNonTrivialToPrimitiveDefaultInitialize() const {
+  if (const auto *RT =
+          getTypePtr()->getBaseElementTypeUnsafe()->getAs<RecordType>())
+    if (RT->getDecl()->isNonTrivialToPrimitiveDefaultInitialize())
+      return PDIK_Struct;
+
+  Qualifiers::ObjCLifetime Lifetime = getQualifiers().getObjCLifetime();
+  if (Lifetime == Qualifiers::OCL_Strong)
+    return PDIK_ARCStrong;
+
+  return PDIK_Trivial;
+}
+
+QualType::PrimitiveCopyKind QualType::isNonTrivialToPrimitiveCopy() const {
+  if (const auto *RT =
+          getTypePtr()->getBaseElementTypeUnsafe()->getAs<RecordType>())
+    if (RT->getDecl()->isNonTrivialToPrimitiveCopy())
+      return PCK_Struct;
+
+  Qualifiers Qs = getQualifiers();
+  if (Qs.getObjCLifetime() == Qualifiers::OCL_Strong)
+    return PCK_ARCStrong;
+
+  return Qs.hasVolatile() ? PCK_VolatileTrivial : PCK_Trivial;
+}
+
+QualType::PrimitiveCopyKind
+QualType::isNonTrivialToPrimitiveDestructiveMove() const {
+  return isNonTrivialToPrimitiveCopy();
+}
+
 bool Type::isLiteralType(const ASTContext &Ctx) const {
   if (isDependentType())
     return false;
 
   // C++1y [basic.types]p10:
   //   A type is a literal type if it is:
   //   -- cv void; or
   if (Ctx.getLangOpts().CPlusPlus14 && isVoidType())
case Qualifiers::OCL_Autoreleasing:
     break;
 
   case Qualifiers::OCL_Strong:
     return DK_objc_strong_lifetime;
   case Qualifiers::OCL_Weak:
     return DK_objc_weak_lifetime;
   }
 
-  /// Currently, the only destruction kind we recognize is C++ objects
-  /// with non-trivial destructors.
-  const CXXRecordDecl *record =
-    type->getBaseElementTypeUnsafe()->getAsCXXRecordDecl();
-  if (record && record->hasDefinition() && !record->hasTrivialDestructor())
+  if (const auto *RT =
+          type->getBaseElementTypeUnsafe()->getAs<RecordType>()) {
+    const RecordDecl *RD = RT->getDecl();
+    if (const auto *CXXRD = dyn_cast<CXXRecordDecl>(RD)) {
+      /// Check if this is a C++ object with a non-trivial destructor.
+      if (CXXRD->hasDefinition() && !CXXRD->hasTrivialDestructor())
         return DK_cxx_destructor;
+    } else {
+      /// Check if this is a C struct that is non-trivial to destroy or an array
+      /// that contains such a struct.
+      if (RD->isNonTrivialToPrimitiveDestroy())
+        return DK_nontrivial_c_struct;
+    }
+  }
 
   return DK_none;
 }
 
 CXXRecordDecl *MemberPointerType::getMostRecentCXXRecordDecl() const {
   return getClass()->getAsCXXRecordDecl()->getMostRecentDecl();
 }

cfe/trunk/lib/CodeGen/CGBlocks.cpp

// Block pointers require copy/dispose.  So do Objective-C pointers.
         lifetime = Qualifiers::OCL_Strong;
       }
 
     // So do types that require non-trivial copy construction.
     } else if (CI.hasCopyExpr()) {
       info.NeedsCopyDispose = true;
       info.HasCXXObject = true;
 
+    // So do C structs that require non-trivial copy construction or
+    // destruction.
+    } else if (variable->getType().isNonTrivialToPrimitiveCopy() ==
+                   QualType::PCK_Struct ||
+               variable->getType().isDestructedType() ==
+                   QualType::DK_nontrivial_c_struct) {
+      info.NeedsCopyDispose = true;
+
     // And so do types with destructors.
     } else if (CGM.getLangOpts().CPlusPlus) {
       if (const CXXRecordDecl *record =
             variable->getType()->getAsCXXRecordDecl()) {
         if (!record->hasTrivialDestructor()) {
           info.HasCXXObject = true;
           info.NeedsCopyDispose = true;
         }

 namespace {
 
 /// Represents a type of copy/destroy operation that should be performed for an
 /// entity that's captured by a block.
 enum class BlockCaptureEntityKind {
   CXXRecord, // Copy or destroy
   ARCWeak,
   ARCStrong,
+  NonTrivialCStruct,
   BlockObject, // Assign or release
   None
 };
 
 /// Represents a captured entity that requires extra operations in order for
 /// this entity to be copied or destroyed correctly.
 struct BlockCaptureManagedEntity {
   BlockCaptureEntityKind Kind;
computeCopyInfoForBlockCapture(const BlockDecl::Capture &CI, QualType T,
   }
   BlockFieldFlags Flags;
   if (CI.isByRef()) {
     Flags = BLOCK_FIELD_IS_BYREF;
     if (T.isObjCGCWeak())
       Flags |= BLOCK_FIELD_IS_WEAK;
     return std::make_pair(BlockCaptureEntityKind::BlockObject, Flags);
   }
-  if (!T->isObjCRetainableType())
-    // For all other types, the memcpy is fine.
-    return std::make_pair(BlockCaptureEntityKind::None, Flags);
 
   Flags = BLOCK_FIELD_IS_OBJECT;
   bool isBlockPointer = T->isBlockPointerType();
   if (isBlockPointer)
     Flags = BLOCK_FIELD_IS_BLOCK;
 
+  switch (T.isNonTrivialToPrimitiveCopy()) {
+  case QualType::PCK_Struct:
+    return std::make_pair(BlockCaptureEntityKind::NonTrivialCStruct,
+                          BlockFieldFlags());
+  case QualType::PCK_ARCStrong:
+    // We need to retain the copied value for __strong direct captures.
+    // If it's a block pointer, we have to copy the block and assign that to
+    // the destination pointer, so we might as well use _Block_object_assign.
+    // Otherwise we can avoid that.
+    return std::make_pair(!isBlockPointer ? BlockCaptureEntityKind::ARCStrong
+                                          : BlockCaptureEntityKind::BlockObject,
+                          Flags);
+  case QualType::PCK_Trivial:
+  case QualType::PCK_VolatileTrivial: {
+    if (!T->isObjCRetainableType())
+      // For all other types, the memcpy is fine.
+      return std::make_pair(BlockCaptureEntityKind::None, BlockFieldFlags());
+
     // Special rules for ARC captures:
     Qualifiers QS = T.getQualifiers();
 
     // We need to register __weak direct captures with the runtime.
     if (QS.getObjCLifetime() == Qualifiers::OCL_Weak)
       return std::make_pair(BlockCaptureEntityKind::ARCWeak, Flags);
 
-  // We need to retain the copied value for __strong direct captures.
-  if (QS.getObjCLifetime() == Qualifiers::OCL_Strong) {
-    // If it's a block pointer, we have to copy the block and
-    // assign that to the destination pointer, so we might as
-    // well use _Block_object_assign.  Otherwise we can avoid that.
-    return std::make_pair(!isBlockPointer ? BlockCaptureEntityKind::ARCStrong
-                                          : BlockCaptureEntityKind::BlockObject,
-                          Flags);
-  }
-
     // Non-ARC captures of retainable pointers are strong and
     // therefore require a call to _Block_object_assign.
     if (!QS.getObjCLifetime() && !LangOpts.ObjCAutoRefCount)
       return std::make_pair(BlockCaptureEntityKind::BlockObject, Flags);
 
     // Otherwise the memcpy is fine.
-  return std::make_pair(BlockCaptureEntityKind::None, Flags);
+    return std::make_pair(BlockCaptureEntityKind::None, BlockFieldFlags());
+  }
+  }
 }
 
 /// Find the set of block captures that need to be explicitly copied or destroy.
 static void findBlockCapturedManagedEntities(
     const CGBlockInfo &BlockInfo, const LangOptions &LangOpts,
     SmallVectorImpl<BlockCaptureManagedEntity> &ManagedCaptures,
     llvm::function_ref<std::pair<BlockCaptureEntityKind, BlockFieldFlags>(
         const BlockDecl::Capture &, QualType, const LangOptions &)>
for (const auto &CopiedCapture : CopiedCaptures) {
     Address dstField = Builder.CreateStructGEP(dst, index, capture.getOffset());
 
     // If there's an explicit copy expression, we do that.
     if (CI.getCopyExpr()) {
       assert(CopiedCapture.Kind == BlockCaptureEntityKind::CXXRecord);
       EmitSynthesizedCXXCopyCtor(dstField, srcField, CI.getCopyExpr());
     } else if (CopiedCapture.Kind == BlockCaptureEntityKind::ARCWeak) {
       EmitARCCopyWeak(dstField, srcField);
+    // If this is a C struct that requires non-trivial copy construction, emit a
+    // call to its copy constructor.
+    } else if (CopiedCapture.Kind ==
+               BlockCaptureEntityKind::NonTrivialCStruct) {
+      QualType varType = CI.getVariable()->getType();
+      callCStructCopyConstructor(MakeAddrLValue(dstField, varType),
+                                 MakeAddrLValue(srcField, varType));
     } else {
       llvm::Value *srcValue = Builder.CreateLoad(srcField, "blockcopy.src");
       if (CopiedCapture.Kind == BlockCaptureEntityKind::ARCStrong) {
         // At -O0, store null into the destination field (so that the
         // storeStrong doesn't over-release) and then call storeStrong.
         // This is a workaround to not having an initStrong call.
         if (CGM.getCodeGenOpts().OptimizationLevel == 0) {
           auto *ty = cast<llvm::PointerType>(srcValue->getType());
for (const auto &CopiedCapture : CopiedCaptures) {
     }
   }
 
   FinishFunction();
 
   return llvm::ConstantExpr::getBitCast(Fn, VoidPtrTy);
 }
 
+static BlockFieldFlags
+getBlockFieldFlagsForObjCObjectPointer(const BlockDecl::Capture &CI,
+                                       QualType T) {
+  BlockFieldFlags Flags = BLOCK_FIELD_IS_OBJECT;
+  if (T->isBlockPointerType())
+    Flags = BLOCK_FIELD_IS_BLOCK;
+  return Flags;
+}
+
 static std::pair<BlockCaptureEntityKind, BlockFieldFlags>
 computeDestroyInfoForBlockCapture(const BlockDecl::Capture &CI, QualType T,
                                   const LangOptions &LangOpts) {
-  BlockFieldFlags Flags;
   if (CI.isByRef()) {
-    Flags = BLOCK_FIELD_IS_BYREF;
+    BlockFieldFlags Flags = BLOCK_FIELD_IS_BYREF;
     if (T.isObjCGCWeak())
       Flags |= BLOCK_FIELD_IS_WEAK;
     return std::make_pair(BlockCaptureEntityKind::BlockObject, Flags);
   }
 
-  if (const CXXRecordDecl *Record = T->getAsCXXRecordDecl()) {
-    if (Record->hasTrivialDestructor())
-      return std::make_pair(BlockCaptureEntityKind::None, BlockFieldFlags());
+  switch (T.isDestructedType()) {
+  case QualType::DK_cxx_destructor:
     return std::make_pair(BlockCaptureEntityKind::CXXRecord, BlockFieldFlags());
-  }
-
-  // Other types don't need to be destroy explicitly.
-  if (!T->isObjCRetainableType())
-    return std::make_pair(BlockCaptureEntityKind::None, Flags);
-
-  Flags = BLOCK_FIELD_IS_OBJECT;
-  if (T->isBlockPointerType())
-    Flags = BLOCK_FIELD_IS_BLOCK;
-
-  // Special rules for ARC captures.
-  Qualifiers QS = T.getQualifiers();
-
+  case QualType::DK_objc_strong_lifetime:
     // Use objc_storeStrong for __strong direct captures; the
     // dynamic tools really like it when we do this.
-  if (QS.getObjCLifetime() == Qualifiers::OCL_Strong)
-    return std::make_pair(BlockCaptureEntityKind::ARCStrong, Flags);
-
+    return std::make_pair(BlockCaptureEntityKind::ARCStrong,
+                          getBlockFieldFlagsForObjCObjectPointer(CI, T));
+  case QualType::DK_objc_weak_lifetime:
     // Support __weak direct captures.
-  if (QS.getObjCLifetime() == Qualifiers::OCL_Weak)
-    return std::make_pair(BlockCaptureEntityKind::ARCWeak, Flags);
-
-  // Non-ARC captures are strong, and we need to use
-  // _Block_object_dispose.
-  if (!QS.hasObjCLifetime() && !LangOpts.ObjCAutoRefCount)
-    return std::make_pair(BlockCaptureEntityKind::BlockObject, Flags);
-
+    return std::make_pair(BlockCaptureEntityKind::ARCWeak,
+                          getBlockFieldFlagsForObjCObjectPointer(CI, T));
+  case QualType::DK_nontrivial_c_struct:
+    return std::make_pair(BlockCaptureEntityKind::NonTrivialCStruct,
+                          BlockFieldFlags());
+  case QualType::DK_none: {
+    // Non-ARC captures are strong, and we need to use _Block_object_dispose.
+    if (T->isObjCRetainableType() && !T.getQualifiers().hasObjCLifetime() &&
+        !LangOpts.ObjCAutoRefCount)
+      return std::make_pair(BlockCaptureEntityKind::BlockObject,
+                            getBlockFieldFlagsForObjCObjectPointer(CI, T));
     // Otherwise, we have nothing to do.
-  return std::make_pair(BlockCaptureEntityKind::None, Flags);
+    return std::make_pair(BlockCaptureEntityKind::None, BlockFieldFlags());
+  }
+  }
 }
 
 /// Generate the destroy-helper function for a block closure object:
 ///   static void block_destroy_helper(block_t *theBlock);
 ///
 /// Note that this destroys a heap-allocated block closure object;
 /// it should not be confused with a 'byref destroy helper', which
 /// destroys the heap-allocated contents of an individual __block
if (DestroyedCapture.Kind == BlockCaptureEntityKind::CXXRecord) {
       // If this is a __weak capture, emit the release directly.
     } else if (DestroyedCapture.Kind == BlockCaptureEntityKind::ARCWeak) {
       EmitARCDestroyWeak(srcField);
 
     // Destroy strong objects with a call if requested.
     } else if (DestroyedCapture.Kind == BlockCaptureEntityKind::ARCStrong) {
       EmitARCDestroyStrong(srcField, ARCImpreciseLifetime);
 
+    // If this is a C struct that requires non-trivial destruction, emit a call
+    // to its destructor.
+    } else if (DestroyedCapture.Kind ==
+               BlockCaptureEntityKind::NonTrivialCStruct) {
+      QualType varType = CI.getVariable()->getType();
+      pushDestroy(varType.isDestructedType(), srcField, varType);
+
     // Otherwise we call _Block_object_dispose.  It wouldn't be too
     // hard to just emit this as a cleanup if we wanted to make sure
     // that things were done in reverse.
     } else {
       assert(DestroyedCapture.Kind == BlockCaptureEntityKind::BlockObject);
       llvm::Value *value = Builder.CreateLoad(srcField);
       value = Builder.CreateBitCast(value, VoidPtrTy);
       BuildBlockRelease(value, flags);
void emitDispose(CodeGenFunction &CGF, Address field) override {
     CGF.PushDestructorCleanup(VarType, field);
     CGF.PopCleanupBlocks(cleanupDepth);
   }
 
   void profileImpl(llvm::FoldingSetNodeID &id) const override {
     id.AddPointer(VarType.getCanonicalType().getAsOpaquePtr());
   }
 };
+
+/// Emits the copy/dispose helpers for a __block variable that is a non-trivial
+/// C struct.
+class NonTrivialCStructByrefHelpers final : public BlockByrefHelpers {
+  QualType VarType;
+
+public:
+  NonTrivialCStructByrefHelpers(CharUnits alignment, QualType type)
+    : BlockByrefHelpers(alignment), VarType(type) {}
+
+  void emitCopy(CodeGenFunction &CGF, Address destField,
+                Address srcField) override {
+    CGF.callCStructMoveConstructor(CGF.MakeAddrLValue(destField, VarType),
+                                   CGF.MakeAddrLValue(srcField, VarType));
+  }
+
+  bool needsDispose() const override {
+    return VarType.isDestructedType();
+  }
+
+  void emitDispose(CodeGenFunction &CGF, Address field) override {
+    EHScopeStack::stable_iterator cleanupDepth = CGF.EHStack.stable_begin();
+    CGF.pushDestroy(VarType.isDestructedType(), field, VarType);
+    CGF.PopCleanupBlocks(cleanupDepth);
+  }
+
+  void profileImpl(llvm::FoldingSetNodeID &id) const override {
+    id.AddPointer(VarType.getCanonicalType().getAsOpaquePtr());
+  }
+};
 } // end anonymous namespace
 
 static llvm::Constant *
 generateByrefCopyHelper(CodeGenFunction &CGF, const BlockByrefInfo &byrefInfo,
                         BlockByrefHelpers &generator) {
   ASTContext &Context = CGF.getContext();
 
   QualType R = Context.VoidTy;
CodeGenFunction::buildByrefHelpers(llvm::StructType &byrefType,
   if (const CXXRecordDecl *record = type->getAsCXXRecordDecl()) {
     const Expr *copyExpr = CGM.getContext().getBlockVarCopyInits(&var);
     if (!copyExpr && record->hasTrivialDestructor()) return nullptr;
 
     return ::buildByrefHelpers(
         CGM, byrefInfo, CXXByrefHelpers(valueAlignment, type, copyExpr));
   }
 
+  // If type is a non-trivial C struct type that is non-trivial to
+  // destructly move or destroy, build the copy and dispose helpers.
+  if (type.isNonTrivialToPrimitiveDestructiveMove() == QualType::PCK_Struct ||
+      type.isDestructedType() == QualType::DK_nontrivial_c_struct)
+    return ::buildByrefHelpers(
+        CGM, byrefInfo, NonTrivialCStructByrefHelpers(valueAlignment, type));
+
   // Otherwise, if we don't have a retainable type, there's nothing to do.
   // that the runtime does extra copies.
   if (!type->isObjCRetainableType()) return nullptr;
 
   Qualifiers qs = type.getQualifiers();
 
   // If we have lifetime, that dominates.
   if (Qualifiers::ObjCLifetime lifetime = qs.getObjCLifetime()) {

cfe/trunk/lib/CodeGen/CGCall.cpp

 struct DestroyUnpassedArg final : EHScopeStack::Cleanup {
   DestroyUnpassedArg(Address Addr, QualType Ty)
       : Addr(Addr), Ty(Ty) {}
 
   Address Addr;
   QualType Ty;
 
   void Emit(CodeGenFunction &CGF, Flags flags) override {
+    QualType::DestructionKind DtorKind = Ty.isDestructedType();
+    if (DtorKind == QualType::DK_cxx_destructor) {
       const CXXDestructorDecl *Dtor = Ty->getAsCXXRecordDecl()->getDestructor();
       assert(!Dtor->isTrivial());
       CGF.EmitCXXDestructorCall(Dtor, Dtor_Complete, /*for vbase*/ false,
                                 /*Delegating=*/false, Addr);
+    } else {
+      CGF.callCStructDestructor(CGF.MakeAddrLValue(Addr, Ty));
+    }
   }
 };
 
 struct DisableDebugLocationUpdates {
   CodeGenFunction &CGF;
   bool disabledDebugInfo;
   DisableDebugLocationUpdates(CodeGenFunction &CGF, const Expr *E) : CGF(CGF) {
     if ((disabledDebugInfo = isa<CXXDefaultArgExpr>(E) && CGF.getDebugInfo()))
if (HasAggregateEvalKind && getContext().isParamDestroyedInCallee(type)) {
     // If we're using inalloca, use the argument memory.  Otherwise, use a
     // temporary.
     AggValueSlot Slot;
     if (args.isUsingInAlloca())
       Slot = createPlaceholderSlot(*this, type);
     else
       Slot = CreateAggTemp(type, "agg.tmp");
 
-    const CXXRecordDecl *RD = type->getAsCXXRecordDecl();
-    bool DestroyedInCallee =
+    bool DestroyedInCallee = true, NeedsEHCleanup = true;
+    if (const auto *RD = type->getAsCXXRecordDecl()) {
+      DestroyedInCallee =
           RD && RD->hasNonTrivialDestructor() &&
           (CGM.getCXXABI().getRecordArgABI(RD) != CGCXXABI::RAA_Default ||
            RD->hasTrivialABIOverride());
+    } else {
+      NeedsEHCleanup = needsEHCleanup(type.isDestructedType());
+    }
+
     if (DestroyedInCallee)
       Slot.setExternallyDestructed();
 
     EmitAggExpr(E, Slot);
     RValue RV = Slot.asRValue();
     args.add(RV, type);
 
-    if (DestroyedInCallee) {
+    if (DestroyedInCallee && NeedsEHCleanup) {
       // Create a no-op GEP between the placeholder and the cleanup so we can
       // RAUW it successfully.  It also serves as a marker of the first
       // instruction where the cleanup is active.
       pushFullExprCleanup<DestroyUnpassedArg>(EHCleanup, Slot.getAddress(),
                                               type);
       // This unreachable is a temporary marker which will be removed later.
       llvm::Instruction *IsActive = Builder.CreateUnreachable();
       args.addArgCleanupDeactivation(EHStack.getInnermostEHScope(), IsActive);

cfe/trunk/lib/CodeGen/CGDecl.cpp

if (!HaveInsertPoint()) {
     if (!Init || !ContainsLabel(Init)) return;
     EnsureInsertPoint();
   }
 
   // Initialize the structure of a __block variable.
   if (emission.IsByRef)
     emitByrefStructureInit(emission);
 
+  // Initialize the variable here if it doesn't have a initializer and it is a
+  // C struct that is non-trivial to initialize or an array containing such a
+  // struct.
+  if (!Init &&
+      type.isNonTrivialToPrimitiveDefaultInitialize() ==
+          QualType::PDIK_Struct) {
+    LValue Dst = MakeAddrLValue(emission.getAllocatedAddress(), type);
+    if (emission.IsByRef)
+      drillIntoBlockVariable(*this, Dst, &D);
+    defaultInitNonTrivialCStructVar(Dst);
+    return;
+  }
+
   if (isTrivialInitializer(Init))
     return;
 
   // Check whether this is a byref variable that's potentially
   // captured and moved by its own initializer.  If so, we'll need to
   // emit the initializer first, then copy into the variable.
   bool capturedByInit = emission.IsByRef && isCapturedBy(D, Init);
 
case QualType::DK_objc_strong_lifetime:
 
     // Use the imprecise destroyer by default.
     if (!var->hasAttr<ObjCPreciseLifetimeAttr>())
       destroyer = CodeGenFunction::destroyARCStrongImprecise;
     break;
 
   case QualType::DK_objc_weak_lifetime:
     break;
+
+  case QualType::DK_nontrivial_c_struct:
+    destroyer = CodeGenFunction::destroyNonTrivialCStruct;
+    break;
   }
 
   // If we haven't chosen a more specific destroyer, use the default.
   if (!destroyer) destroyer = getDestroyer(dtorKind);
 
   // Use an EH cleanup in array destructors iff the destructor itself
   // is being pushed as an EH cleanup.
   bool useEHCleanup = (cleanupKind & EHCleanup);
CodeGenFunction::getDestroyer(QualType::DestructionKind kind) {
   switch (kind) {
   case QualType::DK_none: llvm_unreachable("no destroyer for trivial dtor");
   case QualType::DK_cxx_destructor:
     return destroyCXXObject;
   case QualType::DK_objc_strong_lifetime:
     return destroyARCStrongPrecise;
   case QualType::DK_objc_weak_lifetime:
     return destroyARCWeak;
+  case QualType::DK_nontrivial_c_struct:
+    return destroyNonTrivialCStruct;
   }
   llvm_unreachable("Unknown DestructionKind");
 }
 
 /// pushEHDestroy - Push the standard destructor for the given type as
 /// an EH-only cleanup.
 void CodeGenFunction::pushEHDestroy(QualType::DestructionKind dtorKind,
                                     Address addr, QualType type) {
if (Arg.isIndirect()) {
     if (DeclPtr.getType() != IRTy)
       DeclPtr = Builder.CreateBitCast(DeclPtr, IRTy, D.getName());
 
     // Push a destructor cleanup for this parameter if the ABI requires it.
     // Don't push a cleanup in a thunk for a method that will also emit a
     // cleanup.
     if (!IsScalar && !CurFuncIsThunk &&
         getContext().isParamDestroyedInCallee(Ty)) {
-      const CXXRecordDecl *RD = Ty->getAsCXXRecordDecl();
-      if (RD && RD->hasNonTrivialDestructor())
-        pushDestroy(QualType::DK_cxx_destructor, DeclPtr, Ty);
+      if (QualType::DestructionKind DtorKind = Ty.isDestructedType()) {
+        assert((DtorKind == QualType::DK_cxx_destructor ||
+                DtorKind == QualType::DK_nontrivial_c_struct) &&
+               "unexpected destructor type");
+        pushDestroy(DtorKind, DeclPtr, Ty);
+      }
     }
   } else {
     // Otherwise, create a temporary to hold the value.
     DeclPtr = CreateMemTemp(Ty, getContext().getDeclAlign(&D),
                             D.getName() + ".addr");
     DoStore = true;
   }
 

cfe/trunk/lib/CodeGen/CGDeclCXX.cpp

static void EmitDeclDestroy(CodeGenFunction &CGF, const VarDecl &D,
   case QualType::DK_none:
     return;
 
   case QualType::DK_cxx_destructor:
     break;
 
   case QualType::DK_objc_strong_lifetime:
   case QualType::DK_objc_weak_lifetime:
+  case QualType::DK_nontrivial_c_struct:
     // We don't care about releasing objects during process teardown.
     assert(!D.getTLSKind() && "should have rejected this");
     return;
   }
 
   llvm::Constant *function;
   llvm::Constant *argument;
 

cfe/trunk/lib/CodeGen/CGExprAgg.cpp

public:
   //                               Utilities
   //===--------------------------------------------------------------------===//
 
   /// EmitAggLoadOfLValue - Given an expression with aggregate type that
   /// represents a value lvalue, this method emits the address of the lvalue,
   /// then loads the result into DestPtr.
   void EmitAggLoadOfLValue(const Expr *E);
 
+  enum ExprValueKind {
+    EVK_RValue,
+    EVK_NonRValue
+  };
+
   /// EmitFinalDestCopy - Perform the final copy to DestPtr, if desired.
-  void EmitFinalDestCopy(QualType type, const LValue &src);
+  /// SrcIsRValue is true if source comes from an RValue.
+  void EmitFinalDestCopy(QualType type, const LValue &src,
+                         ExprValueKind SrcValueKind = EVK_NonRValue);
   void EmitFinalDestCopy(QualType type, RValue src);
   void EmitCopy(QualType type, const AggValueSlot &dest,
                 const AggValueSlot &src);
 
   void EmitMoveFromReturnSlot(const Expr *E, RValue Src);
 
   void EmitArrayInit(Address DestPtr, llvm::ArrayType *AType,
                      QualType ArrayQTy, InitListExpr *E);

 /// The idea is that you do something like this:
 ///   RValue Result = EmitSomething(..., getReturnValueSlot());
 ///   EmitMoveFromReturnSlot(E, Result);
 ///
 /// If nothing interferes, this will cause the result to be emitted
 /// directly into the return value slot.  Otherwise, a final move
 /// will be performed.
 void AggExprEmitter::EmitMoveFromReturnSlot(const Expr *E, RValue src) {
+  // Push destructor if the result is ignored and the type is a C struct that
+  // is non-trivial to destroy.
+  QualType Ty = E->getType();
+  if (Dest.isIgnored() &&
+      Ty.isDestructedType() == QualType::DK_nontrivial_c_struct)
+    CGF.pushDestroy(Ty.isDestructedType(), src.getAggregateAddress(), Ty);
+
   if (shouldUseDestForReturnSlot()) {
     // Logically, Dest.getAddr() should equal Src.getAggregateAddr().
     // The possibility of undef rvalues complicates that a lot,
     // though, so we can't really assert.
     return;
   }
 
   // Otherwise, copy from there to the destination.
   assert(Dest.getPointer() != src.getAggregatePointer());
   EmitFinalDestCopy(E->getType(), src);
 }
 
 /// EmitFinalDestCopy - Perform the final copy to DestPtr, if desired.
 void AggExprEmitter::EmitFinalDestCopy(QualType type, RValue src) {
   assert(src.isAggregate() && "value must be aggregate value!");
   LValue srcLV = CGF.MakeAddrLValue(src.getAggregateAddress(), type);
-  EmitFinalDestCopy(type, srcLV);
+  EmitFinalDestCopy(type, srcLV, EVK_RValue);
 }
 
 /// EmitFinalDestCopy - Perform the final copy to DestPtr, if desired.
-void AggExprEmitter::EmitFinalDestCopy(QualType type, const LValue &src) {
+void AggExprEmitter::EmitFinalDestCopy(QualType type, const LValue &src,
+                                       ExprValueKind SrcValueKind) {
   // If Dest is ignored, then we're evaluating an aggregate expression
   // in a context that doesn't care about the result.  Note that loads
   // from volatile l-values force the existence of a non-ignored
   // destination.
   if (Dest.isIgnored())
     return;
 
+  // Copy non-trivial C structs here.
+  LValue DstLV = CGF.MakeAddrLValue(
+      Dest.getAddress(), Dest.isVolatile() ? type.withVolatile() : type);
+
+  if (SrcValueKind == EVK_RValue) {
+    if (type.isNonTrivialToPrimitiveDestructiveMove() == QualType::PCK_Struct) {
+      if (Dest.isPotentiallyAliased())
+        CGF.callCStructMoveAssignmentOperator(DstLV, src);
+      else
+        CGF.callCStructMoveConstructor(DstLV, src);
+      return;
+    }
+  } else {
+    if (type.isNonTrivialToPrimitiveCopy() == QualType::PCK_Struct) {
+      if (Dest.isPotentiallyAliased())
+        CGF.callCStructCopyAssignmentOperator(DstLV, src);
+      else
+        CGF.callCStructCopyConstructor(DstLV, src);
+      return;
+    }
+  }
+
   AggValueSlot srcAgg =
     AggValueSlot::forLValue(src, AggValueSlot::IsDestructed,
                             needsGC(type), AggValueSlot::IsAliased);
   EmitCopy(type, Dest, srcAgg);
 }
 
 /// Perform a copy from the source into the destination.
 ///

cfe/trunk/lib/CodeGen/CGNonTrivialStruct.cpp

+//===--- CGNonTrivialStruct.cpp - Emit Special Functions for C Structs ----===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file defines functions to generate various special functions for C
+// structs.
+//
+//===----------------------------------------------------------------------===//
+
+#include "CodeGenFunction.h"
+#include "CodeGenModule.h"
+#include "llvm/Support/ScopedPrinter.h"
+#include <array>
+
+using namespace clang;
+using namespace CodeGen;
+
+// Return the size of a field in number of bits.
+static uint64_t getFieldSize(const FieldDecl *FD, ASTContext &Ctx) {
+  if (FD->isBitField())
+    return FD->getBitWidthValue(Ctx);
+  return Ctx.getTypeSize(FD->getType());
+}
+
+namespace {
+enum { DstIdx = 0, SrcIdx = 1 };
+const char *ValNameStr[2] = {"dst", "src"};
+
+template <class Derived, class RetTy = void> struct DestructedTypeVisitor {
+  template <class... Ts> RetTy visit(QualType FT, Ts &&... Args) {
+    return asDerived().visit(FT.isDestructedType(), FT,
+                             std::forward<Ts>(Args)...);
+  }
+
+  template <class... Ts>
+  RetTy visit(QualType::DestructionKind DK, QualType FT, Ts &&... Args) {
+    if (asDerived().getContext().getAsArrayType(FT))
+      return asDerived().visitArray(DK, FT, std::forward<Ts>(Args)...);
+
+    switch (DK) {
+    case QualType::DK_objc_strong_lifetime:
+      return asDerived().visitARCStrong(FT, std::forward<Ts>(Args)...);
+    case QualType::DK_nontrivial_c_struct:
+      return asDerived().visitStruct(FT, std::forward<Ts>(Args)...);
+    case QualType::DK_none:
+      return asDerived().visitTrivial(FT, std::forward<Ts>(Args)...);
+    case QualType::DK_cxx_destructor:
+      return asDerived().visitCXXDestructor(FT, std::forward<Ts>(Args)...);
+    case QualType::DK_objc_weak_lifetime:
+      return asDerived().visitARCWeak(FT, std::forward<Ts>(Args)...);
+    }
+
+    llvm_unreachable("unknown destruction kind");
+  }
+
+  Derived &asDerived() { return static_cast<Derived &>(*this); }
+};
+
+template <class Derived, class RetTy = void>
+struct DefaultInitializedTypeVisitor {
+  template <class... Ts> RetTy visit(QualType FT, Ts &&... Args) {
+    return asDerived().visit(FT.isNonTrivialToPrimitiveDefaultInitialize(), FT,
+                             std::forward<Ts>(Args)...);
+  }
+
+  template <class... Ts>
+  RetTy visit(QualType::PrimitiveDefaultInitializeKind PDIK, QualType FT,
+              Ts &&... Args) {
+    if (asDerived().getContext().getAsArrayType(FT))
+      return asDerived().visitArray(PDIK, FT, std::forward<Ts>(Args)...);
+
+    switch (PDIK) {
+    case QualType::PDIK_ARCStrong:
+      return asDerived().visitARCStrong(FT, std::forward<Ts>(Args)...);
+    case QualType::PDIK_Struct:
+      return asDerived().visitStruct(FT, std::forward<Ts>(Args)...);
+    case QualType::PDIK_Trivial:
+      return asDerived().visitTrivial(FT, std::forward<Ts>(Args)...);
+    }
+
+    llvm_unreachable("unknown default-initialize kind");
+  }
+
+  Derived &asDerived() { return static_cast<Derived &>(*this); }
+};
+
+template <class Derived, bool IsMove, class RetTy = void>
+struct CopiedTypeVisitor {
+  template <class... Ts> RetTy visit(QualType FT, Ts &&... Args) {
+    QualType::PrimitiveCopyKind PCK =
+        IsMove ? FT.isNonTrivialToPrimitiveDestructiveMove()
+               : FT.isNonTrivialToPrimitiveCopy();
+    return asDerived().visit(PCK, FT, std::forward<Ts>(Args)...);
+  }
+
+  template <class... Ts>
+  RetTy visit(QualType::PrimitiveCopyKind PCK, QualType FT, Ts &&... Args) {
+    asDerived().preVisit(PCK, FT, std::forward<Ts>(Args)...);
+
+    if (asDerived().getContext().getAsArrayType(FT))
+      return asDerived().visitArray(PCK, FT, std::forward<Ts>(Args)...);
+
+    switch (PCK) {
+    case QualType::PCK_ARCStrong:
+      return asDerived().visitARCStrong(FT, std::forward<Ts>(Args)...);
+    case QualType::PCK_Struct:
+      return asDerived().visitStruct(FT, std::forward<Ts>(Args)...);
+    case QualType::PCK_Trivial:
+      return asDerived().visitTrivial(FT, std::forward<Ts>(Args)...);
+    case QualType::PCK_VolatileTrivial:
+      return asDerived().visitVolatileTrivial(FT, std::forward<Ts>(Args)...);
+    }
+
+    llvm_unreachable("unknown primitive copy kind");
+  }
+
+  Derived &asDerived() { return static_cast<Derived &>(*this); }
+};
+
+template <class Derived> struct StructVisitor {
+  StructVisitor(ASTContext &Ctx) : Ctx(Ctx) {}
+
+  template <class... Ts>
+  void visitStructFields(QualType QT, CharUnits CurStructOffset, Ts... Args) {
+    const RecordDecl *RD = QT->castAs<RecordType>()->getDecl();
+
+    // Iterate over the fields of the struct.
+    for (const FieldDecl *FD : RD->fields()) {
+      QualType FT = FD->getType();
+      FT = QT.isVolatileQualified() ? FT.withVolatile() : FT;
+      asDerived().visit(FT, FD, CurStructOffset, Args...);
+    }
+
+    asDerived().flushTrivialFields(Args...);
+  }
+
+  template <class... Ts> void visitTrivial(Ts... Args) {}
+
+  template <class... Ts> void visitARCWeak(Ts... Args) {
+    // FIXME: remove this when visitARCWeak is implemented in the subclasses.
+    llvm_unreachable("weak field is not expected");
+  }
+
+  template <class... Ts> void visitCXXDestructor(Ts... Args) {
+    llvm_unreachable("field of a C++ struct type is not expected");
+  }
+
+  template <class... Ts> void flushTrivialFields(Ts... Args) {}
+
+  uint64_t getFieldOffsetInBits(const FieldDecl *FD) {
+    return FD ? Ctx.getASTRecordLayout(FD->getParent())
+                    .getFieldOffset(FD->getFieldIndex())
+              : 0;
+  }
+
+  CharUnits getFieldOffset(const FieldDecl *FD) {
+    return Ctx.toCharUnitsFromBits(getFieldOffsetInBits(FD));
+  }
+
+  Derived &asDerived() { return static_cast<Derived &>(*this); }
+
+  ASTContext &getContext() { return Ctx; }
+  ASTContext &Ctx;
+};
+
+template <class Derived, bool IsMove>
+struct CopyStructVisitor : StructVisitor<Derived>,
+                           CopiedTypeVisitor<Derived, IsMove> {
+  using StructVisitor<Derived>::asDerived;
+
+  CopyStructVisitor(ASTContext &Ctx) : StructVisitor<Derived>(Ctx) {}
+
+  template <class... Ts>
+  void preVisit(QualType::PrimitiveCopyKind PCK, QualType FT,
+                const FieldDecl *FD, CharUnits CurStructOffsset,
+                Ts &&... Args) {
+    if (PCK)
+      asDerived().flushTrivialFields(std::forward<Ts>(Args)...);
+  }
+
+  template <class... Ts>
+  void visitTrivial(QualType FT, const FieldDecl *FD, CharUnits CurStructOffset,
+                    Ts... Args) {
+    assert(!FT.isVolatileQualified() && "volatile field not expected");
+    ASTContext &Ctx = asDerived().getContext();
+    uint64_t FieldSize = getFieldSize(FD, Ctx);
+
+    // Ignore zero-sized fields.
+    if (FieldSize == 0)
+      return;
+
+    uint64_t FStartInBits = asDerived().getFieldOffsetInBits(FD);
+    uint64_t FEndInBits = FStartInBits + FieldSize;
+    uint64_t RoundedFEnd = llvm::alignTo(FEndInBits, Ctx.getCharWidth());
+
+    // Set Start if this is the first field of a sequence of trivial fields.
+    if (Start == End)
+      Start = CurStructOffset + Ctx.toCharUnitsFromBits(FStartInBits);
+    End = CurStructOffset + Ctx.toCharUnitsFromBits(RoundedFEnd);
+  }
+
+  CharUnits Start = CharUnits::Zero(), End = CharUnits::Zero();
+};
+
+// This function creates the mangled name of a special function of a non-trivial
+// C struct. Since there is no ODR in C, the function is mangled based on the
+// struct contents and not the name. The mangled name has the following
+// structure:
+//
+// <function-name> ::= <prefix> <alignment-info> "_" <struct-field-info>
+// <prefix> ::= "__destructor_" | "__default_constructor_" |
+//              "__copy_constructor_" | "__move_constructor_" |
+//              "__copy_assignment_" | "__move_assignment_"
+// <alignment-info> ::= <dst-alignment> ["_" <src-alignment>]
+// <struct-field-info> ::= <field-info>+
+// <field-info> ::= <struct-or-scalar-field-info> | <array-field-info>
+// <struct-or-scalar-field-info> ::= <struct-field-info> | <strong-field-info> |
+//                                   <trivial-field-info>
+// <array-field-info> ::= "_AB" <array-offset> "s" <element-size> "n"
+//                        <num-elements> <innermost-element-info> "_AE"
+// <innermost-element-info> ::= <struct-or-scalar-field-info>
+// <strong-field-info> ::= "_s" ["b"] ["v"] <field-offset>
+// <trivial-field-info> ::= "_t" ["v"] <field-offset> "_" <field-size>
+
+template <class Derived> struct GenFuncNameBase {
+  std::string getVolatileOffsetStr(bool IsVolatile, CharUnits Offset) {
+    std::string S;
+    if (IsVolatile)
+      S = "v";
+    S += llvm::to_string(Offset.getQuantity());
+    return S;
+  }
+
+  void visitARCStrong(QualType FT, const FieldDecl *FD,
+                      CharUnits CurStructOffset) {
+    appendStr("_s");
+    if (FT->isBlockPointerType())
+      appendStr("b");
+    CharUnits FieldOffset = CurStructOffset + asDerived().getFieldOffset(FD);
+    appendStr(getVolatileOffsetStr(FT.isVolatileQualified(), FieldOffset));
+  }
+
+  void visitStruct(QualType QT, const FieldDecl *FD,
+                   CharUnits CurStructOffset) {
+    CharUnits FieldOffset = CurStructOffset + asDerived().getFieldOffset(FD);
+    asDerived().visitStructFields(QT, FieldOffset);
+  }
+
+  template <class FieldKind>
+  void visitArray(FieldKind FK, QualType QT, const FieldDecl *FD,
+                  CharUnits CurStructOffset) {
+    // String for non-volatile trivial fields is emitted when
+    // flushTrivialFields is called.
+    if (!FK)
+      return asDerived().visitTrivial(QT, FD, CurStructOffset);
+
+    CharUnits FieldOffset = CurStructOffset + asDerived().getFieldOffset(FD);
+    ASTContext &Ctx = asDerived().getContext();
+    const auto *AT = Ctx.getAsConstantArrayType(QT);
+    unsigned NumElts = Ctx.getConstantArrayElementCount(AT);
+    QualType EltTy = Ctx.getBaseElementType(AT);
+    CharUnits EltSize = Ctx.getTypeSizeInChars(EltTy);
+    appendStr("_AB" + llvm::to_string(FieldOffset.getQuantity()) + "s" +
+              llvm::to_string(EltSize.getQuantity()) + "n" +
+              llvm::to_string(NumElts));
+    EltTy = QT.isVolatileQualified() ? EltTy.withVolatile() : EltTy;
+    asDerived().visit(FK, EltTy, nullptr, FieldOffset);
+    appendStr("_AE");
+  }
+
+  void appendStr(StringRef Str) { Name += Str; }
+
+  std::string getName(QualType QT, bool IsVolatile) {
+    QT = IsVolatile ? QT.withVolatile() : QT;
+    asDerived().visitStructFields(QT, CharUnits::Zero());
+    return Name;
+  }
+
+  Derived &asDerived() { return static_cast<Derived &>(*this); }
+
+  std::string Name;
+};
+
+template <class Derived>
+struct GenUnaryFuncName : StructVisitor<Derived>, GenFuncNameBase<Derived> {
+  GenUnaryFuncName(StringRef Prefix, CharUnits DstAlignment, ASTContext &Ctx)
+      : StructVisitor<Derived>(Ctx) {
+    this->appendStr(Prefix);
+    this->appendStr(llvm::to_string(DstAlignment.getQuantity()));
+  }
+};
+
+// Helper function to create a null constant.
+static llvm::Constant *getNullForVariable(Address Addr) {
+  llvm::Type *Ty = Addr.getElementType();
+  return llvm::ConstantPointerNull::get(cast<llvm::PointerType>(Ty));
+}
+
+template <bool IsMove>
+struct GenBinaryFuncName : CopyStructVisitor<GenBinaryFuncName<IsMove>, IsMove>,
+                           GenFuncNameBase<GenBinaryFuncName<IsMove>> {
+
+  GenBinaryFuncName(StringRef Prefix, CharUnits DstAlignment,
+                    CharUnits SrcAlignment, ASTContext &Ctx)
+      : CopyStructVisitor<GenBinaryFuncName<IsMove>, IsMove>(Ctx) {
+    this->appendStr(Prefix);
+    this->appendStr(llvm::to_string(DstAlignment.getQuantity()));
+    this->appendStr("_" + llvm::to_string(SrcAlignment.getQuantity()));
+  }
+
+  void flushTrivialFields() {
+    if (this->Start == this->End)
+      return;
+
+    this->appendStr("_t" + llvm::to_string(this->Start.getQuantity()) + "w" +
+                    llvm::to_string((this->End - this->Start).getQuantity()));
+
+    this->Start = this->End = CharUnits::Zero();
+  }
+
+  void visitVolatileTrivial(QualType FT, const FieldDecl *FD,
+                            CharUnits CurStackOffset) {
+    // Because volatile fields can be bit-fields and are individually copied,
+    // their offset and width are in bits.
+    uint64_t OffsetInBits =
+        this->Ctx.toBits(CurStackOffset) + this->getFieldOffsetInBits(FD);
+    this->appendStr("_tv" + llvm::to_string(OffsetInBits) + "w" +
+                    llvm::to_string(getFieldSize(FD, this->Ctx)));
+  }
+};
+
+struct GenDefaultInitializeFuncName
+    : GenUnaryFuncName<GenDefaultInitializeFuncName>,
+      DefaultInitializedTypeVisitor<GenDefaultInitializeFuncName> {
+  GenDefaultInitializeFuncName(CharUnits DstAlignment, ASTContext &Ctx)
+      : GenUnaryFuncName<GenDefaultInitializeFuncName>("__default_constructor_",
+                                                       DstAlignment, Ctx) {}
+};
+
+struct GenDestructorFuncName : GenUnaryFuncName<GenDestructorFuncName>,
+                               DestructedTypeVisitor<GenDestructorFuncName> {
+  GenDestructorFuncName(CharUnits DstAlignment, ASTContext &Ctx)
+      : GenUnaryFuncName<GenDestructorFuncName>("__destructor_", DstAlignment,
+                                                Ctx) {}
+};
+
+// Helper function that creates CGFunctionInfo for an N-ary special function.
+template <size_t N>
+static const CGFunctionInfo &getFunctionInfo(CodeGenModule &CGM,
+                                             FunctionArgList &Args) {
+  ASTContext &Ctx = CGM.getContext();
+  llvm::SmallVector<ImplicitParamDecl *, N> Params;
+  QualType ParamTy = Ctx.getPointerType(Ctx.VoidPtrTy);
+
+  for (unsigned I = 0; I < N; ++I)
+    Params.push_back(ImplicitParamDecl::Create(
+        Ctx, nullptr, SourceLocation(), &Ctx.Idents.get(ValNameStr[I]), ParamTy,
+        ImplicitParamDecl::Other));
+
+  for (auto &P : Params)
+    Args.push_back(P);
+
+  return CGM.getTypes().arrangeBuiltinFunctionDeclaration(Ctx.VoidTy, Args);
+}
+
+// Template classes that are used as bases for classes that emit special
+// functions.
+template <class Derived> struct GenFuncBase {
+  template <size_t N>
+  void visitStruct(QualType FT, const FieldDecl *FD, CharUnits CurStackOffset,
+                   std::array<Address, N> Addrs) {
+    this->asDerived().callSpecialFunction(
+        FT, CurStackOffset + asDerived().getFieldOffset(FD), Addrs);
+  }
+
+  template <class FieldKind, size_t N>
+  void visitArray(FieldKind FK, QualType QT, const FieldDecl *FD,
+                  CharUnits CurStackOffset, std::array<Address, N> Addrs) {
+    // Non-volatile trivial fields are copied when flushTrivialFields is called.
+    if (!FK)
+      return asDerived().visitTrivial(QT, FD, CurStackOffset, Addrs);
+
+    CodeGenFunction &CGF = *this->CGF;
+    ASTContext &Ctx = CGF.getContext();
+
+    // Compute the end address.
+    QualType BaseEltQT;
+    std::array<Address, N> StartAddrs = Addrs;
+    for (unsigned I = 0; I < N; ++I)
+      StartAddrs[I] = getAddrWithOffset(Addrs[I], CurStackOffset, FD);
+    Address DstAddr = StartAddrs[DstIdx];
+    llvm::Value *NumElts =
+        CGF.emitArrayLength(Ctx.getAsArrayType(QT), BaseEltQT, DstAddr);
+    unsigned BaseEltSize = Ctx.getTypeSizeInChars(BaseEltQT).getQuantity();
+    llvm::Value *BaseEltSizeVal =
+        llvm::ConstantInt::get(NumElts->getType(), BaseEltSize);
+    llvm::Value *SizeInBytes =
+        CGF.Builder.CreateNUWMul(BaseEltSizeVal, NumElts);
+    Address BC = CGF.Builder.CreateBitCast(DstAddr, CGF.CGM.Int8PtrTy);
+    llvm::Value *DstArrayEnd =
+        CGF.Builder.CreateInBoundsGEP(BC.getPointer(), SizeInBytes);
+    DstArrayEnd = CGF.Builder.CreateBitCast(DstArrayEnd, CGF.CGM.Int8PtrPtrTy,
+                                            "dstarray.end");
+    llvm::BasicBlock *PreheaderBB = CGF.Builder.GetInsertBlock();
+
+    // Create the header block and insert the phi instructions.
+    llvm::BasicBlock *HeaderBB = CGF.createBasicBlock("loop.header");
+    CGF.EmitBlock(HeaderBB);
+    llvm::PHINode *PHIs[N];
+
+    for (unsigned I = 0; I < N; ++I) {
+      PHIs[I] = CGF.Builder.CreatePHI(CGF.CGM.Int8PtrPtrTy, 2, "addr.cur");
+      PHIs[I]->addIncoming(StartAddrs[I].getPointer(), PreheaderBB);
+    }
+
+    // Create the exit and loop body blocks.
+    llvm::BasicBlock *ExitBB = CGF.createBasicBlock("loop.exit");
+    llvm::BasicBlock *LoopBB = CGF.createBasicBlock("loop.body");
+
+    // Emit the comparison and conditional branch instruction that jumps to
+    // either the exit or the loop body.
+    llvm::Value *Done =
+        CGF.Builder.CreateICmpEQ(PHIs[DstIdx], DstArrayEnd, "done");
+    CGF.Builder.CreateCondBr(Done, ExitBB, LoopBB);
+
+    // Visit the element of the array in the loop body.
+    CGF.EmitBlock(LoopBB);
+    QualType EltQT = Ctx.getAsArrayType(QT)->getElementType();
+    CharUnits EltSize = Ctx.getTypeSizeInChars(EltQT);
+    std::array<Address, N> NewAddrs = Addrs;
+
+    for (unsigned I = 0; I < N; ++I)
+      NewAddrs[I] = Address(
+          PHIs[I], StartAddrs[I].getAlignment().alignmentAtOffset(EltSize));
+
+    EltQT = QT.isVolatileQualified() ? EltQT.withVolatile() : EltQT;
+    this->asDerived().visit(EltQT, nullptr, CharUnits::Zero(), NewAddrs);
+
+    LoopBB = CGF.Builder.GetInsertBlock();
+
+    for (unsigned I = 0; I < N; ++I) {
+      // Instrs to update the destination and source addresses.
+      // Update phi instructions.
+      NewAddrs[I] = getAddrWithOffset(NewAddrs[I], EltSize);
+      PHIs[I]->addIncoming(NewAddrs[I].getPointer(), LoopBB);
+    }
+
+    // Insert an unconditional branch to the header block.
+    CGF.Builder.CreateBr(HeaderBB);
+    CGF.EmitBlock(ExitBB);
+  }
+
+  /// Return an address with the specified offset from the passed address.
+  Address getAddrWithOffset(Address Addr, CharUnits Offset) {
+    assert(Addr.isValid() && "invalid address");
+    if (Offset.getQuantity() == 0)
+      return Addr;
+    Addr = CGF->Builder.CreateBitCast(Addr, CGF->CGM.Int8PtrTy);
+    Addr = CGF->Builder.CreateConstInBoundsGEP(Addr, Offset.getQuantity(),
+                                               CharUnits::One());
+    return CGF->Builder.CreateBitCast(Addr, CGF->CGM.Int8PtrPtrTy);
+  }
+
+  Address getAddrWithOffset(Address Addr, CharUnits StructFieldOffset,
+                            const FieldDecl *FD) {
+    return getAddrWithOffset(Addr, StructFieldOffset +
+                                       asDerived().getFieldOffset(FD));
+  }
+
+  template <size_t N>
+  llvm::Function *
+  getFunction(StringRef FuncName, QualType QT, std::array<Address, N> Addrs,
+              std::array<CharUnits, N> Alignments, CodeGenModule &CGM) {
+    // If the special function already exists in the module, return it.
+    if (llvm::Function *F = CGM.getModule().getFunction(FuncName)) {
+      bool WrongType = false;
+      if (!F->getReturnType()->isVoidTy())
+        WrongType = true;
+      else {
+        for (const llvm::Argument &Arg : F->args())
+          if (Arg.getType() != CGM.Int8PtrPtrTy)
+            WrongType = true;
+      }
+
+      if (WrongType) {
+        std::string FuncName = F->getName();
+        SourceLocation Loc = QT->castAs<RecordType>()->getDecl()->getLocation();
+        CGM.Error(Loc, "special function " + FuncName +
+                           " for non-trivial C struct has incorrect type");
+        return nullptr;
+      }
+      return F;
+    }
+
+    ASTContext &Ctx = CGM.getContext();
+    FunctionArgList Args;
+    const CGFunctionInfo &FI = getFunctionInfo<N>(CGM, Args);
+    llvm::FunctionType *FuncTy = CGM.getTypes().GetFunctionType(FI);
+    llvm::Function *F =
+        llvm::Function::Create(FuncTy, llvm::GlobalValue::LinkOnceODRLinkage,
+                               FuncName, &CGM.getModule());
+    F->setVisibility(llvm::GlobalValue::HiddenVisibility);
+    CGM.SetLLVMFunctionAttributes(nullptr, FI, F);
+    CGM.SetLLVMFunctionAttributesForDefinition(nullptr, F);
+    IdentifierInfo *II = &Ctx.Idents.get(FuncName);
+    FunctionDecl *FD = FunctionDecl::Create(
+        Ctx, Ctx.getTranslationUnitDecl(), SourceLocation(), SourceLocation(),
+        II, Ctx.VoidTy, nullptr, SC_PrivateExtern, false, false);
+    CodeGenFunction NewCGF(CGM);
+    setCGF(&NewCGF);
+    CGF->StartFunction(FD, Ctx.VoidTy, F, FI, Args);
+
+    for (unsigned I = 0; I < N; ++I) {
+      llvm::Value *V = CGF->Builder.CreateLoad(CGF->GetAddrOfLocalVar(Args[I]));
+      Addrs[I] = Address(V, Alignments[I]);
+    }
+
+    asDerived().visitStructFields(QT, CharUnits::Zero(), Addrs);
+    CGF->FinishFunction();
+    return F;
+  }
+
+  template <size_t N>
+  void callFunc(StringRef FuncName, QualType QT, std::array<Address, N> Addrs,
+                CodeGenFunction &CallerCGF) {
+    std::array<CharUnits, N> Alignments;
+    llvm::Value *Ptrs[N];
+
+    for (unsigned I = 0; I < N; ++I) {
+      Alignments[I] = Addrs[I].getAlignment();
+      Ptrs[I] =
+          CallerCGF.Builder.CreateBitCast(Addrs[I], CallerCGF.CGM.Int8PtrPtrTy)
+              .getPointer();
+    }
+
+    if (llvm::Function *F =
+            getFunction(FuncName, QT, Addrs, Alignments, CallerCGF.CGM))
+      CallerCGF.EmitNounwindRuntimeCall(F, Ptrs);
+  }
+
+  Derived &asDerived() { return static_cast<Derived &>(*this); }
+
+  void setCGF(CodeGenFunction *F) { CGF = F; }
+
+  CodeGenFunction *CGF = nullptr;
+};
+
+template <class Derived, bool IsMove>
+struct GenBinaryFunc : CopyStructVisitor<Derived, IsMove>,
+                       GenFuncBase<Derived> {
+  GenBinaryFunc(ASTContext &Ctx) : CopyStructVisitor<Derived, IsMove>(Ctx) {}
+
+  void flushTrivialFields(std::array<Address, 2> Addrs) {
+    CharUnits Size = this->End - this->Start;
+
+    if (Size.getQuantity() == 0)
+      return;
+
+    Address DstAddr = this->getAddrWithOffset(Addrs[DstIdx], this->Start);
+    Address SrcAddr = this->getAddrWithOffset(Addrs[SrcIdx], this->Start);
+
+    // Emit memcpy.
+    if (Size.getQuantity() >= 16 || !llvm::isPowerOf2_32(Size.getQuantity())) {
+      llvm::Value *SizeVal =
+          llvm::ConstantInt::get(this->CGF->SizeTy, Size.getQuantity());
+      DstAddr =
+          this->CGF->Builder.CreateElementBitCast(DstAddr, this->CGF->Int8Ty);
+      SrcAddr =
+          this->CGF->Builder.CreateElementBitCast(SrcAddr, this->CGF->Int8Ty);
+      this->CGF->Builder.CreateMemCpy(DstAddr, SrcAddr, SizeVal, false);
+    } else {
+      llvm::Type *Ty = llvm::Type::getIntNTy(
+          this->CGF->getLLVMContext(),
+          Size.getQuantity() * this->CGF->getContext().getCharWidth());
+      DstAddr = this->CGF->Builder.CreateElementBitCast(DstAddr, Ty);
+      SrcAddr = this->CGF->Builder.CreateElementBitCast(SrcAddr, Ty);
+      llvm::Value *SrcVal = this->CGF->Builder.CreateLoad(SrcAddr, false);
+      this->CGF->Builder.CreateStore(SrcVal, DstAddr, false);
+    }
+
+    this->Start = this->End = CharUnits::Zero();
+  }
+
+  template <class... Ts>
+  void visitVolatileTrivial(QualType FT, const FieldDecl *FD, CharUnits Offset,
+                            std::array<Address, 2> Addrs) {
+    QualType RT = QualType(FD->getParent()->getTypeForDecl(), 0);
+    llvm::PointerType *PtrTy = this->CGF->ConvertType(RT)->getPointerTo();
+    Address DstAddr = this->getAddrWithOffset(Addrs[DstIdx], Offset);
+    LValue DstBase = this->CGF->MakeAddrLValue(
+        this->CGF->Builder.CreateBitCast(DstAddr, PtrTy), FT);
+    LValue DstLV = this->CGF->EmitLValueForField(DstBase, FD);
+    Address SrcAddr = this->getAddrWithOffset(Addrs[SrcIdx], Offset);
+    LValue SrcBase = this->CGF->MakeAddrLValue(
+        this->CGF->Builder.CreateBitCast(SrcAddr, PtrTy), FT);
+    LValue SrcLV = this->CGF->EmitLValueForField(SrcBase, FD);
+    RValue SrcVal = this->CGF->EmitLoadOfLValue(SrcLV, SourceLocation());
+    this->CGF->EmitStoreThroughLValue(SrcVal, DstLV);
+  }
+};
+
+// These classes that emit the special functions for a non-trivial struct.
+struct GenDestructor : StructVisitor<GenDestructor>,
+                       GenFuncBase<GenDestructor>,
+                       DestructedTypeVisitor<GenDestructor> {
+  GenDestructor(ASTContext &Ctx) : StructVisitor<GenDestructor>(Ctx) {}
+  void visitARCStrong(QualType QT, const FieldDecl *FD,
+                      CharUnits CurStackOffset, std::array<Address, 1> Addrs) {
+    CGF->destroyARCStrongImprecise(
+        *CGF, getAddrWithOffset(Addrs[DstIdx], CurStackOffset, FD), QT);
+  }
+
+  void callSpecialFunction(QualType FT, CharUnits Offset,
+                           std::array<Address, 1> Addrs) {
+    CGF->callCStructDestructor(
+        CGF->MakeAddrLValue(getAddrWithOffset(Addrs[DstIdx], Offset), FT));
+  }
+};
+
+struct GenDefaultInitialize
+    : StructVisitor<GenDefaultInitialize>,
+      GenFuncBase<GenDefaultInitialize>,
+      DefaultInitializedTypeVisitor<GenDefaultInitialize> {
+  typedef GenFuncBase<GenDefaultInitialize> GenFuncBaseTy;
+  GenDefaultInitialize(ASTContext &Ctx)
+      : StructVisitor<GenDefaultInitialize>(Ctx) {}
+
+  void visitARCStrong(QualType QT, const FieldDecl *FD,
+                      CharUnits CurStackOffset, std::array<Address, 1> Addrs) {
+    CGF->EmitNullInitialization(
+        getAddrWithOffset(Addrs[DstIdx], CurStackOffset, FD), QT);
+  }
+
+  template <class FieldKind, size_t... Is>
+  void visitArray(FieldKind FK, QualType QT, const FieldDecl *FD,
+                  CharUnits CurStackOffset, std::array<Address, 1> Addrs) {
+    if (!FK)
+      return visitTrivial(QT, FD, CurStackOffset, Addrs);
+
+    ASTContext &Ctx = getContext();
+    CharUnits Size = Ctx.getTypeSizeInChars(QT);
+    QualType EltTy = Ctx.getBaseElementType(QT);
+
+    if (Size < CharUnits::fromQuantity(16) || EltTy->getAs<RecordType>()) {
+      GenFuncBaseTy::visitArray(FK, QT, FD, CurStackOffset, Addrs);
+      return;
+    }
+
+    llvm::Constant *SizeVal = CGF->Builder.getInt64(Size.getQuantity());
+    Address DstAddr = getAddrWithOffset(Addrs[DstIdx], CurStackOffset, FD);
+    Address Loc = CGF->Builder.CreateElementBitCast(DstAddr, CGF->Int8Ty);
+    CGF->Builder.CreateMemSet(Loc, CGF->Builder.getInt8(0), SizeVal,
+                              QT.isVolatileQualified());
+  }
+
+  void callSpecialFunction(QualType FT, CharUnits Offset,
+                           std::array<Address, 1> Addrs) {
+    CGF->callCStructDefaultConstructor(
+        CGF->MakeAddrLValue(getAddrWithOffset(Addrs[DstIdx], Offset), FT));
+  }
+};
+
+struct GenCopyConstructor : GenBinaryFunc<GenCopyConstructor, false> {
+  GenCopyConstructor(ASTContext &Ctx)
+      : GenBinaryFunc<GenCopyConstructor, false>(Ctx) {}
+
+  void visitARCStrong(QualType QT, const FieldDecl *FD,
+                      CharUnits CurStackOffset, std::array<Address, 2> Addrs) {
+    Addrs[DstIdx] = getAddrWithOffset(Addrs[DstIdx], CurStackOffset, FD);
+    Addrs[SrcIdx] = getAddrWithOffset(Addrs[SrcIdx], CurStackOffset, FD);
+    llvm::Value *SrcVal = CGF->EmitLoadOfScalar(
+        Addrs[SrcIdx], QT.isVolatileQualified(), QT, SourceLocation());
+    llvm::Value *Val = CGF->EmitARCRetain(QT, SrcVal);
+    CGF->EmitStoreOfScalar(Val, CGF->MakeAddrLValue(Addrs[DstIdx], QT), true);
+  }
+  void callSpecialFunction(QualType FT, CharUnits Offset,
+                           std::array<Address, 2> Addrs) {
+    CGF->callCStructCopyConstructor(CGF->MakeAddrLValue(Addrs[DstIdx], FT),
+                                    CGF->MakeAddrLValue(Addrs[SrcIdx], FT));
+  }
+};
+
+struct GenMoveConstructor : GenBinaryFunc<GenMoveConstructor, true> {
+  GenMoveConstructor(ASTContext &Ctx)
+      : GenBinaryFunc<GenMoveConstructor, true>(Ctx) {}
+
+  void visitARCStrong(QualType QT, const FieldDecl *FD,
+                      CharUnits CurStackOffset, std::array<Address, 2> Addrs) {
+    Addrs[DstIdx] = getAddrWithOffset(Addrs[DstIdx], CurStackOffset, FD);
+    Addrs[SrcIdx] = getAddrWithOffset(Addrs[SrcIdx], CurStackOffset, FD);
+    LValue SrcLV = CGF->MakeAddrLValue(Addrs[SrcIdx], QT);
+    llvm::Value *SrcVal =
+        CGF->EmitLoadOfLValue(SrcLV, SourceLocation()).getScalarVal();
+    CGF->EmitStoreOfScalar(getNullForVariable(SrcLV.getAddress()), SrcLV);
+    CGF->EmitStoreOfScalar(SrcVal, CGF->MakeAddrLValue(Addrs[DstIdx], QT),
+                           /* isInitialization */ true);
+  }
+  void callSpecialFunction(QualType FT, CharUnits Offset,
+                           std::array<Address, 2> Addrs) {
+    CGF->callCStructMoveConstructor(CGF->MakeAddrLValue(Addrs[DstIdx], FT),
+                                    CGF->MakeAddrLValue(Addrs[SrcIdx], FT));
+  }
+};
+
+struct GenCopyAssignment : GenBinaryFunc<GenCopyAssignment, false> {
+  GenCopyAssignment(ASTContext &Ctx)
+      : GenBinaryFunc<GenCopyAssignment, false>(Ctx) {}
+
+  void visitARCStrong(QualType QT, const FieldDecl *FD,
+                      CharUnits CurStackOffset, std::array<Address, 2> Addrs) {
+    Addrs[DstIdx] = getAddrWithOffset(Addrs[DstIdx], CurStackOffset, FD);
+    Addrs[SrcIdx] = getAddrWithOffset(Addrs[SrcIdx], CurStackOffset, FD);
+    llvm::Value *SrcVal = CGF->EmitLoadOfScalar(
+        Addrs[SrcIdx], QT.isVolatileQualified(), QT, SourceLocation());
+    CGF->EmitARCStoreStrong(CGF->MakeAddrLValue(Addrs[DstIdx], QT), SrcVal,
+                            false);
+  }
+  void callSpecialFunction(QualType FT, CharUnits Offset,
+                           std::array<Address, 2> Addrs) {
+    CGF->callCStructCopyAssignmentOperator(
+        CGF->MakeAddrLValue(Addrs[DstIdx], FT),
+        CGF->MakeAddrLValue(Addrs[SrcIdx], FT));
+  }
+};
+
+struct GenMoveAssignment : GenBinaryFunc<GenMoveAssignment, true> {
+  GenMoveAssignment(ASTContext &Ctx)
+      : GenBinaryFunc<GenMoveAssignment, true>(Ctx) {}
+
+  void visitARCStrong(QualType QT, const FieldDecl *FD,
+                      CharUnits CurStackOffset, std::array<Address, 2> Addrs) {
+    Addrs[DstIdx] = getAddrWithOffset(Addrs[DstIdx], CurStackOffset, FD);
+    Addrs[SrcIdx] = getAddrWithOffset(Addrs[SrcIdx], CurStackOffset, FD);
+    LValue SrcLV = CGF->MakeAddrLValue(Addrs[SrcIdx], QT);
+    llvm::Value *SrcVal =
+        CGF->EmitLoadOfLValue(SrcLV, SourceLocation()).getScalarVal();
+    CGF->EmitStoreOfScalar(getNullForVariable(SrcLV.getAddress()), SrcLV);
+    LValue DstLV = CGF->MakeAddrLValue(Addrs[DstIdx], QT);
+    llvm::Value *DstVal =
+        CGF->EmitLoadOfLValue(DstLV, SourceLocation()).getScalarVal();
+    CGF->EmitStoreOfScalar(SrcVal, DstLV);
+    CGF->EmitARCRelease(DstVal, ARCImpreciseLifetime);
+  }
+
+  void callSpecialFunction(QualType FT, CharUnits Offset,
+                           std::array<Address, 2> Addrs) {
+    CGF->callCStructMoveAssignmentOperator(
+        CGF->MakeAddrLValue(Addrs[DstIdx], FT),
+        CGF->MakeAddrLValue(Addrs[SrcIdx], FT));
+  }
+};
+
+} // namespace
+
+void CodeGenFunction::destroyNonTrivialCStruct(CodeGenFunction &CGF,
+                                               Address Addr, QualType Type) {
+  CGF.callCStructDestructor(CGF.MakeAddrLValue(Addr, Type));
+}
+
+// Default-initialize a variable that is a non-trivial struct or an array of
+// such structure.
+void CodeGenFunction::defaultInitNonTrivialCStructVar(LValue Dst) {
+  GenDefaultInitialize Gen(getContext());
+  Address DstPtr = Builder.CreateBitCast(Dst.getAddress(), CGM.Int8PtrPtrTy);
+  Gen.setCGF(this);
+  QualType QT = Dst.getType();
+  QT = Dst.isVolatile() ? QT.withVolatile() : QT;
+  Gen.visit(QT, nullptr, CharUnits::Zero(), std::array<Address, 1>({{DstPtr}}));
+}
+
+template <class G, size_t N>
+static void callSpecialFunction(G &&Gen, StringRef FuncName, QualType QT,
+                                bool IsVolatile, CodeGenFunction &CGF,
+                                std::array<Address, N> Addrs) {
+  for (unsigned I = 0; I < N; ++I)
+    Addrs[I] = CGF.Builder.CreateBitCast(Addrs[I], CGF.CGM.Int8PtrPtrTy);
+  QT = IsVolatile ? QT.withVolatile() : QT;
+  Gen.callFunc(FuncName, QT, Addrs, CGF);
+}
+
+// Functions to emit calls to the special functions of a non-trivial C struct.
+void CodeGenFunction::callCStructDefaultConstructor(LValue Dst) {
+  bool IsVolatile = Dst.isVolatile();
+  Address DstPtr = Dst.getAddress();
+  QualType QT = Dst.getType();
+  GenDefaultInitializeFuncName GenName(DstPtr.getAlignment(), getContext());
+  std::string FuncName = GenName.getName(QT, IsVolatile);
+  callSpecialFunction(GenDefaultInitialize(getContext()), FuncName, QT,
+                      IsVolatile, *this, std::array<Address, 1>({{DstPtr}}));
+}
+
+void CodeGenFunction::callCStructDestructor(LValue Dst) {
+  bool IsVolatile = Dst.isVolatile();
+  Address DstPtr = Dst.getAddress();
+  QualType QT = Dst.getType();
+  GenDestructorFuncName GenName(DstPtr.getAlignment(), getContext());
+  std::string FuncName = GenName.getName(QT, IsVolatile);
+  callSpecialFunction(GenDestructor(getContext()), FuncName, QT, IsVolatile,
+                      *this, std::array<Address, 1>({{DstPtr}}));
+}
+
+void CodeGenFunction::callCStructCopyConstructor(LValue Dst, LValue Src) {
+  bool IsVolatile = Dst.isVolatile() || Src.isVolatile();
+  Address DstPtr = Dst.getAddress(), SrcPtr = Src.getAddress();
+  QualType QT = Dst.getType();
+  GenBinaryFuncName<false> GenName("__copy_constructor_", DstPtr.getAlignment(),
+                                   SrcPtr.getAlignment(), getContext());
+  std::string FuncName = GenName.getName(QT, IsVolatile);
+  callSpecialFunction(GenCopyConstructor(getContext()), FuncName, QT,
+                      IsVolatile, *this,
+                      std::array<Address, 2>({{DstPtr, SrcPtr}}));
+}
+
+void CodeGenFunction::callCStructCopyAssignmentOperator(LValue Dst, LValue Src
+
+) {
+  bool IsVolatile = Dst.isVolatile() || Src.isVolatile();
+  Address DstPtr = Dst.getAddress(), SrcPtr = Src.getAddress();
+  QualType QT = Dst.getType();
+  GenBinaryFuncName<false> GenName("__copy_assignment_", DstPtr.getAlignment(),
+                                   SrcPtr.getAlignment(), getContext());
+  std::string FuncName = GenName.getName(QT, IsVolatile);
+  callSpecialFunction(GenCopyAssignment(getContext()), FuncName, QT, IsVolatile,
+                      *this, std::array<Address, 2>({{DstPtr, SrcPtr}}));
+}
+
+void CodeGenFunction::callCStructMoveConstructor(LValue Dst, LValue Src) {
+  bool IsVolatile = Dst.isVolatile() || Src.isVolatile();
+  Address DstPtr = Dst.getAddress(), SrcPtr = Src.getAddress();
+  QualType QT = Dst.getType();
+  GenBinaryFuncName<true> GenName("__move_constructor_", DstPtr.getAlignment(),
+                                  SrcPtr.getAlignment(), getContext());
+  std::string FuncName = GenName.getName(QT, IsVolatile);
+  callSpecialFunction(GenMoveConstructor(getContext()), FuncName, QT,
+                      IsVolatile, *this,
+                      std::array<Address, 2>({{DstPtr, SrcPtr}}));
+}
+
+void CodeGenFunction::callCStructMoveAssignmentOperator(LValue Dst, LValue Src
+
+) {
+  bool IsVolatile = Dst.isVolatile() || Src.isVolatile();
+  Address DstPtr = Dst.getAddress(), SrcPtr = Src.getAddress();
+  QualType QT = Dst.getType();
+  GenBinaryFuncName<true> GenName("__move_assignment_", DstPtr.getAlignment(),
+                                  SrcPtr.getAlignment(), getContext());
+  std::string FuncName = GenName.getName(QT, IsVolatile);
+  callSpecialFunction(GenMoveAssignment(getContext()), FuncName, QT, IsVolatile,
+                      *this, std::array<Address, 2>({{DstPtr, SrcPtr}}));
+}

cfe/trunk/lib/CodeGen/CMakeLists.txt

add_clang_library(clangCodeGen
   CGExpr.cpp
   CGExprAgg.cpp
   CGExprCXX.cpp
   CGExprComplex.cpp
   CGExprConstant.cpp
   CGExprScalar.cpp
   CGGPUBuiltin.cpp
   CGLoopInfo.cpp
+  CGNonTrivialStruct.cpp
   CGObjC.cpp
   CGObjCGNU.cpp
   CGObjCMac.cpp
   CGObjCRuntime.cpp
   CGOpenCLRuntime.cpp
   CGOpenMPRuntime.cpp
   CGOpenMPRuntimeNVPTX.cpp
   CGRecordLayoutBuilder.cpp

cfe/trunk/lib/CodeGen/CodeGenFunction.h

public:
   /// Determines whether an EH cleanup is required to destroy a type
   /// with the given destruction kind.
   bool needsEHCleanup(QualType::DestructionKind kind) {
     switch (kind) {
     case QualType::DK_none:
       return false;
     case QualType::DK_cxx_destructor:
     case QualType::DK_objc_weak_lifetime:
+    case QualType::DK_nontrivial_c_struct:
       return getLangOpts().Exceptions;
     case QualType::DK_objc_strong_lifetime:
       return getLangOpts().Exceptions &&
              CGM.getCodeGenOpts().ObjCAutoRefCountExceptions;
     }
     llvm_unreachable("bad destruction kind");
   }
 
public:
   CGCallee BuildAppleKextVirtualCall(const CXXMethodDecl *MD,
                                      NestedNameSpecifier *Qual,
                                      llvm::Type *Ty);
 
   CGCallee BuildAppleKextVirtualDestructorCall(const CXXDestructorDecl *DD,
                                                CXXDtorType Type,
                                                const CXXRecordDecl *RD);
 
+  // These functions emit calls to the special functions of non-trivial C
+  // structs.
+  void defaultInitNonTrivialCStructVar(LValue Dst);
+  void callCStructDefaultConstructor(LValue Dst);
+  void callCStructDestructor(LValue Dst);
+  void callCStructCopyConstructor(LValue Dst, LValue Src);
+  void callCStructMoveConstructor(LValue Dst, LValue Src);
+  void callCStructCopyAssignmentOperator(LValue Dst, LValue Src);
+  void callCStructMoveAssignmentOperator(LValue Dst, LValue Src);
+
   RValue
   EmitCXXMemberOrOperatorCall(const CXXMethodDecl *Method,
                               const CGCallee &Callee,
                               ReturnValueSlot ReturnValue, llvm::Value *This,
                               llvm::Value *ImplicitParam,
                               QualType ImplicitParamTy, const CallExpr *E,
                               CallArgList *RtlArgs);
   RValue EmitCXXDestructorCall(const CXXDestructorDecl *DD,
public:
   llvm::Value *EmitARCUnsafeUnretainedScalarExpr(const Expr *expr);
 
   void EmitARCIntrinsicUse(ArrayRef<llvm::Value*> values);
 
   static Destroyer destroyARCStrongImprecise;
   static Destroyer destroyARCStrongPrecise;
   static Destroyer destroyARCWeak;
   static Destroyer emitARCIntrinsicUse;
+  static Destroyer destroyNonTrivialCStruct;
 
   void EmitObjCAutoreleasePoolPop(llvm::Value *Ptr);
   llvm::Value *EmitObjCAutoreleasePoolPush();
   llvm::Value *EmitObjCMRRAutoreleasePoolPush();
   void EmitObjCAutoreleasePoolCleanup(llvm::Value *Ptr);
   void EmitObjCMRRAutoreleasePoolPop(llvm::Value *Ptr);
 
   /// \brief Emits a reference binding to the passed in expression.

cfe/trunk/lib/Lex/PPMacroExpansion.cpp

return llvm::StringSwitch<bool>(Feature)
       .Case("dataflow_sanitizer",
             LangOpts.Sanitize.has(SanitizerKind::DataFlow))
       .Case("efficiency_sanitizer",
             LangOpts.Sanitize.hasOneOf(SanitizerKind::Efficiency))
       .Case("scudo", LangOpts.Sanitize.hasOneOf(SanitizerKind::Scudo))
       // Objective-C features
       .Case("objc_arr", LangOpts.ObjCAutoRefCount) // FIXME: REMOVE?
       .Case("objc_arc", LangOpts.ObjCAutoRefCount)
+      .Case("objc_arc_fields", true)
       .Case("objc_arc_weak", LangOpts.ObjCWeak)
       .Case("objc_default_synthesize_properties", LangOpts.ObjC2)
       .Case("objc_fixed_enum", LangOpts.ObjC2)
       .Case("objc_instancetype", LangOpts.ObjC2)
       .Case("objc_kindof", LangOpts.ObjC2)
       .Case("objc_modules", LangOpts.ObjC2 && LangOpts.Modules)
       .Case("objc_nonfragile_abi", LangOpts.ObjCRuntime.isNonFragile())
       .Case("objc_property_explicit_atomic",

cfe/trunk/lib/Sema/JumpDiagnostics.cpp

if (VD->hasLocalStorage()) {
       case QualType::DK_objc_strong_lifetime:
         return ScopePair(diag::note_protected_by_objc_strong_init,
                          diag::note_exits_objc_strong);
 
       case QualType::DK_objc_weak_lifetime:
         return ScopePair(diag::note_protected_by_objc_weak_init,
                          diag::note_exits_objc_weak);
 
+      case QualType::DK_nontrivial_c_struct:
+        return ScopePair(diag::note_protected_by_non_trivial_c_struct_init,
+                         diag::note_exits_dtor);
+
       case QualType::DK_cxx_destructor:
         OutDiag = diag::note_exits_dtor;
         break;
 
       case QualType::DK_none:
         break;
       }
     }
switch (destructKind) {
       case QualType::DK_objc_strong_lifetime:
         Diags = ScopePair(diag::note_enters_block_captures_strong,
                           diag::note_exits_block_captures_strong);
         break;
       case QualType::DK_objc_weak_lifetime:
         Diags = ScopePair(diag::note_enters_block_captures_weak,
                           diag::note_exits_block_captures_weak);
         break;
+      case QualType::DK_nontrivial_c_struct:
+        Diags = ScopePair(diag::note_enters_block_captures_non_trivial_c_struct,
+                          diag::note_exits_block_captures_non_trivial_c_struct);
+        break;
       case QualType::DK_none:
         llvm_unreachable("non-lifetime captured variable");
     }
     SourceLocation Loc = D->getLocation();
     if (Loc.isInvalid())
       Loc = BDecl->getLocation();
     Scopes.push_back(GotoScope(ParentScope,
                                Diags.first, Diags.second, Loc));

cfe/trunk/lib/Sema/SemaDecl.cpp

if (getLangOpts().ObjC1 &&
 
     case Qualifiers::OCL_Weak:
     case Qualifiers::OCL_Strong:
       getCurFunction()->setHasBranchProtectedScope();
       break;
     }
   }
 
+  if (var->getType().isDestructedType() == QualType::DK_nontrivial_c_struct)
+    getCurFunction()->setHasBranchProtectedScope();
+
   // Warn about externally-visible variables being defined without a
   // prior declaration.  We only want to do this for global
   // declarations, but we also specifically need to avoid doing it for
   // class members because the linkage of an anonymous class can
   // change if it's later given a typedef name.
   if (var->isThisDeclarationADefinition() &&
       var->getDeclContext()->getRedeclContext()->isFileContext() &&
       var->isExternallyVisible() && var->hasLinkage() &&
void Sema::ActOnFields(Scope *S, SourceLocation RecLoc, Decl *EnclosingDecl,
 
   bool ObjCFieldLifetimeErrReported = false;
   for (ArrayRef<Decl *>::iterator i = Fields.begin(), end = Fields.end();
        i != end; ++i) {
     FieldDecl *FD = cast<FieldDecl>(*i);
 
     // Get the type for the field.
     const Type *FDTy = FD->getType().getTypePtr();
+    Qualifiers QS = FD->getType().getQualifiers();
 
     if (!FD->isAnonymousStructOrUnion()) {
       // Remember all fields written by the user.
       RecFields.push_back(FD);
     }
 
     // If the field is already invalid for some reason, don't emit more
     // diagnostics about it.
for (ArrayRef<Decl *>::iterator i = Fields.begin(), end = Fields.end();
     } else if (FDTy->isObjCObjectType()) {
       /// A field cannot be an Objective-c object
       Diag(FD->getLocation(), diag::err_statically_allocated_object)
         << FixItHint::CreateInsertion(FD->getLocation(), "*");
       QualType T = Context.getObjCObjectPointerType(FD->getType());
       FD->setType(T);
     } else if (getLangOpts().allowsNonTrivialObjCLifetimeQualifiers() &&
                Record && !ObjCFieldLifetimeErrReported &&
-               (!getLangOpts().CPlusPlus || Record->isUnion())) {
+               ((!getLangOpts().CPlusPlus &&
+                 QS.getObjCLifetime() == Qualifiers::OCL_Weak) ||
+                Record->isUnion())) {
       // It's an error in ARC or Weak if a field has lifetime.
       // We don't want to report this in a system header, though,
       // so we just make the field unavailable.
       // FIXME: that's really not sufficient; we need to make the type
       // itself invalid to, say, initialize or copy.
       QualType T = FD->getType();
       if (T.hasNonTrivialObjCLifetime()) {
         SourceLocation loc = FD->getLocation();
if (FDTy->isFunctionType()) {
         if (BaseType->isRecordType() &&
             BaseType->getAs<RecordType>()->getDecl()->hasObjectMember())
           Record->setHasObjectMember(true);
         else if (BaseType->isObjCObjectPointerType() ||
                  BaseType.isObjCGCStrong())
                Record->setHasObjectMember(true);
       }
     }
+
+    if (Record && !getLangOpts().CPlusPlus) {
+      QualType FT = FD->getType();
+      if (FT.isNonTrivialToPrimitiveDefaultInitialize())
+        Record->setNonTrivialToPrimitiveDefaultInitialize();
+      QualType::PrimitiveCopyKind PCK = FT.isNonTrivialToPrimitiveCopy();
+      if (PCK != QualType::PCK_Trivial && PCK != QualType::PCK_VolatileTrivial)
+        Record->setNonTrivialToPrimitiveCopy();
+      if (FT.isDestructedType())
+        Record->setNonTrivialToPrimitiveDestroy();
+    }
+
     if (Record && FD->getType().isVolatileQualified())
       Record->setHasVolatileMember(true);
     // Keep track of the number of named members.
     if (FD->getIdentifier())
       ++NumNamedMembers;
   }
 
   // Okay, we successfully defined 'Record'.

cfe/trunk/lib/Sema/SemaExpr.cpp

if (Ty->isIncompleteType()) {
     if (Ty->isVoidType())
       return VAK_Invalid;
 
     if (Ty->isObjCObjectType())
       return VAK_Invalid;
     return VAK_Valid;
   }
 
+  if (Ty.isDestructedType() == QualType::DK_nontrivial_c_struct)
+    return VAK_Invalid;
+
   if (Ty.isCXX98PODType(Context))
     return VAK_Valid;
 
   // C++11 [expr.call]p7:
   //   Passing a potentially-evaluated argument of class type (Clause 9)
   //   having a non-trivial copy constructor, a non-trivial move constructor,
   //   or a non-trivial destructor, with no corresponding parameter,
   //   is conditionally-supported with implementation-defined semantics.
void Sema::checkVariadicArgument(const Expr *E, VariadicCallType CT) {
   case VAK_MSVCUndefined:
     DiagRuntimeBehavior(
         E->getLocStart(), nullptr,
         PDiag(diag::warn_cannot_pass_non_pod_arg_to_vararg)
           << getLangOpts().CPlusPlus11 << Ty << CT);
     break;
 
   case VAK_Invalid:
-    if (Ty->isObjCObjectType())
+    if (Ty.isDestructedType() == QualType::DK_nontrivial_c_struct)
+      Diag(E->getLocStart(),
+           diag::err_cannot_pass_non_trivial_c_struct_to_vararg) << Ty << CT;
+    else if (Ty->isObjCObjectType())
       DiagRuntimeBehavior(
           E->getLocStart(), nullptr,
           PDiag(diag::err_cannot_pass_objc_interface_to_vararg)
             << Ty << CT);
     else
       Diag(E->getLocStart(), diag::err_cannot_pass_to_vararg)
         << isa<InitListExpr>(E) << Ty << CT;
     break;

cfe/trunk/test/ARCMT/checking.m

void test1(A *a, BOOL b, struct UnsafeS *unsafeS) {
   s = @selector(release); // expected-error {{ARC forbids use of 'release' in a @selector}}
   s = @selector(autorelease); // expected-error {{ARC forbids use of 'autorelease' in a @selector}}
   s = @selector(dealloc); // expected-error {{ARC forbids use of 'dealloc' in a @selector}}
 
   static id __autoreleasing X1; // expected-error {{global variables cannot have __autoreleasing ownership}}
 }
 
 struct S {
-  A* a; // expected-error {{ARC forbids Objective-C objects in struct}}
+  A* a;
 };
 
 @interface B
 -(id)alloc;
 - (id)initWithInt: (int) i;
 @end
 
 void rdar8861761() {

cfe/trunk/test/CodeGenObjC/nontrivial-c-struct-exception.m

+// RUN: %clang_cc1 -triple arm64-apple-ios11 -fobjc-arc -fblocks -fobjc-runtime=ios-11.0 -fobjc-exceptions -fexceptions -fobjc-arc-exceptions -emit-llvm -o - %s | FileCheck %s
+
+// CHECK: %[[STRUCT_STRONG:.*]] = type { i32, i8* }
+
+typedef struct {
+  int i;
+  id f1;
+} Strong;
+
+// CHECK: define void @testStrongException()
+// CHECK: %[[AGG_TMP:.*]] = alloca %[[STRUCT_STRONG]], align 8
+// CHECK: %[[AGG_TMP1:.*]] = alloca %[[STRUCT_STRONG]], align 8
+// CHECK: %[[CALL:.*]] = call [2 x i64] @genStrong()
+// CHECK: %[[V0:.*]] = bitcast %[[STRUCT_STRONG]]* %[[AGG_TMP]] to [2 x i64]*
+// CHECK: store [2 x i64] %[[CALL]], [2 x i64]* %[[V0]], align 8
+// CHECK: invoke [2 x i64] @genStrong()
+
+// CHECK: call void @calleeStrong([2 x i64] %{{.*}}, [2 x i64] %{{.*}})
+// CHECK-NEXT: ret void
+
+// CHECK: landingpad { i8*, i32 }
+// CHECK: %[[V9:.*]] = bitcast %[[STRUCT_STRONG]]* %[[AGG_TMP]] to i8**
+// CHECK: call void @__destructor_8_s8(i8** %[[V9]])
+// CHECK: br label
+
+// CHECK: resume
+
+Strong genStrong(void);
+void calleeStrong(Strong, Strong);
+
+void testStrongException(void) {
+  calleeStrong(genStrong(), genStrong());
+}

cfe/trunk/test/CodeGenObjC/nontrivial-c-struct-func-name-collision.m

+// RUN: %clang_cc1 -triple arm64-apple-ios11 -fobjc-arc -fblocks  -fobjc-runtime=ios-11.0 -emit-llvm -verify -o - %s
+
+typedef struct { // expected-error {{special function __default_constructor_8_s8 for non-trivial C struct has incorrect type}}
+  int i;
+  id f1;
+} StrongSmall;
+
+int __default_constructor_8_s8(double a) {
+  return 0;
+}
+
+void testIncorrectFunctionType(void) {
+  StrongSmall x;
+}

cfe/trunk/test/CodeGenObjC/strong-in-c-struct.m

+// RUN: %clang_cc1 -triple arm64-apple-ios11 -fobjc-arc -fblocks  -fobjc-runtime=ios-11.0 -emit-llvm -o - %s | FileCheck %s
+
+typedef void (^BlockTy)(void);
+
+typedef struct {
+  int a[4];
+} Trivial;
+
+typedef struct {
+  Trivial f0;
+  id f1;
+} Strong;
+
+typedef struct {
+  int i;
+  id f1;
+} StrongSmall;
+
+typedef struct {
+  Strong f0;
+  id f1;
+  double d;
+} StrongOuter;
+
+typedef struct {
+  int f0;
+  volatile id f1;
+} StrongVolatile;
+
+typedef struct {
+  BlockTy f0;
+} StrongBlock;
+
+typedef struct {
+  int i;
+  id f0[2][2];
+} IDArray;
+
+typedef struct {
+  double d;
+  Strong f0[2][2];
+} StructArray;
+
+typedef struct {
+  id f0;
+  int i : 9;
+} Bitfield0;
+
+typedef struct {
+  char c;
+  int i0 : 2;
+  int i1 : 4;
+  id f0;
+  int i2 : 31;
+  int i3 : 1;
+  id f1;
+  int : 0;
+  int a[3];
+  id f2;
+  double d;
+  int i4 : 1;
+  volatile int i5 : 2;
+  volatile char i6;
+} Bitfield1;
+
+StrongSmall getStrongSmall(void);
+StrongOuter getStrongOuter(void);
+void calleeStrongSmall(StrongSmall);
+void func(Strong *);
+
+// CHECK: %[[STRUCT_BITFIELD1:.*]] = type { i8, i8, i8*, i32, i8*, [3 x i32], i8*, double, i8, i8 }
+
+// CHECK: define void @test_constructor_destructor_StrongOuter()
+// CHECK: %[[T:.*]] = alloca %[[STRUCT_STRONGOUTER:.*]], align 8
+// CHECK: %[[V0:.*]] = bitcast %[[STRUCT_STRONGOUTER]]* %[[T]] to i8**
+// CHECK: call void @__default_constructor_8_s16_s24(i8** %[[V0]])
+// CHECK: %[[V1:.*]] = bitcast %[[STRUCT_STRONGOUTER]]* %[[T]] to i8**
+// CHECK: call void @__destructor_8_s16_s24(i8** %[[V1]])
+// CHECK: ret void
+
+// CHECK: define linkonce_odr hidden void @__default_constructor_8_s16_s24(i8** %[[DST:.*]])
+// CHECK: %[[DST_ADDR:.*]] = alloca i8**, align 8
+// CHECK: store i8** %[[DST]], i8*** %[[DST_ADDR]], align 8
+// CHECK: %[[V0:.*]] = load i8**, i8*** %[[DST_ADDR]], align 8
+// CHECK: call void @__default_constructor_8_s16(i8** %[[V0]])
+// CHECK: %[[V1:.*]] = bitcast i8** %[[V0]] to i8*
+// CHECK: %[[V2:.*]] = getelementptr inbounds i8, i8* %[[V1]], i64 24
+// CHECK: %[[V3:.*]] = bitcast i8* %[[V2]] to i8**
+// CHECK: %[[V4:.*]] = bitcast i8** %[[V3]] to i8*
+// CHECK: call void @llvm.memset.p0i8.i64(i8* align 8 %[[V4]], i8 0, i64 8, i1 false)
+// CHECK: ret void
+
+// CHECK: define linkonce_odr hidden void @__default_constructor_8_s16(i8** %[[DST:.*]])
+// CHECK: %[[DST_ADDR:.*]] = alloca i8**, align 8
+// CHECK: store i8** %[[DST]], i8*** %[[DST_ADDR]], align 8
+// CHECK: %[[V0:.*]] = load i8**, i8*** %[[DST_ADDR]], align 8
+// CHECK: %[[V1:.*]] = bitcast i8** %[[V0]] to i8*
+// CHECK: %[[V2:.*]] = getelementptr inbounds i8, i8* %[[V1]], i64 16
+// CHECK: %[[V3:.*]] = bitcast i8* %[[V2]] to i8**
+// CHECK: %[[V4:.*]] = bitcast i8** %[[V3]] to i8*
+// CHECK: call void @llvm.memset.p0i8.i64(i8* align 8 %[[V4]], i8 0, i64 8, i1 false)
+// CHECK: ret void
+
+// CHECK: define linkonce_odr hidden void @__destructor_8_s16_s24(i8** %[[DST:.*]])
+// CHECK: %[[DST_ADDR:.*]] = alloca i8**, align 8
+// CHECK: store i8** %[[DST]], i8*** %[[DST_ADDR]], align 8
+// CHECK: %[[V0:.*]] = load i8**, i8*** %[[DST_ADDR]], align 8
+// CHECK: call void @__destructor_8_s16(i8** %[[V0]])
+// CHECK: %[[V1:.*]] = bitcast i8** %[[V0]] to i8*
+// CHECK: %[[V2:.*]] = getelementptr inbounds i8, i8* %[[V1]], i64 24
+// CHECK: %[[V3:.*]] = bitcast i8* %[[V2]] to i8**
+// CHECK: call void @objc_storeStrong(i8** %[[V3]], i8* null)
+// CHECK: ret void
+
+// CHECK: define linkonce_odr hidden void @__destructor_8_s16(i8** %[[DST:.*]])
+// CHECK: %[[DST_ADDR:.*]] = alloca i8**, align 8
+// CHECK: store i8** %[[DST]], i8*** %[[DST_ADDR]], align 8
+// CHECK: %[[V0:.*]] = load i8**, i8*** %[[DST_ADDR]], align 8
+// CHECK: %[[V1:.*]] = bitcast i8** %[[V0]] to i8*
+// CHECK: %[[V2:.*]] = getelementptr inbounds i8, i8* %[[V1]], i64 16
+// CHECK: %[[V3:.*]] = bitcast i8* %[[V2]] to i8**
+// CHECK: call void @objc_storeStrong(i8** %[[V3]], i8* null)
+// CHECK: ret void
+
+void test_constructor_destructor_StrongOuter(void) {
+  StrongOuter t;
+}
+
+// CHECK: define void @test_copy_constructor_StrongOuter(%[[STRUCT_STRONGOUTER:.*]]* %[[S:.*]])
+// CHECK: %[[S_ADDR:.*]] = alloca %[[STRUCT_STRONGOUTER]]*, align 8
+// CHECK: %[[T:.*]] = alloca %[[STRUCT_STRONGOUTER]], align 8
+// CHECK: store %[[STRUCT_STRONGOUTER]]* %[[S]], %[[STRUCT_STRONGOUTER]]** %[[S_ADDR]], align 8
+// CHECK: %[[V0:.*]] = load %[[STRUCT_STRONGOUTER]]*, %[[STRUCT_STRONGOUTER]]** %[[S_ADDR]], align 8
+// CHECK: %[[V1:.*]] = bitcast %[[STRUCT_STRONGOUTER]]* %[[T]] to i8**
+// CHECK: %[[V2:.*]] = bitcast %[[STRUCT_STRONGOUTER]]* %[[V0]] to i8**
+// CHECK: call void @__copy_constructor_8_8_t0w16_s16_s24_t32w8(i8** %[[V1]], i8** %[[V2]])
+// CHECK: %[[V3:.*]] = bitcast %[[STRUCT_STRONGOUTER]]* %[[T]] to i8**
+// CHECK: call void @__destructor_8_s16_s24(i8** %[[V3]])
+// CHECK: ret void
+
+// CHECK: define linkonce_odr hidden void @__copy_constructor_8_8_t0w16_s16_s24_t32w8(i8** %[[DST:.*]], i8** %[[SRC:.*]])
+// CHECK: %[[DST_ADDR:.*]] = alloca i8**, align 8
+// CHECK: %[[SRC_ADDR:.*]] = alloca i8**, align 8
+// CHECK: store i8** %[[DST]], i8*** %[[DST_ADDR]], align 8
+// CHECK: store i8** %[[SRC]], i8*** %[[SRC_ADDR]], align 8
+// CHECK: %[[V0:.*]] = load i8**, i8*** %[[DST_ADDR]], align 8
+// CHECK: %[[V1:.*]] = load i8**, i8*** %[[SRC_ADDR]], align 8
+// CHECK: call void @__copy_constructor_8_8_t0w16_s16(i8** %[[V0]], i8** %[[V1]])
+// CHECK: %[[V2:.*]] = bitcast i8** %[[V0]] to i8*
+// CHECK: %[[V3:.*]] = getelementptr inbounds i8, i8* %[[V2]], i64 24
+// CHECK: %[[V4:.*]] = bitcast i8* %[[V3]] to i8**
+// CHECK: %[[V5:.*]] = bitcast i8** %[[V1]] to i8*
+// CHECK: %[[V6:.*]] = getelementptr inbounds i8, i8* %[[V5]], i64 24
+// CHECK: %[[V7:.*]] = bitcast i8* %[[V6]] to i8**
+// CHECK: %[[V8:.*]] = load i8*, i8** %[[V7]], align 8
+// CHECK: %[[V9:.*]] = call i8* @objc_retain(i8* %[[V8]])
+// CHECK: store i8* %[[V9]], i8** %[[V4]], align 8
+// CHECK: %[[V10:.*]] = bitcast i8** %[[V0]] to i8*
+// CHECK: %[[V11:.*]] = getelementptr inbounds i8, i8* %[[V10]], i64 32
+// CHECK: %[[V12:.*]] = bitcast i8* %[[V11]] to i8**
+// CHECK: %[[V13:.*]] = bitcast i8** %[[V1]] to i8*
+// CHECK: %[[V14:.*]] = getelementptr inbounds i8, i8* %[[V13]], i64 32
+// CHECK: %[[V15:.*]] = bitcast i8* %[[V14]] to i8**
+// CHECK: %[[V16:.*]] = bitcast i8** %[[V12]] to i64*
+// CHECK: %[[V17:.*]] = bitcast i8** %[[V15]] to i64*
+// CHECK: %[[V18:.*]] = load i64, i64* %[[V17]], align 8
+// CHECK: store i64 %[[V18]], i64* %[[V16]], align 8
+// CHECK: ret void
+
+// CHECK: define linkonce_odr hidden void @__copy_constructor_8_8_t0w16_s16(i8** %[[DST:.*]], i8** %[[SRC:.*]])
+// CHECK: %[[DST_ADDR:.*]] = alloca i8**, align 8
+// CHECK: %[[SRC_ADDR:.*]] = alloca i8**, align 8
+// CHECK: store i8** %[[DST]], i8*** %[[DST_ADDR]], align 8
+// CHECK: store i8** %[[SRC]], i8*** %[[SRC_ADDR]], align 8
+// CHECK: %[[V0:.*]] = load i8**, i8*** %[[DST_ADDR]], align 8
+// CHECK: %[[V1:.*]] = load i8**, i8*** %[[SRC_ADDR]], align 8
+// CHECK: %[[V2:.*]] = bitcast i8** %[[V0]] to i8*
+// CHECK: %[[V3:.*]] = bitcast i8** %[[V1]] to i8*
+// CHECK: call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 8 %[[V2]], i8* align 8 %[[V3]], i64 16, i1 false)
+// CHECK: %[[V4:.*]] = bitcast i8** %[[V0]] to i8*
+// CHECK: %[[V5:.*]] = getelementptr inbounds i8, i8* %[[V4]], i64 16
+// CHECK: %[[V6:.*]] = bitcast i8* %[[V5]] to i8**
+// CHECK: %[[V7:.*]] = bitcast i8** %[[V1]] to i8*
+// CHECK: %[[V8:.*]] = getelementptr inbounds i8, i8* %[[V7]], i64 16
+// CHECK: %[[V9:.*]] = bitcast i8* %[[V8]] to i8**
+// CHECK: %[[V10:.*]] = load i8*, i8** %[[V9]], align 8
+// CHECK: %[[V11:.*]] = call i8* @objc_retain(i8* %[[V10]])
+// CHECK: store i8* %[[V11]], i8** %[[V6]], align 8
+// CHECK: ret void
+
+void test_copy_constructor_StrongOuter(StrongOuter *s) {
+  StrongOuter t = *s;
+}
+
+/// CHECK: define linkonce_odr hidden void @__copy_assignment_8_8_t0w16_s16_s24_t32w8(i8** %[[DST:.*]], i8** %[[SRC:.*]])
+// CHECK: %[[DST_ADDR:.*]] = alloca i8**, align 8
+// CHECK: %[[SRC_ADDR:.*]] = alloca i8**, align 8
+// CHECK: store i8** %[[DST]], i8*** %[[DST_ADDR]], align 8
+// CHECK: store i8** %[[SRC]], i8*** %[[SRC_ADDR]], align 8
+// CHECK: %[[V0:.*]] = load i8**, i8*** %[[DST_ADDR]], align 8
+// CHECK: %[[V1:.*]] = load i8**, i8*** %[[SRC_ADDR]], align 8
+// CHECK: %[[V2:.*]] = bitcast i8** %[[V0]] to i8*
+// CHECK: %[[V3:.*]] = getelementptr inbounds i8, i8* %[[V2]], i64 24
+// CHECK: %[[V4:.*]] = bitcast i8* %[[V3]] to i8**
+// CHECK: %[[V5:.*]] = bitcast i8** %[[V1]] to i8*
+// CHECK: %[[V6:.*]] = getelementptr inbounds i8, i8* %[[V5]], i64 24
+// CHECK: %[[V7:.*]] = bitcast i8* %[[V6]] to i8**
+// CHECK: %[[V8:.*]] = load i8*, i8** %[[V7]], align 8
+// CHECK: call void @objc_storeStrong(i8** %[[V4]], i8* %[[V8]])
+
+void test_copy_assignment_StrongOuter(StrongOuter *d, StrongOuter *s) {
+  *d = *s;
+}
+
+// CHECK: define void @test_move_constructor_StrongOuter()
+// CHECK: %[[T1:.*]] = getelementptr inbounds %[[STRUCT_BLOCK_BYREF_T:.*]], %[[STRUCT_BLOCK_BYREF_T]]* %{{.*}}, i32 0, i32 7
+// CHECK: %[[V1:.*]] = bitcast %[[STRUCT_STRONGOUTER]]* %[[T1]] to i8**
+// CHECK: call void @__default_constructor_8_s16_s24(i8** %[[V1]])
+// CHECK: %[[T2:.*]] = getelementptr inbounds %[[STRUCT_BLOCK_BYREF_T]], %[[STRUCT_BLOCK_BYREF_T]]* %{{.*}}, i32 0, i32 7
+// CHECK: %[[V9:.*]] = bitcast %[[STRUCT_STRONGOUTER]]* %[[T2]] to i8**
+// CHECK: call void @__destructor_8_s16_s24(i8** %[[V9]])
+
+// CHECK: define internal void @__Block_byref_object_copy_(i8*, i8*)
+// CHECK: call void @__move_constructor_8_8_t0w16_s16_s24_t32w8(
+
+// CHECK: define linkonce_odr hidden void @__move_constructor_8_8_t0w16_s16_s24_t32w8(i8** %[[DST:.*]], i8** %[[SRC:.*]])
+// CHECK: %[[DST_ADDR:.*]] = alloca i8**, align 8
+// CHECK: %[[SRC_ADDR:.*]] = alloca i8**, align 8
+// CHECK: store i8** %[[DST]], i8*** %[[DST_ADDR]], align 8
+// CHECK: store i8** %[[SRC]], i8*** %[[SRC_ADDR]], align 8
+// CHECK: %[[V0:.*]] = load i8**, i8*** %[[DST_ADDR]], align 8
+// CHECK: %[[V1:.*]] = load i8**, i8*** %[[SRC_ADDR]], align 8
+// CHECK: call void @__move_constructor_8_8_t0w16_s16(i8** %[[V0]], i8** %[[V1]])
+// CHECK: %[[V2:.*]] = bitcast i8** %[[V0]] to i8*
+// CHECK: %[[V3:.*]] = getelementptr inbounds i8, i8* %[[V2]], i64 24
+// CHECK: %[[V4:.*]] = bitcast i8* %[[V3]] to i8**
+// CHECK: %[[V5:.*]] = bitcast i8** %[[V1]] to i8*
+// CHECK: %[[V6:.*]] = getelementptr inbounds i8, i8* %[[V5]], i64 24
+// CHECK: %[[V7:.*]] = bitcast i8* %[[V6]] to i8**
+// CHECK: %[[V8:.*]] = load i8*, i8** %[[V7]], align 8
+// CHECK: store i8* null, i8** %[[V7]], align 8
+// CHECK: store i8* %[[V8]], i8** %[[V4]], align 8
+
+// CHECK: define internal void @__Block_byref_object_dispose_(i8*)
+// CHECK: call void @__destructor_8_s16_s24(
+
+void test_move_constructor_StrongOuter(void) {
+  __block StrongOuter t;
+  BlockTy b = ^{ (void)t; };
+}
+
+// CHECK: define linkonce_odr hidden void @__move_assignment_8_8_t0w16_s16_s24_t32w8(i8** %[[DST:.*]], i8** %[[SRC:.*]])
+// CHECK: %[[DST_ADDR:.*]] = alloca i8**, align 8
+// CHECK: %[[SRC_ADDR:.*]] = alloca i8**, align 8
+// CHECK: store i8** %[[DST]], i8*** %[[DST_ADDR]], align 8
+// CHECK: store i8** %[[SRC]], i8*** %[[SRC_ADDR]], align 8
+// CHECK: %[[V0:.*]] = load i8**, i8*** %[[DST_ADDR]], align 8
+// CHECK: %[[V1:.*]] = load i8**, i8*** %[[SRC_ADDR]], align 8
+// CHECK: call void @__move_assignment_8_8_t0w16_s16(i8** %[[V0]], i8** %[[V1]])
+// CHECK: %[[V2:.*]] = bitcast i8** %[[V0]] to i8*
+// CHECK: %[[V3:.*]] = getelementptr inbounds i8, i8* %[[V2]], i64 24
+// CHECK: %[[V4:.*]] = bitcast i8* %[[V3]] to i8**
+// CHECK: %[[V5:.*]] = bitcast i8** %[[V1]] to i8*
+// CHECK: %[[V6:.*]] = getelementptr inbounds i8, i8* %[[V5]], i64 24
+// CHECK: %[[V7:.*]] = bitcast i8* %[[V6]] to i8**
+// CHECK: %[[V8:.*]] = load i8*, i8** %[[V7]], align 8
+// CHECK: store i8* null, i8** %[[V7]], align 8
+// CHECK: %[[V9:.*]] = load i8*, i8** %[[V4]], align 8
+// CHECK: store i8* %[[V8]], i8** %[[V4]], align 8
+// CHECK: call void @objc_release(i8* %[[V9]])
+
+void test_move_assignment_StrongOuter(StrongOuter *p) {
+  *p = getStrongOuter();
+}
+
+// CHECK: define void @test_parameter_StrongSmall([2 x i64] %[[A_COERCE:.*]])
+// CHECK: %[[A:.*]] = alloca %[[STRUCT_STRONG:.*]], align 8
+// CHECK: %[[V0:.*]] = bitcast %[[STRUCT_STRONG]]* %[[A]] to [2 x i64]*
+// CHECK: store [2 x i64] %[[A_COERCE]], [2 x i64]* %[[V0]], align 8
+// CHECK: %[[V1:.*]] = bitcast %[[STRUCT_STRONG]]* %[[A]] to i8**
+// CHECK: call void @__destructor_8_s8(i8** %[[V1]])
+// CHECK: ret void
+
+void test_parameter_StrongSmall(StrongSmall a) {
+}
+
+// CHECK: define void @test_argument_StrongSmall([2 x i64] %[[A_COERCE:.*]])
+// CHECK: %[[A:.*]] = alloca %[[STRUCT_STRONGSMALL:.*]], align 8
+// CHECK: %[[TEMP_LVALUE:.*]] = alloca %[[STRUCT_STRONGSMALL]], align 8
+// CHECK: %[[V0:.*]] = bitcast %[[STRUCT_STRONGSMALL]]* %[[A]] to [2 x i64]*
+// CHECK: store [2 x i64] %[[A_COERCE]], [2 x i64]* %[[V0]], align 8
+// CHECK: %[[V1:.*]] = bitcast %[[STRUCT_STRONGSMALL]]* %[[TEMP_LVALUE]] to i8**
+// CHECK: %[[V2:.*]] = bitcast %[[STRUCT_STRONGSMALL]]* %[[A]] to i8**
+// CHECK: call void @__copy_constructor_8_8_t0w4_s8(i8** %[[V1]], i8** %[[V2]])
+// CHECK: %[[V3:.*]] = bitcast %[[STRUCT_STRONGSMALL]]* %[[TEMP_LVALUE]] to [2 x i64]*
+// CHECK: %[[V4:.*]] = load [2 x i64], [2 x i64]* %[[V3]], align 8
+// CHECK: call void @calleeStrongSmall([2 x i64] %[[V4]])
+// CHECK: %[[V5:.*]] = bitcast %[[STRUCT_STRONGSMALL]]* %[[A]] to i8**
+// CHECK: call void @__destructor_8_s8(i8** %[[V5]])
+// CHECK: ret void
+
+void test_argument_StrongSmall(StrongSmall a) {
+  calleeStrongSmall(a);
+}
+
+// CHECK: define [2 x i64] @test_return_StrongSmall([2 x i64] %[[A_COERCE:.*]])
+// CHECK: %[[RETVAL:.*]] = alloca %[[STRUCT_STRONGSMALL:.*]], align 8
+// CHECK: %[[A:.*]] = alloca %[[STRUCT_STRONGSMALL]], align 8
+// CHECK: %[[V0:.*]] = bitcast %[[STRUCT_STRONGSMALL]]* %[[A]] to [2 x i64]*
+// CHECK: store [2 x i64] %[[A_COERCE]], [2 x i64]* %[[V0]], align 8
+// CHECK: %[[V1:.*]] = bitcast %[[STRUCT_STRONGSMALL]]* %[[RETVAL]] to i8**
+// CHECK: %[[V2:.*]] = bitcast %[[STRUCT_STRONGSMALL]]* %[[A]] to i8**
+// CHECK: call void @__copy_constructor_8_8_t0w4_s8(i8** %[[V1]], i8** %[[V2]])
+// CHECK: %[[V3:.*]] = bitcast %[[STRUCT_STRONGSMALL]]* %[[A]] to i8**
+// CHECK: call void @__destructor_8_s8(i8** %[[V3]])
+// CHECK: %[[V4:.*]] = bitcast %[[STRUCT_STRONGSMALL]]* %[[RETVAL]] to [2 x i64]*
+// CHECK: %[[V5:.*]] = load [2 x i64], [2 x i64]* %[[V4]], align 8
+// CHECK: ret [2 x i64] %[[V5]]
+
+StrongSmall test_return_StrongSmall(StrongSmall a) {
+  return a;
+}
+
+// CHECK: define void @test_destructor_ignored_result()
+// CHECK: %[[COERCE:.*]] = alloca %[[STRUCT_STRONGSMALL:.*]], align 8
+// CHECK: %[[CALL:.*]] = call [2 x i64] @getStrongSmall()
+// CHECK: %[[V0:.*]] = bitcast %[[STRUCT_STRONGSMALL]]* %[[COERCE]] to [2 x i64]*
+// CHECK: store [2 x i64] %[[CALL]], [2 x i64]* %[[V0]], align 8
+// CHECK: %[[V1:.*]] = bitcast %[[STRUCT_STRONGSMALL]]* %[[COERCE]] to i8**
+// CHECK: call void @__destructor_8_s8(i8** %[[V1]])
+// CHECK: ret void
+
+void test_destructor_ignored_result(void) {
+  getStrongSmall();
+}
+
+// CHECK: define void @test_copy_constructor_StrongBlock(
+// CHECK: call void @__copy_constructor_8_8_sb0(
+// CHECK: call void @__destructor_8_sb0(
+// CHECK: ret void
+
+// CHECK: define linkonce_odr hidden void @__copy_constructor_8_8_sb0(i8** %[[DST:.*]], i8** %[[SRC:.*]])
+// CHECK: %[[DST_ADDR:.*]] = alloca i8**, align 8
+// CHECK: %[[SRC_ADDR:.*]] = alloca i8**, align 8
+// CHECK: store i8** %[[DST]], i8*** %[[DST_ADDR]], align 8
+// CHECK: store i8** %[[SRC]], i8*** %[[SRC_ADDR]], align 8
+// CHECK: %[[V0:.*]] = load i8**, i8*** %[[DST_ADDR]], align 8
+// CHECK: %[[V1:.*]] = load i8**, i8*** %[[SRC_ADDR]], align 8
+// CHECK: %[[V2:.*]] = load i8*, i8** %[[V1]], align 8
+// CHECK: %[[V3:.*]] = call i8* @objc_retainBlock(i8* %[[V2]])
+// CHECK: store i8* %[[V3]], i8** %[[V0]], align 8
+// CHECK: ret void
+
+void test_copy_constructor_StrongBlock(StrongBlock *s) {
+  StrongBlock t = *s;
+}
+
+// CHECK: define void @test_copy_assignment_StrongBlock(%[[STRUCT_STRONGBLOCK:.*]]* %[[D:.*]], %[[STRUCT_STRONGBLOCK]]* %[[S:.*]])
+// CHECK: call void @__copy_assignment_8_8_sb0(
+
+// CHECK: define linkonce_odr hidden void @__copy_assignment_8_8_sb0(i8** %[[DST:.*]], i8** %[[SRC:.*]])
+// CHECK: %[[DST_ADDR:.*]] = alloca i8**, align 8
+// CHECK: %[[SRC_ADDR:.*]] = alloca i8**, align 8
+// CHECK: store i8** %[[DST]], i8*** %[[DST_ADDR]], align 8
+// CHECK: store i8** %[[SRC]], i8*** %[[SRC_ADDR]], align 8
+// CHECK: %[[V0:.*]] = load i8**, i8*** %[[DST_ADDR]], align 8
+// CHECK: %[[V1:.*]] = load i8**, i8*** %[[SRC_ADDR]], align 8
+// CHECK: %[[V2:.*]] = load i8*, i8** %[[V1]], align 8
+// CHECK: %[[V3:.*]] = call i8* @objc_retainBlock(i8* %[[V2]])
+// CHECK: %[[V4:.*]] = load i8*, i8** %[[V0]], align 8
+// CHECK: store i8* %[[V3]], i8** %[[V0]], align 8
+// CHECK: call void @objc_release(i8* %[[V4]])
+// CHECK: ret void
+
+void test_copy_assignment_StrongBlock(StrongBlock *d, StrongBlock *s) {
+  *d = *s;
+}
+
+// CHECK: define void @test_copy_constructor_StrongVolatile0(
+// CHECK: call void @__copy_constructor_8_8_t0w4_sv8(
+// CHECK: call void @__destructor_8_sv8(
+
+// CHECK: define linkonce_odr hidden void @__copy_constructor_8_8_t0w4_sv8(
+// CHECK: %[[V8:.*]] = load volatile i8*, i8** %{{.*}}, align 8
+// CHECK: %[[V9:.*]] = call i8* @objc_retain(i8* %[[V8]])
+// CHECK: store volatile i8* %[[V9]], i8** %{{.*}}, align 8
+
+void test_copy_constructor_StrongVolatile0(StrongVolatile *s) {
+  StrongVolatile t = *s;
+}
+
+// CHECK: define void @test_copy_constructor_StrongVolatile1(
+// CHECK: call void @__copy_constructor_8_8_tv0w128_sv16(
+
+void test_copy_constructor_StrongVolatile1(Strong *s) {
+  volatile Strong t = *s;
+}
+
+// CHECK: define void @test_block_capture_Strong()
+// CHECK: call void @__default_constructor_8_s16(
+// CHECK: call void @__copy_constructor_8_8_t0w16_s16(
+// CHECK: call void @__destructor_8_s16(
+// CHECK: call void @__destructor_8_s16(
+// CHECK: ret void
+
+// CHECK: define internal void @__copy_helper_block_.1(i8*, i8*)
+// CHECK: call void @__copy_constructor_8_8_t0w16_s16(
+// CHECK: ret void
+
+// CHECK: define internal void @__destroy_helper_block_.2(
+// CHECK: call void @__destructor_8_s16(
+// CHECK: ret void
+
+void test_block_capture_Strong(void) {
+  Strong t;
+  BlockTy b = ^(){ (void)t; };
+}
+
+// CHECK: define void @test_variable_length_array(i32 %[[N:.*]])
+// CHECK: %[[N_ADDR:.*]] = alloca i32, align 4
+// CHECK: store i32 %[[N]], i32* %[[N_ADDR]], align 4
+// CHECK: %[[V0:.*]] = load i32, i32* %[[N_ADDR]], align 4
+// CHECK: %[[V1:.*]] = zext i32 %[[V0]] to i64
+// CHECK: %[[VLA:.*]] = alloca %[[STRUCT_STRONG:.*]], i64 %[[V1]], align 8
+// CHECK: %[[V3:.*]] = bitcast %[[STRUCT_STRONG]]* %[[VLA]] to i8**
+// CHECK: %[[V4:.*]] = mul nuw i64 24, %[[V1]]
+// CHECK: %[[V5:.*]] = bitcast i8** %[[V3]] to i8*
+// CHECK: %[[V6:.*]] = getelementptr inbounds i8, i8* %[[V5]], i64 %[[V4]]
+// CHECK: %[[DSTARRAY_END:.*]] = bitcast i8* %[[V6]] to i8**
+// CHECK: br label
+
+// CHECK: %[[DSTADDR_CUR:.*]] = phi i8** [ %[[V3]], {{.*}} ], [ %[[V7:.*]], {{.*}} ]
+// CHECK: %[[DONE:.*]] = icmp eq i8** %[[DSTADDR_CUR]], %[[DSTARRAY_END]]
+// CHECK: br i1 %[[DONE]], label
+
+// CHECK: call void @__default_constructor_8_s16(i8** %[[DSTADDR_CUR]])
+// CHECK: %[[V8:.*]] = bitcast i8** %[[DSTADDR_CUR]] to i8*
+// CHECK: %[[V9:.*]] = getelementptr inbounds i8, i8* %[[V8]], i64 24
+// CHECK: %[[V7]] = bitcast i8* %[[V9]] to i8**
+// CHECK: br label
+
+// CHECK: call void @func(%[[STRUCT_STRONG]]* %[[VLA]])
+// CHECK: %[[V10:.*]] = getelementptr inbounds %[[STRUCT_STRONG]], %[[STRUCT_STRONG]]* %[[VLA]], i64 %[[V1]]
+// CHECK: %[[ARRAYDESTROY_ISEMPTY:.*]] = icmp eq %[[STRUCT_STRONG]]* %[[VLA]], %[[V10]]
+// CHECK: br i1 %[[ARRAYDESTROY_ISEMPTY]], label
+
+// CHECK: %[[ARRAYDESTROY_ELEMENTPAST:.*]] = phi %[[STRUCT_STRONG]]* [ %[[V10]], {{.*}} ], [ %[[ARRAYDESTROY_ELEMENT:.*]], {{.*}} ]
+// CHECK: %[[ARRAYDESTROY_ELEMENT]] = getelementptr inbounds %[[STRUCT_STRONG]], %[[STRUCT_STRONG]]* %[[ARRAYDESTROY_ELEMENTPAST]], i64 -1
+// CHECK: %[[V11:.*]] = bitcast %[[STRUCT_STRONG]]* %[[ARRAYDESTROY_ELEMENT]] to i8**
+// CHECK: call void @__destructor_8_s16(i8** %[[V11]])
+// CHECK: %[[ARRAYDESTROY_DONE:.*]] = icmp eq %[[STRUCT_STRONG]]* %[[ARRAYDESTROY_ELEMENT]], %[[VLA]]
+// CHECK: br i1 %[[ARRAYDESTROY_DONE]], label
+
+// CHECK: ret void
+
+void test_variable_length_array(int n) {
+  Strong a[n];
+  func(a);
+}
+
+// CHECK: define linkonce_odr hidden void @__default_constructor_8_AB8s8n4_s8_AE(
+// CHECK: call void @llvm.memset.p0i8.i64(i8* align 8 %{{.*}}, i8 0, i64 32, i1 false)
+void test_constructor_destructor_IDArray(void) {
+  IDArray t;
+}
+
+// CHECK: define linkonce_odr hidden void @__default_constructor_8_AB8s24n4_s24_AE(
+void test_constructor_destructor_StructArray(void) {
+  StructArray t;
+}
+
+// Check that IRGen copies the 9-bit bitfield emitting i16 load and store.
+
+// CHECK: define linkonce_odr hidden void @__copy_constructor_8_8_s0_t8w2(
+// CHECK: %[[V4:.*]] = bitcast i8** %{{.*}} to i8*
+// CHECK: %[[V5:.*]] = getelementptr inbounds i8, i8* %[[V4]], i64 8
+// CHECK: %[[V6:.*]] = bitcast i8* %[[V5]] to i8**
+// CHECK: %[[V7:.*]] = bitcast i8** %{{.*}} to i8*
+// CHECK: %[[V8:.*]] = getelementptr inbounds i8, i8* %[[V7]], i64 8
+// CHECK: %[[V9:.*]] = bitcast i8* %[[V8]] to i8**
+// CHECK: %[[V10:.*]] = bitcast i8** %[[V6]] to i16*
+// CHECK: %[[V11:.*]] = bitcast i8** %[[V9]] to i16*
+// CHECK: %[[V12:.*]] = load i16, i16* %[[V11]], align 8
+// CHECK: store i16 %[[V12]], i16* %[[V10]], align 8
+// CHECK: ret void
+
+void test_copy_constructor_Bitfield0(Bitfield0 *a) {
+  Bitfield0 t = *a;
+}
+
+// CHECK: define linkonce_odr hidden void @__copy_constructor_8_8_t0w2_s8_t16w4_s24_t32w12_s48_t56w9_tv513w2_tv520w8
+// CHECK: %[[V4:.*]] = load i16, i16* %{{.*}}, align 8
+// CHECK: store i16 %[[V4]], i16* %{{.*}}, align 8
+// CHECK: %[[V21:.*]] = load i32, i32* %{{.*}}, align 8
+// CHECK: store i32 %[[V21]], i32* %{{.*}}, align 8
+// CHECK: call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 8 %{{.*}}, i8* align 8 %{{.*}}, i64 12, i1 false)
+// CHECK: call void @llvm.memcpy.p0i8.p0i8.i64(i8* align 8 %{{.*}}, i8* align 8 %{{.*}}, i64 9, i1 false)
+// CHECK: %[[V54:.*]] = bitcast i8** %[[V0:.*]] to %[[STRUCT_BITFIELD1]]*
+// CHECK: %[[I5:.*]] = getelementptr inbounds %[[STRUCT_BITFIELD1]], %[[STRUCT_BITFIELD1]]* %[[V54]], i32 0, i32 8
+// CHECK: %[[V55:.*]] = bitcast i8** %[[V1:.*]] to %[[STRUCT_BITFIELD1]]*
+// CHECK: %[[I51:.*]] = getelementptr inbounds %[[STRUCT_BITFIELD1]], %[[STRUCT_BITFIELD1]]* %[[V55]], i32 0, i32 8
+// CHECK: %[[BF_LOAD:.*]] = load volatile i8, i8* %[[I51]], align 8
+// CHECK: %[[BF_SHL:.*]] = shl i8 %[[BF_LOAD]], 5
+// CHECK: %[[BF_ASHR:.*]] = ashr i8 %[[BF_SHL]], 6
+// CHECK: %[[BF_CAST:.*]] = sext i8 %[[BF_ASHR]] to i32
+// CHECK: %[[V56:.*]] = trunc i32 %[[BF_CAST]] to i8
+// CHECK: %[[BF_LOAD2:.*]] = load volatile i8, i8* %[[I5]], align 8
+// CHECK: %[[BF_VALUE:.*]] = and i8 %[[V56]], 3
+// CHECK: %[[BF_SHL3:.*]] = shl i8 %[[BF_VALUE]], 1
+// CHECK: %[[BF_CLEAR:.*]] = and i8 %[[BF_LOAD2]], -7
+// CHECK: %[[BF_SET:.*]] = or i8 %[[BF_CLEAR]], %[[BF_SHL3]]
+// CHECK: store volatile i8 %[[BF_SET]], i8* %[[I5]], align 8
+// CHECK: %[[V57:.*]] = bitcast i8** %[[V0]] to %[[STRUCT_BITFIELD1]]*
+// CHECK: %[[I6:.*]] = getelementptr inbounds %[[STRUCT_BITFIELD1]], %[[STRUCT_BITFIELD1]]* %[[V57]], i32 0, i32 9
+// CHECK: %[[V58:.*]] = bitcast i8** %[[V1]] to %[[STRUCT_BITFIELD1]]*
+// CHECK: %[[I64:.*]] = getelementptr inbounds %[[STRUCT_BITFIELD1]], %[[STRUCT_BITFIELD1]]* %[[V58]], i32 0, i32 9
+// CHECK: %[[V59:.*]] = load volatile i8, i8* %[[I64]], align 1
+// CHECK: store volatile i8 %[[V59]], i8* %[[I6]], align 1
+
+void test_copy_constructor_Bitfield1(Bitfield1 *a) {
+  Bitfield1 t = *a;
+}

cfe/trunk/test/Lexer/has_feature_objc_arc.m

 // RUN: %clang_cc1 -E %s -fobjc-arc "-triple" "x86_64-apple-macosx10.7.0"  -fobjc-runtime-has-weak | FileCheck --check-prefix=CHECK-ARC %s
 // RUN: %clang_cc1 -E %s -fobjc-arc "-triple" "x86_64-apple-macosx10.6.0" | FileCheck --check-prefix=CHECK-ARCLITE %s
 
 #if __has_feature(objc_arc)
 void has_objc_arc_feature();
 #else
 void no_objc_arc_feature();
 #endif
 
 #if __has_feature(objc_arc_weak)
 void has_objc_arc_weak_feature();
 #else
 void no_objc_arc_weak_feature();
 #endif
 
+#if __has_feature(objc_arc_fields)
+void has_objc_arc_fields();
+#else
+void no_objc_arc_fields();
+#endif
+
 // CHECK-ARC: void has_objc_arc_feature();
 // CHECK-ARC: void has_objc_arc_weak_feature();
+// CHECK-ARC: void has_objc_arc_fields();
 
 // CHECK-ARCLITE: void has_objc_arc_feature();
 // CHECK-ARCLITE: void no_objc_arc_weak_feature();
+// CHECK-ARCLITE: void has_objc_arc_fields();

cfe/trunk/test/SemaObjC/arc-decls.m

 // RUN: %clang_cc1 -fsyntax-only -fblocks -fobjc-arc -verify -Wno-objc-root-class %s
 
 // rdar://8843524
 
 struct A {
-    id x; // expected-error {{ARC forbids Objective-C objects in struct}}
+    id x;
 };
 
 union u {
     id u; // expected-error {{ARC forbids Objective-C objects in union}}
 };
 
 @interface I {
    struct A a; 
    struct B {
-    id y[10][20]; // expected-error {{ARC forbids Objective-C objects in struct}}
+    id y[10][20];
     id z;
    } b;
 
    union u c; 
 };
 @end
 
 // rdar://10260525
 struct r10260525 {
-  id (^block) (); // expected-error {{ARC forbids blocks in struct}}
+  id (^block) ();
 };
 
 struct S { 
     id __attribute__((objc_ownership(none))) i;
     void * vp;
     int i1;
 };
 

cfe/trunk/test/SemaObjC/arc-system-header.m

 
 void test4(Test4 *p) {
   p->field1 = 0; // expected-error {{'field1' is unavailable in ARC}}
                  // expected-note@arc-system-header.h:19 {{declaration uses type that is ill-formed in ARC}}
   p->field2 = 0;
 }
 
 void test5(struct Test5 *p) {
-  p->field = 0; // expected-error {{'field' is unavailable in ARC}}
-                // expected-note@arc-system-header.h:25 {{field has non-trivial ownership qualification}}
+  p->field = 0;
 }
 
 id test6() {
   // This is actually okay to use if declared in a system header.
   id x;
   x = (id) kMagicConstant;
   x = (id) (x ? kMagicConstant : kMagicConstant);
   x = (id) (x ? kMagicConstant : (void*) 0);
 
   extern void test6_helper();
   x = (id) (test6_helper(), kMagicConstant);
 }
 
 void test7(Test7 *p) {
   *p.prop = 0; // expected-error {{'prop' is unavailable in ARC}}
   p.prop = 0; // expected-error {{'prop' is unavailable in ARC}}
   *[p prop] = 0; // expected-error {{'prop' is unavailable in ARC}}
   [p setProp: 0]; // expected-error {{'setProp:' is unavailable in ARC}}
 // expected-note@arc-system-header.h:41 4 {{declaration uses type that is ill-formed in ARC}}
 // expected-note@arc-system-header.h:41 2 {{property 'prop' is declared unavailable here}}
 }
 
 extern void doSomething(Test9 arg);
 void test9() {
-    Test9 foo2 = {0, 0}; // expected-error {{'field' is unavailable in ARC}}
-                         // expected-note@arc-system-header.h:56 {{field has non-trivial ownership qualification}}
+    Test9 foo2 = {0, 0};
     doSomething(foo2);
 }
 #endif
 
 // test8 in header

cfe/trunk/test/SemaObjC/strong-in-c-struct.m

+// RUN: %clang_cc1 -triple arm64-apple-ios11 -fobjc-arc -fblocks  -fobjc-runtime=ios-11.0 -fsyntax-only -verify %s
+
+typedef struct {
+  id a;
+} Strong;
+
+void callee_variadic(const char *, ...);
+
+void test_variadic(void) {
+  Strong t;
+  callee_variadic("s", t); // expected-error {{cannot pass non-trivial C object of type 'Strong' by value to variadic function}}
+}
+
+void test_jump0(int cond) {
+  switch (cond) {
+  case 0:
+    ;
+    Strong x; // expected-note {{jump bypasses initialization of variable of non-trivial C struct type}}
+    break;
+  case 1: // expected-error {{cannot jump from switch statement to this case label}}
+    x.a = 0;
+    break;
+  }
+}
+
+void test_jump1(void) {
+  static void *ips[] = { &&L0 };
+L0:  // expected-note {{possible target of indirect goto}}
+  ;
+  Strong x; // expected-note {{jump exits scope of variable with non-trivial destructor}}
+  goto *ips; // expected-error {{cannot jump}}
+}
+
+typedef void (^BlockTy)(void);
+void func(BlockTy);
+void func2(Strong);
+
+void test_block_scope0(int cond) {
+  Strong x; // expected-note {{jump enters lifetime of block which captures a C struct that is non-trivial to destroy}}
+  switch (cond) {
+  case 0:
+    func(^{ func2(x); });
+    break;
+  default: // expected-error {{cannot jump from switch statement to this case label}}
+    break;
+  }
+}
+
+void test_block_scope1(void) {
+  static void *ips[] = { &&L0 };
+L0:  // expected-note {{possible target of indirect goto}}
+  ;
+  Strong x; // expected-note {{jump exits scope of variable with non-trivial destructor}} expected-note {{jump exits lifetime of block which captures a C struct that is non-trivial to destroy}}
+  func(^{ func2(x); });
+  goto *ips; // expected-error {{cannot jump}}
+}