CodeGen: Fix PR40605 by splitting constant struct initializers

[glider]

Summary

When emitting initializers for local structures for code built with
-ftrivial-auto-var-init, replace constant structures with sequences of
stores.

This appears to greatly help removing dead initialization stores to those
locals that are later overwritten by other data.
This also removes a lot of .rodata constants (see PR40605), replacing most
of them with immediate values (for Linux kernel the .rodata size is
reduced by ~1.9%)

[jfb]

Thanks for taking this on! As I said in the bug I think we definitely want something like this change.

Can you add a link to bug 40605 in the commit message, as well as the size results you quoted there?

You should have some tests that show the resulting difference in code. I'm especially interested in seeing empty structs, large structs, nested structs, unions, arrays.

Maybe a separate change: I think you should do the same thing for small arrays.

tools/clang/lib/CodeGen/CGDecl.cpp

1099 ↗

(On Diff #185768)

If you do keep this forInit, you should use an enum class (instead of a bool) instead to be self-documenting.

1143 ↗

(On Diff #185768)

I think you need a heuristic here, where you don't emit multiple stores if getNumOperands is greater than some number. In that case you'd keep doing memcpy instead. Note that a struct containing sub-structs (or arrays) should also count the number of sub-structs (so just checking getNumOperands isn't sufficient).

Other parts of this code chose 6 stores...

1765 ↗

(On Diff #185768)

Can you explain why this case needs to be different? IIUC it's because the types can differ which makes the struct's elements not match up the ones from the constant in the loop above? I think the code above should automatically handle that case instead of passing in a flag here.

You also definitely need to test the case where that happens :)

[glider]

tools/clang/lib/CodeGen/CGDecl.cpp

1143 ↗

(On Diff #185768)

Do we ever need to not split these stores? Can't the MemCpyOpt pass take care of them later on?

1765 ↗

(On Diff #185768)

I just thought we shouldn't break the constants that we didn't create with -ftrivial-var-auto-init.
I'll take a closer look though (indeed, this crashes on one of the structs from the Linux kernel)

[glider]

Sorry, didn't mean to mark these done.

[glider]

Updated the patch: instead of passing around forInit, match the constant type to pointer type.

I'm not quite sure how to show the resulting difference in code. Do you mean we need Clang to support both modes and to compare the resulting assembly?

[glider]

Re: case when STy != Loc.getElementType() - this is already covered by other Clang tests.

I'm still unsure about the heuristic here. I believe that for auto-initialization we want to be quite aggressive with these splits (unlike for regular constant stores).
Perhaps we should do the split in the case all bytes are pattern bytes? (This is probably another use case for forInit)

[jfb]

Can you add a link to bug 40605 in the commit message?

I'm not quite sure how to show the resulting difference in code. Do you mean we need Clang to support both modes and to compare the resulting assembly?

I only meant tests that show codegen, as you've now added auto-var-init.cpp. There might be interesting cases which I wasn't testing when I wrote it, so if you think of any please do make sure you add some. You mentioned a Linux struct that used to crash? That would be a useful test (I imagine it's one with the Loc adjustment).

tools/clang/lib/CodeGen/CGDecl.cpp

1143 ↗

(On Diff #185768)

You're assuming the optimizer is running at all :-)
In general your approach seems to care about -O2, and I agree that's usually what we want to tune. However, clang also support -O0 which won't touch these stores at all, and in -O0 it's usually nice to just see a copy from a global while debugging. Further, configurations such as -Os and -Oz would be hampered by excessive codegen.

You also need to consider the compile-time hit the expansion you're making would add. More code means more time to compile, and I'd expect little to no win from emitting more than a handful of stores. If someone has a large thing on the stack, and it can be optimized, then we should teach the optimizer to deal with memcpy better. I don't think e.g. a 400+ byte struct makes sense to initialize with a bunch of store.

So I think you want to check what the size of the struct is (check out when x86-64 and ARM64 inline memcpy / memset, I assume those are decent guesses for size to inline).

Further, your current code sill do something silly for code like this:

struct S { int j; char arr[8]; float f; };

Before you'd have a single memcpy, but now it'll be a store, a memset, and a store. I think you really want to have the same heuristic for arrays (so you'd have just stores), and you want to make sure that if you're going to split up a struct you do so for all the fields inside that struct.

1765 ↗

(On Diff #185768)

Yeah I think it's from the Loc adjustment above. I think your change is worth doing for all cases, not just trivial var auto-init. It'll lead us to optimize all of it much better IMO (and support things like -O0 -Os and -Oz uniformly well too).

tools/clang/test/CodeGenCXX/auto-var-init.cpp

37 ↗

(On Diff #185990)

How come @__const.test_small_custom.custom still gets emitted? I guess custom initialization goes through a different code path?

496 ↗

(On Diff #185990)

I wonder if (maybe in a separate patch) you also want to avoid memset when something is pretty small.

664 ↗

(On Diff #185990)

This one is particularly silly since it's a zero-sized memset :)

[glider]

Can you add a link to bug 40605 in the commit message?

It's in the title now, doesn't that count? :)

[jfb]

In D57898#1390816, @glider wrote:

Can you add a link to bug 40605 in the commit message?

It's in the title now, doesn't that count? :)

Oh OK I hadn't seen that.

[glider]

tools/clang/test/CodeGenCXX/auto-var-init.cpp

496 ↗

(On Diff #185990)

If I'm understanding your request correctly, the same code also handles the memset() case for zero-initialization.

[glider]

Added a helper function to decide whether we want to break the structure or not.
Right now it only checks for optimization level and structure size.

The constant 64 is picked to match the cacheline size. Other interesting numbers I've considered were:

• 160 - a size of a kernel structure for which the initialization was inlined without any noticeable profit (no stores to that uninit structure in the same function)
• 256 - the maximum size for an inlined memset/memcpy with -mno-sse
• 512 - the maximum size for an inlined memset/memcpy with SSE enabled/

I don't think we have enough data to back any of the choices though.

I've also fixed the code to skip zero-size memset() calls.

[glider]

This patch shaved off 2.5% of performance slowdown on https://github.com/kamalmarhubi/linux-ipc-benchmarks/blob/master/af_inet_loopback.c on a kernel built in pattern-initialization mode.
The .rodata size went down by only 12K (0.5%, since we now split only structures smaller than 64 bytes)
The .text size went down by 19K (0.4%)

[jfb]

Overall this LGTM besides a few nits, and wanting input from @rjmccall.

As follow-ups (which I can take on):

• Handle the case where the types don't match (because Loc was adjusted).
• Handle small arrays (and any other cases where the struct stores get broken down into components which still contain a memcpy/ memset).

tools/clang/lib/CodeGen/CGDecl.cpp

975 ↗

(On Diff #186469)

You don't use Init in the function.

979 ↗

(On Diff #186469)

The general 64-byte heuristic is fine with me. It's just a bit weird to special-case -O0, but we do it elsewhere too (and it keeps the current status-quo of letting the backend decide what to do). From that perspective I'm fine with it.

@rjmccall do you have a strong preference either way?

One small nit: can you use ByteSize instead of just Size? Makes it easier to figure out what's going on in code IMO.

tools/clang/test/CodeGenCXX/auto-var-init.cpp

476 ↗

(On Diff #186469)

The tests aren't matching labels anymore: PATTERN-LABEL is a dead label check now. I think forking things at -O0 and -O1 is fine, but I think you want a small -O0 test (separate from this patch) which does one or two things, and then you want this test to only look at -O1. That way you don't need so much stuff changing in the test (and -O0 is still tested).

[glider]

Addressed the comments.

tools/clang/lib/CodeGen/CGDecl.cpp

979 ↗

(On Diff #186469)

I don't have a strong opinion on variable naming, but this:

974 static bool shouldSplitStructStore(CodeGenModule &CGM,
975                                    uint64_t GlobalByteSize) {
976   // Don't break structures that occupy more than one cacheline.
977   uint64_t ByteSizeLimit = 64;
978   if (CGM.getCodeGenOpts().OptimizationLevel == 0)
979     return false;
980   if (GlobalByteSize <= ByteSizeLimit)
981     return true;
982   return false;

looks a bit more heavyweight than the previous function, in which we also have GlobalSize and SizeLimit.

tools/clang/test/CodeGenCXX/auto-var-init.cpp

476 ↗

(On Diff #186469)

Why? We're passing both PATTERN and PATTERN-OX to FileCheck when testing.
I've checked that replacing @test_empty_uninit() on this line with a different function name makes the test fail.

[glider]

Well, now we somewhat break padding initialization.

E.g. for the following struct:

struct s { 
  int a;
  char b;
  long int c;
};

we generate the following constant initializer with -O0:

.L__const.foo.local:
        .long   2863311530              # 0xaaaaaaaa
        .byte   170                     # 0xaa
        .zero   3   
        .quad   -6148914691236517206    # 0xaaaaaaaaaaaaaaaa
        .size   .L__const.foo.local, 16

, which overwrites the padding bytes on stack, but with a wrong constant.

OTOH with -O1 (i.e. with my patch) we generate the following sequence of stores:

movl    $-1431655766, 8(%rsp)   # imm = 0xAAAAAAAA
movb    $-86, 12(%rsp)
movabsq $-6148914691236517206, %rax # imm = 0xAAAAAAAAAAAAAAAA
movq    %rax, 16(%rsp)

[glider]

... which happily skips the padding.

It occurs to me that we need to properly handle the padding in patternFor before we'll be able to split the structures.

[rjmccall]

tools/clang/lib/CodeGen/CGDecl.cpp

979 ↗

(On Diff #186469)

What are the cases we actually want to make sure we emit as separate stores? Basically just really small types that happen to be wrapped up in several layers of struct for abstraction purposes, right? Maybe this heuristic should primarily consider element counts (and types) and use overall sizes at the top level.

[jfb]

In D57898#1395953, @glider wrote:

... which happily skips the padding.

I don't think padding is an issue right now. It's valid to either initialize it or not. It is slightly unfortunate to lose the information about padding (which the struct approach keeps). One alternative could be to store a struct when it has padding (i.e. not two stores of char+int, but a single store struct of char+int). From what I've seen the optimization pipeline doesn't do as well with that type of stores.

You might instead just choose to not do anything in this patch if the struct has padding. We can do something later.

FWIW I intend to write a separate patch which lets you opt in to padding always being set. I'll be touching this code anyways.

It occurs to me that we need to properly handle the padding in patternFor before we'll be able to split the structures.

patternFor handles padding already: it treats it as any value, and chooses something adjacent.

tools/clang/lib/CodeGen/CGDecl.cpp

979 ↗

(On Diff #186469)

Yes, small types.

[glider]

FWIW I think I've almost finished handling the padding: https://reviews.llvm.org/D58188
I haven't checked whether it works correctly with padding in custom initializers (like struct s local = {1, 2}), but TEST_UNINIT and TEST_BRACES are covered already.

[jfb]

I think (with test updates) this will be good to go once D58188 lands.

tools/clang/lib/CodeGen/CGDecl.cpp

1158 ↗

(On Diff #186605)

Can you leave a FIXME here to handle STY != Loc.getElementType(), as well as another FIXME to handle non-struct aggregate types such as arrays? I'm happy to do them as a follow-up.

[glider]

Rebased the code, updated the tests, added FIXME.

[jfb]

Can you change the title to be more descriptive?

Small test fixes, otherwise this LGTM, will check box after update.

clang/test/CodeGenCXX/auto-var-init.cpp

123

-NOT is in the wrong place above.

130

-NOT is in the wrong place above.

[glider]

clang/test/CodeGenCXX/auto-var-init.cpp

130

Hm, I wonder if lit could automatically check that all the patterns in the file are used in RUN lines.
(Probably no, it's hard to tell if an all-caps word is a pattern or just a comment)

[jfb]

LGTM, thanks!

[glider]

Landed r355181, thank you!

clang/lib/CodeGen/CGDecl.cpp

 static llvm::Value *shouldUseMemSetToInitialize(llvm::Constant *Init,
                                                 uint64_t GlobalSize) {
   uint64_t SizeLimit = 32;
   if (GlobalSize <= SizeLimit)
     return nullptr;
   return llvm::isBytewiseValue(Init);
 }
 
+/// Decide whether we want to split a constant structure store into a sequence
+/// of its fields' stores. This may cost us code size and compilation speed,
+/// but plays better with store optimizations.
+static bool shouldSplitStructStore(CodeGenModule &CGM,
+                                   uint64_t GlobalByteSize) {
+  // Don't break structures that occupy more than one cacheline.
+  uint64_t ByteSizeLimit = 64;
+  if (CGM.getCodeGenOpts().OptimizationLevel == 0)
+    return false;
+  if (GlobalByteSize <= ByteSizeLimit)
+    return true;
+  return false;
+}
+
 static llvm::Constant *patternFor(CodeGenModule &CGM, llvm::Type *Ty) {
   // The following value is a guaranteed unmappable pointer value and has a
   // repeated byte-pattern which makes it easier to synthesize. We use it for
   // pointers as well as integers so that aggregates are likely to be
   // initialized with this repeated value.
   constexpr uint64_t LargeValue = 0xAAAAAAAAAAAAAAAAull;
   // For 32-bit platforms it's a bit trickier because, across systems, only the
   // zero page can reasonably be expected to be unmapped, and even then we need
static void emitStoresForConstant(CodeGenModule &CGM, const VarDecl &D,
   }
 
   auto *Int8Ty = llvm::IntegerType::getInt8Ty(CGM.getLLVMContext());
   auto *IntPtrTy = CGM.getDataLayout().getIntPtrType(CGM.getLLVMContext());
 
   // If the initializer is all or mostly the same, codegen with bzero / memset
   // then do a few stores afterward.
   uint64_t ConstantSize = CGM.getDataLayout().getTypeAllocSize(Ty);
+  if (!ConstantSize)
+    return;
   auto *SizeVal = llvm::ConstantInt::get(IntPtrTy, ConstantSize);
   if (shouldUseBZeroPlusStoresToInitialize(constant, ConstantSize)) {
     Builder.CreateMemSet(Loc, llvm::ConstantInt::get(Int8Ty, 0), SizeVal,
                          isVolatile);
 
     bool valueAlreadyCorrect =
         constant->isNullValue() || isa<llvm::UndefValue>(constant);
     if (!valueAlreadyCorrect) {
if (!isa<llvm::UndefValue>(Pattern)) {
       assert(AP.getBitWidth() <= 8);
       Value = AP.getLimitedValue();
     }
     Builder.CreateMemSet(Loc, llvm::ConstantInt::get(Int8Ty, Value), SizeVal,
                          isVolatile);
     return;
   }
 
+  llvm::StructType *STy = dyn_cast<llvm::StructType>(Ty);
+  // FIXME: handle the case when STy != Loc.getElementType().
+  // FIXME: handle non-struct aggregate types.
+  if (STy && (STy == Loc.getElementType()) &&
+      shouldSplitStructStore(CGM, ConstantSize)) {
+    for (unsigned i = 0; i != constant->getNumOperands(); i++) {
+      Address EltPtr = Builder.CreateStructGEP(Loc, i);
+      emitStoresForConstant(
+          CGM, D, EltPtr, isVolatile, Builder,
+          cast<llvm::Constant>(Builder.CreateExtractValue(constant, i)));
+    }
+    return;
+  }
+
   Builder.CreateMemCpy(
       Loc,
       createUnnamedGlobalFrom(CGM, D, Builder, constant, Loc.getAlignment()),
       SizeVal, isVolatile);
 }
 
 static void emitStoresForZeroInit(CodeGenModule &CGM, const VarDecl &D,
                                   Address Loc, bool isVolatile,

clang/test/CodeGenCXX/auto-var-init.cpp

-// RUN: %clang_cc1 -std=c++14 -triple x86_64-unknown-unknown -fblocks %s -emit-llvm -o - | FileCheck %s
-// RUN: %clang_cc1 -std=c++14 -triple x86_64-unknown-unknown -fblocks -ftrivial-auto-var-init=pattern %s -emit-llvm -o - | FileCheck %s -check-prefix=PATTERN
-// RUN: %clang_cc1 -std=c++14 -triple x86_64-unknown-unknown -fblocks -ftrivial-auto-var-init=zero %s -emit-llvm -o - | FileCheck %s -check-prefix=ZERO
+// RUN: %clang_cc1 -std=c++14 -triple x86_64-unknown-unknown -fblocks %s -emit-llvm -o - | FileCheck %s -check-prefixes=CHECK,CHECK-O0
+// RUN: %clang_cc1 -std=c++14 -triple x86_64-unknown-unknown -fblocks -ftrivial-auto-var-init=pattern %s -emit-llvm -o - | FileCheck %s -check-prefixes=CHECK-O0,PATTERN,PATTERN-O0
+// RUN: %clang_cc1 -std=c++14 -triple x86_64-unknown-unknown -fblocks -ftrivial-auto-var-init=pattern %s -O1 -emit-llvm -o - | FileCheck %s -check-prefixes=CHECK-O1,PATTERN,PATTERN-O1
+// RUN: %clang_cc1 -std=c++14 -triple x86_64-unknown-unknown -fblocks -ftrivial-auto-var-init=zero %s -emit-llvm -o - | FileCheck %s -check-prefixes=CHECK-O0,ZERO,ZERO-O0
+// RUN: %clang_cc1 -std=c++14 -triple x86_64-unknown-unknown -fblocks -ftrivial-auto-var-init=zero %s -O1 -emit-llvm -o - | FileCheck %s -check-prefixes=CHECK-O1,ZERO,ZERO-O1
 
 template<typename T> void used(T &) noexcept;
 
 #define TEST_UNINIT(NAME, TYPE)                 \
   using type_##NAME = TYPE;                     \
   void test_##NAME##_uninit() {                 \
     type_##NAME uninit;                         \
     used(uninit);                               \
void test_##NAME##_custom() {                 \
     type_##NAME custom __VA_ARGS__;             \
     used(custom);                               \
   }
 
 // None of the synthesized globals should contain `undef`.
 // PATTERN-NOT: undef
 // ZERO-NOT: undef
 
-// PATTERN: @__const.test_empty_uninit.uninit = private unnamed_addr constant %struct.empty { i8 -86 }, align 1
+// PATTERN-O0: @__const.test_empty_uninit.uninit = private unnamed_addr constant %struct.empty { i8 -86 }, align 1
+// PATTERN-O1-NOT: @__const.test_empty_uninit.uninit
 struct empty {};
-// PATTERN: @__const.test_small_uninit.uninit = private unnamed_addr constant %struct.small { i8 -86 }, align 1
-// PATTERN: @__const.test_small_custom.custom = private unnamed_addr constant %struct.small { i8 42 }, align 1
-// ZERO: @__const.test_small_custom.custom = private unnamed_addr constant %struct.small { i8 42 }, align 1
+// PATTERN-O0: @__const.test_small_uninit.uninit = private unnamed_addr constant %struct.small { i8 -86 }, align 1
+// PATTERN-O0: @__const.test_small_custom.custom = private unnamed_addr constant %struct.small { i8 42 }, align 1
+// ZERO-O0: @__const.test_small_custom.custom = private unnamed_addr constant %struct.small { i8 42 }, align 1
+// PATTERN-O1-NOT: @__const.test_small_uninit.uninit
+// PATTERN-O1-NOT: @__const.test_small_custom.custom
+// ZERO-O1-NOT: @__const.test_small_custom.custom
 struct small { char c; };
-// PATTERN: @__const.test_smallinit_uninit.uninit = private unnamed_addr constant %struct.smallinit { i8 -86 }, align 1
-// PATTERN: @__const.test_smallinit_braces.braces = private unnamed_addr constant %struct.smallinit { i8 -86 }, align 1
-// PATTERN: @__const.test_smallinit_custom.custom = private unnamed_addr constant %struct.smallinit { i8 -86 }, align 1
+// PATTERN-O0: @__const.test_smallinit_uninit.uninit = private unnamed_addr constant %struct.smallinit { i8 -86 }, align 1
+// PATTERN-O0: @__const.test_smallinit_braces.braces = private unnamed_addr constant %struct.smallinit { i8 -86 }, align 1
+// PATTERN-O0: @__const.test_smallinit_custom.custom = private unnamed_addr constant %struct.smallinit { i8 -86 }, align 1
+// PATTERN-O1-NOT: @__const.test_smallinit_uninit.uninit
+// PATTERN-O1-NOT: @__const.test_smallinit_braces.braces
+// PATTERN-O1-NOT: @__const.test_smallinit_custom.custom
 struct smallinit { char c = 42; };
-// PATTERN: @__const.test_smallpartinit_uninit.uninit = private unnamed_addr constant %struct.smallpartinit { i8 -86, i8 -86 }, align 1
-// PATTERN: @__const.test_smallpartinit_braces.braces = private unnamed_addr constant %struct.smallpartinit { i8 -86, i8 -86 }, align 1
-// PATTERN: @__const.test_smallpartinit_custom.custom = private unnamed_addr constant %struct.smallpartinit { i8 -86, i8 -86 }, align 1
+// PATTERN-O0: @__const.test_smallpartinit_uninit.uninit = private unnamed_addr constant %struct.smallpartinit { i8 -86, i8 -86 }, align 1
+// PATTERN-O0: @__const.test_smallpartinit_braces.braces = private unnamed_addr constant %struct.smallpartinit { i8 -86, i8 -86 }, align 1
+// PATTERN-O0: @__const.test_smallpartinit_custom.custom = private unnamed_addr constant %struct.smallpartinit { i8 -86, i8 -86 }, align 1
+// PATTERN-O1-NOT: @__const.test_smallpartinit_uninit.uninit
+// PATTERN-O1-NOT: @__const.test_smallpartinit_braces.braces
+// PATTERN-O1-NOT: @__const.test_smallpartinit_custom.custom
 struct smallpartinit { char c = 42, d; };
-// PATTERN: @__const.test_nullinit_uninit.uninit = private unnamed_addr constant %struct.nullinit { i8* inttoptr (i64 -6148914691236517206 to i8*) }, align 8
-// PATTERN: @__const.test_nullinit_braces.braces = private unnamed_addr constant %struct.nullinit { i8* inttoptr (i64 -6148914691236517206 to i8*) }, align 8
-// PATTERN: @__const.test_nullinit_custom.custom = private unnamed_addr constant %struct.nullinit { i8* inttoptr (i64 -6148914691236517206 to i8*) }, align 8
+// PATTERN-O0: @__const.test_nullinit_uninit.uninit = private unnamed_addr constant %struct.nullinit { i8* inttoptr (i64 -6148914691236517206 to i8*) }, align 8
+// PATTERN-O0: @__const.test_nullinit_braces.braces = private unnamed_addr constant %struct.nullinit { i8* inttoptr (i64 -6148914691236517206 to i8*) }, align 8
+// PATTERN-O0: @__const.test_nullinit_custom.custom = private unnamed_addr constant %struct.nullinit { i8* inttoptr (i64 -6148914691236517206 to i8*) }, align 8
+// PATTERN-O1-NOT: @__const.test_nullinit_uninit.uninit
+// PATTERN-O1-NOT: @__const.test_nullinit_braces.braces
+// PATTERN-O1-NOT: @__const.test_nullinit_custom.custom
 struct nullinit { char* null = nullptr; };
-// PATTERN: @__const.test_padded_uninit.uninit = private unnamed_addr constant { i8, [3 x i8], i32 } { i8 -86, [3 x i8] c"\AA\AA\AA", i32 -1431655766 }, align 4
-// PATTERN: @__const.test_padded_custom.custom = private unnamed_addr constant { i8, [3 x i8], i32 } { i8 42, [3 x i8] c"\AA\AA\AA", i32 13371337 }, align 4
-// ZERO: @__const.test_padded_custom.custom = private unnamed_addr constant { i8, [3 x i8], i32 } { i8 42, [3 x i8] zeroinitializer, i32 13371337 }, align 4
+// PATTERN-O0: @__const.test_padded_uninit.uninit = private unnamed_addr constant { i8, [3 x i8], i32 } { i8 -86, [3 x i8] c"\AA\AA\AA", i32 -1431655766 }, align 4
+// PATTERN-O0: @__const.test_padded_custom.custom = private unnamed_addr constant { i8, [3 x i8], i32 } { i8 42, [3 x i8] c"\AA\AA\AA", i32 13371337 }, align 4
+// ZERO-O0: @__const.test_padded_custom.custom = private unnamed_addr constant { i8, [3 x i8], i32 } { i8 42, [3 x i8] zeroinitializer, i32 13371337 }, align 4
+// PATTERN-O1-NOT: @__const.test_padded_uninit.uninit
+// PATTERN-O1-NOT: @__const.test_padded_custom.custom
+// ZERO-O1-NOT: @__const.test_padded_custom.custom
 struct padded { char c; int i; };
-// PATTERN: @__const.test_paddednullinit_uninit.uninit = private unnamed_addr constant { i8, [3 x i8], i32 } { i8 -86, [3 x i8] c"\AA\AA\AA", i32 -1431655766 }, align 4
-// PATTERN: @__const.test_paddednullinit_braces.braces = private unnamed_addr constant { i8, [3 x i8], i32 } { i8 -86, [3 x i8] c"\AA\AA\AA", i32 -1431655766 }, align 4
-// PATTERN: @__const.test_paddednullinit_custom.custom = private unnamed_addr constant { i8, [3 x i8], i32 } { i8 -86, [3 x i8] c"\AA\AA\AA", i32 -1431655766 }, align 4
+// PATTERN-O0: @__const.test_paddednullinit_uninit.uninit = private unnamed_addr constant { i8, [3 x i8], i32 } { i8 -86, [3 x i8] c"\AA\AA\AA", i32 -1431655766 }, align 4
+// PATTERN-O0: @__const.test_paddednullinit_braces.braces = private unnamed_addr constant { i8, [3 x i8], i32 } { i8 -86, [3 x i8] c"\AA\AA\AA", i32 -1431655766 }, align 4
+// PATTERN-O0: @__const.test_paddednullinit_custom.custom = private unnamed_addr constant { i8, [3 x i8], i32 } { i8 -86, [3 x i8] c"\AA\AA\AA", i32 -1431655766 }, align 4
+// PATTERN-O1-NOT: @__const.test_paddednullinit_uninit.uninit
+// PATTERN-O1-NOT: @__const.test_paddednullinit_braces.braces
+// PATTERN-O1-NOT: @__const.test_paddednullinit_custom.custom
 struct paddednullinit { char c = 0; int i = 0; };
-// PATTERN: @__const.test_paddedpacked_uninit.uninit = private unnamed_addr constant %struct.paddedpacked <{ i8 -86, i32 -1431655766 }>, align 1
+// PATTERN-O0: @__const.test_paddedpacked_uninit.uninit = private unnamed_addr constant %struct.paddedpacked <{ i8 -86, i32 -1431655766 }>, align 1
 // PATTERN: @__const.test_paddedpacked_custom.custom = private unnamed_addr constant %struct.paddedpacked <{ i8 42, i32 13371337 }>, align 1
 // ZERO: @__const.test_paddedpacked_custom.custom = private unnamed_addr constant %struct.paddedpacked <{ i8 42, i32 13371337 }>, align 1
 struct paddedpacked { char c; int i; } __attribute__((packed));
-// PATTERN: @__const.test_paddedpackedarray_uninit.uninit = private unnamed_addr constant %struct.paddedpackedarray { [2 x %struct.paddedpacked] [%struct.paddedpacked <{ i8 -86, i32 -1431655766 }>, %struct.paddedpacked <{ i8 -86, i32 -1431655766 }>] }, align 1
+// PATTERN-O0: @__const.test_paddedpackedarray_uninit.uninit = private unnamed_addr constant %struct.paddedpackedarray { [2 x %struct.paddedpacked] [%struct.paddedpacked <{ i8 -86, i32 -1431655766 }>, %struct.paddedpacked <{ i8 -86, i32 -1431655766 }>] }, align 1
 // PATTERN: @__const.test_paddedpackedarray_custom.custom = private unnamed_addr constant %struct.paddedpackedarray { [2 x %struct.paddedpacked] [%struct.paddedpacked <{ i8 42, i32 13371337 }>, %struct.paddedpacked <{ i8 43, i32 13371338 }>] }, align 1
 // ZERO: @__const.test_paddedpackedarray_custom.custom = private unnamed_addr constant %struct.paddedpackedarray { [2 x %struct.paddedpacked] [%struct.paddedpacked <{ i8 42, i32 13371337 }>, %struct.paddedpacked <{ i8 43, i32 13371338 }>] }, align 1
 struct paddedpackedarray { struct paddedpacked p[2]; };
-// PATTERN: @__const.test_paddednested_uninit.uninit = private unnamed_addr constant { { i8, [3 x i8], i32 }, { i8, [3 x i8], i32 } } { { i8, [3 x i8], i32 } { i8 -86, [3 x i8] c"\AA\AA\AA", i32 -1431655766 }, { i8, [3 x i8], i32 } { i8 -86, [3 x i8] c"\AA\AA\AA", i32 -1431655766 } }, align 4
+// PATTERN-O0: @__const.test_paddednested_uninit.uninit = private unnamed_addr constant { { i8, [3 x i8], i32 }, { i8, [3 x i8], i32 } } { { i8, [3 x i8], i32 } { i8 -86, [3 x i8] c"\AA\AA\AA", i32 -1431655766 }, { i8, [3 x i8], i32 } { i8 -86, [3 x i8] c"\AA\AA\AA", i32 -1431655766 } }, align 4
 // PATTERN: @__const.test_paddednested_custom.custom = private unnamed_addr constant { { i8, [3 x i8], i32 }, { i8, [3 x i8], i32 } } { { i8, [3 x i8], i32 } { i8 42, [3 x i8] c"\AA\AA\AA", i32 13371337 }, { i8, [3 x i8], i32 } { i8 43, [3 x i8] c"\AA\AA\AA", i32 13371338 } }, align 4
 // ZERO: @__const.test_paddednested_custom.custom = private unnamed_addr constant { { i8, [3 x i8], i32 }, { i8, [3 x i8], i32 } } { { i8, [3 x i8], i32 } { i8 42, [3 x i8] zeroinitializer, i32 13371337 }, { i8, [3 x i8], i32 } { i8 43, [3 x i8] zeroinitializer, i32 13371338 } }, align 4
 struct paddednested { struct padded p1, p2; };
-// PATTERN: @__const.test_paddedpackednested_uninit.uninit = private unnamed_addr constant %struct.paddedpackednested { %struct.paddedpacked <{ i8 -86, i32 -1431655766 }>, %struct.paddedpacked <{ i8 -86, i32 -1431655766 }> }, align 1
+// PATTERN-O0: @__const.test_paddedpackednested_uninit.uninit = private unnamed_addr constant %struct.paddedpackednested { %struct.paddedpacked <{ i8 -86, i32 -1431655766 }>, %struct.paddedpacked <{ i8 -86, i32 -1431655766 }> }, align 1
 // PATTERN: @__const.test_paddedpackednested_custom.custom = private unnamed_addr constant %struct.paddedpackednested { %struct.paddedpacked <{ i8 42, i32 13371337 }>, %struct.paddedpacked <{ i8 43, i32 13371338 }> }, align 1
 // ZERO: @__const.test_paddedpackednested_custom.custom = private unnamed_addr constant %struct.paddedpackednested { %struct.paddedpacked <{ i8 42, i32 13371337 }>, %struct.paddedpacked <{ i8 43, i32 13371338 }> }, align 1
 struct paddedpackednested { struct paddedpacked p1, p2; };
-// PATTERN: @__const.test_bitfield_uninit.uninit = private unnamed_addr constant %struct.bitfield { i8 -86, [3 x i8] c"\AA\AA\AA" }, align 4
-// PATTERN: @__const.test_bitfield_custom.custom = private unnamed_addr constant %struct.bitfield { i8 20, [3 x i8] c"\AA\AA\AA" }, align 4
-// ZERO: @__const.test_bitfield_custom.custom = private unnamed_addr constant %struct.bitfield { i8 20, [3 x i8] zeroinitializer }, align 4
+// PATTERN-O0: @__const.test_bitfield_uninit.uninit = private unnamed_addr constant %struct.bitfield { i8 -86, [3 x i8] c"\AA\AA\AA" }, align 4
+// PATTERN-O0: @__const.test_bitfield_custom.custom = private unnamed_addr constant %struct.bitfield { i8 20, [3 x i8] c"\AA\AA\AA" }, align 4
+// ZERO-O0: @__const.test_bitfield_custom.custom = private unnamed_addr constant %struct.bitfield { i8 20, [3 x i8] zeroinitializer }, align 4
+// PATTERN-O1-NOT: @__const.test_bitfield_uninit.uninit
+// PATTERN-O1-NOT: @__const.test_bitfield_custom.custom
+// ZERO-O1-NOT: @__const.test_bitfield_custom.custom
 struct bitfield { int i : 4; int j : 2; };
-// PATTERN: @__const.test_bitfieldaligned_uninit.uninit = private unnamed_addr constant %struct.bitfieldaligned { i8 -86, [3 x i8] c"\AA\AA\AA", i8 -86, [3 x i8] c"\AA\AA\AA" }, align 4
-// PATTERN: @__const.test_bitfieldaligned_custom.custom = private unnamed_addr constant %struct.bitfieldaligned { i8 4, [3 x i8] c"\AA\AA\AA", i8 1, [3 x i8] c"\AA\AA\AA" }, align 4
-// ZERO: @__const.test_bitfieldaligned_custom.custom = private unnamed_addr constant %struct.bitfieldaligned { i8 4, [3 x i8] zeroinitializer, i8 1, [3 x i8] zeroinitializer }, align 4
+// PATTERN-O0: @__const.test_bitfieldaligned_uninit.uninit = private unnamed_addr constant %struct.bitfieldaligned { i8 -86, [3 x i8] c"\AA\AA\AA", i8 -86, [3 x i8] c"\AA\AA\AA" }, align 4
+// PATTERN-O0: @__const.test_bitfieldaligned_custom.custom = private unnamed_addr constant %struct.bitfieldaligned { i8 4, [3 x i8] c"\AA\AA\AA", i8 1, [3 x i8] c"\AA\AA\AA" }, align 4
+// ZERO-O0: @__const.test_bitfieldaligned_custom.custom = private unnamed_addr constant %struct.bitfieldaligned { i8 4, [3 x i8] zeroinitializer, i8 1, [3 x i8] zeroinitializer }, align 4
+// PATTERN-O1-NOT: @__const.test_bitfieldaligned_uninit.uninit
+// PATTERN-O1-NOT: @__const.test_bitfieldaligned_custom.custom
+// ZERO-O1-NOT: @__const.test_bitfieldaligned_custom.custom
 struct bitfieldaligned { int i : 4; int : 0; int j : 2; };
 struct big { unsigned a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t, u, v, w, x, y, z; };
-// PATTERN: @__const.test_arraytail_uninit.uninit = private unnamed_addr constant %struct.arraytail { i32 -1431655766, [0 x i32] zeroinitializer }, align 4
-// PATTERN: @__const.test_arraytail_custom.custom = private unnamed_addr constant %struct.arraytail { i32 57005, [0 x i32] zeroinitializer }, align 4
-// ZERO: @__const.test_arraytail_custom.custom = private unnamed_addr constant %struct.arraytail { i32 57005, [0 x i32] zeroinitializer }, align 4
+// PATTERN-O0: @__const.test_arraytail_uninit.uninit = private unnamed_addr constant %struct.arraytail { i32 -1431655766, [0 x i32] zeroinitializer }, align 4
+// PATTERN-O0: @__const.test_arraytail_custom.custom = private unnamed_addr constant %struct.arraytail { i32 57005, [0 x i32] zeroinitializer }, align 4
+// ZERO-O0: @__const.test_arraytail_custom.custom = private unnamed_addr constant %struct.arraytail { i32 57005, [0 x i32] zeroinitializer }, align 4
+// PATTERN-O1-NOT: @__const.test_arraytail_uninit.uninit
+// PATTERN-O1-NOT: @__const.test_arraytail_custom.custom
+// ZERO-O1-NOT: @__const.test_arraytail_custom.custom
 struct arraytail { int i; int arr[]; };
-// PATTERN: @__const.test_int1_uninit.uninit = private unnamed_addr constant [1 x i32] [i32 -1431655766], align 4
-// PATTERN: @__const.test_int1_custom.custom = private unnamed_addr constant [1 x i32] [i32 858993459], align 4
-// ZERO: @__const.test_int1_custom.custom = private unnamed_addr constant [1 x i32] [i32 858993459], align 4
-// PATTERN: @__const.test_bool4_uninit.uninit = private unnamed_addr constant [4 x i8] c"\AA\AA\AA\AA", align 1
-// PATTERN: @__const.test_bool4_custom.custom = private unnamed_addr constant [4 x i8] c"\01\01\01\01", align 1
-// ZERO: @__const.test_bool4_custom.custom = private unnamed_addr constant [4 x i8] c"\01\01\01\01", align 1
+// PATTERN-O0: @__const.test_int1_uninit.uninit = private unnamed_addr constant [1 x i32] [i32 -1431655766], align 4
+// PATTERN-O0: @__const.test_int1_custom.custom = private unnamed_addr constant [1 x i32] [i32 858993459], align 4
+// ZERO-O0: @__const.test_int1_custom.custom = private unnamed_addr constant [1 x i32] [i32 858993459], align 4
+// PATTERN-O1-NOT: @__const.test_int1_uninit.uninit
+// PATTERN-O1-NOT: @__const.test_int1_custom.custom
+// ZERO-O1-NOT: @__const.test_int1_custom.custom
+
+// PATTERN-O0: @__const.test_bool4_uninit.uninit = private unnamed_addr constant [4 x i8] c"\AA\AA\AA\AA", align 1
+// PATTERN-O0: @__const.test_bool4_custom.custom = private unnamed_addr constant [4 x i8] c"\01\01\01\01", align 1
+// ZERO-O0: @__const.test_bool4_custom.custom = private unnamed_addr constant [4 x i8] c"\01\01\01\01", align 1
+// PATTERN-O1-NOT: @__const.test_bool4_uninit.uninit
+// PATTERN-O1-NOT: @__const.test_bool4_custom.custom
+// ZERO-O1-NOT: @__const.test_bool4_custom.custom
+
 // PATTERN: @__const.test_intptr4_uninit.uninit = private unnamed_addr constant [4 x i32*] [i32* inttoptr (i64 -6148914691236517206 to i32*), i32* inttoptr (i64 -6148914691236517206 to i32*), i32* inttoptr (i64 -6148914691236517206 to i32*), i32* inttoptr (i64 -6148914691236517206 to i32*)], align 16
 // PATTERN: @__const.test_intptr4_custom.custom = private unnamed_addr constant [4 x i32*] [i32* inttoptr (i64 572662306 to i32*), i32* inttoptr (i64 572662306 to i32*), i32* inttoptr (i64 572662306 to i32*), i32* inttoptr (i64 572662306 to i32*)], align 16
 // ZERO: @__const.test_intptr4_custom.custom = private unnamed_addr constant [4 x i32*] [i32* inttoptr (i64 572662306 to i32*), i32* inttoptr (i64 572662306 to i32*), i32* inttoptr (i64 572662306 to i32*), i32* inttoptr (i64 572662306 to i32*)], align 16
-// PATTERN: @__const.test_tailpad4_uninit.uninit = private unnamed_addr constant [4 x { i16, i8, [1 x i8] }] [{ i16, i8, [1 x i8] } { i16 -21846, i8 -86, [1 x i8] c"\AA" }, { i16, i8, [1 x i8] } { i16 -21846, i8 -86, [1 x i8] c"\AA" }, { i16, i8, [1 x i8] } { i16 -21846, i8 -86, [1 x i8] c"\AA" }, { i16, i8, [1 x i8] } { i16 -21846, i8 -86, [1 x i8] c"\AA" }], align 16
+// PATTERN-O0: @__const.test_tailpad4_uninit.uninit = private unnamed_addr constant [4 x { i16, i8, [1 x i8] }] [{ i16, i8, [1 x i8] } { i16 -21846, i8 -86, [1 x i8] c"\AA" }, { i16, i8, [1 x i8] } { i16 -21846, i8 -86, [1 x i8] c"\AA" }, { i16, i8, [1 x i8] } { i16 -21846, i8 -86, [1 x i8] c"\AA" }, { i16, i8, [1 x i8] } { i16 -21846, i8 -86, [1 x i8] c"\AA" }], align 16
+// PATTERN-O1-NOT: @__const.test_tailpad4_uninit.uninit
 // PATTERN: @__const.test_tailpad4_custom.custom = private unnamed_addr constant [4 x { i16, i8, [1 x i8] }] [{ i16, i8, [1 x i8] } { i16 257, i8 1, [1 x i8] c"\AA" }, { i16, i8, [1 x i8] } { i16 257, i8 1, [1 x i8] c"\AA" }, { i16, i8, [1 x i8] } { i16 257, i8 1, [1 x i8] c"\AA" }, { i16, i8, [1 x i8] } { i16 257, i8 1, [1 x i8] c"\AA" }], align 16
 // ZERO: @__const.test_tailpad4_custom.custom = private unnamed_addr constant [4 x { i16, i8, [1 x i8] }] [{ i16, i8, [1 x i8] } { i16 257, i8 1, [1 x i8] zeroinitializer }, { i16, i8, [1 x i8] } { i16 257, i8 1, [1 x i8] zeroinitializer }, { i16, i8, [1 x i8] } { i16 257, i8 1, [1 x i8] zeroinitializer }, { i16, i8, [1 x i8] } { i16 257, i8 1, [1 x i8] zeroinitializer }], align 16
 struct tailpad { short s; char c; };
-// PATTERN: @__const.test_atomicnotlockfree_uninit.uninit = private unnamed_addr constant %struct.notlockfree { [4 x i64] [i64 -6148914691236517206, i64 -6148914691236517206, i64 -6148914691236517206, i64 -6148914691236517206] }, align 8
+// PATTERN-O0: @__const.test_atomicnotlockfree_uninit.uninit = private unnamed_addr constant %struct.notlockfree { [4 x i64] [i64 -6148914691236517206, i64 -6148914691236517206, i64 -6148914691236517206, i64 -6148914691236517206] }, align 8
+// PATTERN-O1-NOT: @__const.test_atomicnotlockfree_uninit.uninit
 struct notlockfree { long long a[4]; };
-// PATTERN: @__const.test_atomicpadded_uninit.uninit = private unnamed_addr constant { i8, [3 x i8], i32 } { i8 -86, [3 x i8] c"\AA\AA\AA", i32 -1431655766 }, align 8
-// PATTERN: @__const.test_atomictailpad_uninit.uninit = private unnamed_addr constant { i16, i8, [1 x i8] } { i16 -21846, i8 -86, [1 x i8] c"\AA" }, align 4
-// PATTERN: @__const.test_complexfloat_uninit.uninit = private unnamed_addr constant { float, float } { float 0xFFFFFFFFE0000000, float 0xFFFFFFFFE0000000 }, align 4
-// PATTERN: @__const.test_complexfloat_braces.braces = private unnamed_addr constant { float, float } { float 0xFFFFFFFFE0000000, float 0xFFFFFFFFE0000000 }, align 4
-// PATTERN: @__const.test_complexfloat_custom.custom = private unnamed_addr constant { float, float } { float 0xFFFFFFFFE0000000, float 0xFFFFFFFFE0000000 }, align 4
-// PATTERN: @__const.test_complexdouble_uninit.uninit = private unnamed_addr constant { double, double } { double 0xFFFFFFFFFFFFFFFF, double 0xFFFFFFFFFFFFFFFF }, align 8
-// PATTERN: @__const.test_complexdouble_braces.braces = private unnamed_addr constant { double, double } { double 0xFFFFFFFFFFFFFFFF, double 0xFFFFFFFFFFFFFFFF }, align 8
-// PATTERN: @__const.test_complexdouble_custom.custom = private unnamed_addr constant { double, double } { double 0xFFFFFFFFFFFFFFFF, double 0xFFFFFFFFFFFFFFFF }, align 8
-// PATTERN: @__const.test_semivolatile_uninit.uninit = private unnamed_addr constant %struct.semivolatile { i32 -1431655766, i32 -1431655766 }, align 4
-// PATTERN: @__const.test_semivolatile_custom.custom = private unnamed_addr constant %struct.semivolatile { i32 1145324612, i32 1145324612 }, align 4
+// PATTERN-O0: @__const.test_atomicpadded_uninit.uninit = private unnamed_addr constant { i8, [3 x i8], i32 } { i8 -86, [3 x i8] c"\AA\AA\AA", i32 -1431655766 }, align 8
+// PATTERN-O1-NOT: @__const.test_atomicpadded_uninit.uninit
+// PATTERN-O0: @__const.test_atomictailpad_uninit.uninit = private unnamed_addr constant { i16, i8, [1 x i8] } { i16 -21846, i8 -86, [1 x i8] c"\AA" }, align 4
+// PATTERN-O1-NOT: @__const.test_atomictailpad_uninit.uninit
+// PATTERN-O0: @__const.test_complexfloat_uninit.uninit = private unnamed_addr constant { float, float } { float 0xFFFFFFFFE0000000, float 0xFFFFFFFFE0000000 }, align 4
+// PATTERN-O1-NOT: @__const.test_complexfloat_uninit.uninit
+// PATTERN-O0: @__const.test_complexfloat_braces.braces = private unnamed_addr constant { float, float } { float 0xFFFFFFFFE0000000, float 0xFFFFFFFFE0000000 }, align 4
+// PATTERN-O1-NOT: @__const.test_complexfloat_braces.braces
+// PATTERN-O0: @__const.test_complexfloat_custom.custom = private unnamed_addr constant { float, float } { float 0xFFFFFFFFE0000000, float 0xFFFFFFFFE0000000 }, align 4
+// PATTERN-O1-NOT: @__const.test_complexfloat_custom.custom
+// PATTERN-O0: @__const.test_complexdouble_uninit.uninit = private unnamed_addr constant { double, double } { double 0xFFFFFFFFFFFFFFFF, double 0xFFFFFFFFFFFFFFFF }, align 8
+// PATTERN-O1-NOT: @__const.test_complexdouble_uninit.uninit
+// PATTERN-O0: @__const.test_complexdouble_braces.braces = private unnamed_addr constant { double, double } { double 0xFFFFFFFFFFFFFFFF, double 0xFFFFFFFFFFFFFFFF }, align 8
+// PATTERN-O1-NOT: @__const.test_complexdouble_braces.braces
+// PATTERN-O0: @__const.test_complexdouble_custom.custom = private unnamed_addr constant { double, double } { double 0xFFFFFFFFFFFFFFFF, double 0xFFFFFFFFFFFFFFFF }, align 8
+// PATTERN-O1-NOT: @__const.test_complexdouble_custom.custom
+// PATTERN-O0: @__const.test_semivolatile_uninit.uninit = private unnamed_addr constant %struct.semivolatile { i32 -1431655766, i32 -1431655766 }, align 4
+// PATTERN-O0: @__const.test_semivolatile_custom.custom = private unnamed_addr constant %struct.semivolatile { i32 1145324612, i32 1145324612 }, align 4
+// PATTERN-O1-NOT: @__const.test_semivolatile_custom.custom
 struct semivolatile { int i; volatile int vi; };
-// PATTERN: @__const.test_semivolatileinit_uninit.uninit = private unnamed_addr constant %struct.semivolatileinit { i32 -1431655766, i32 -1431655766 }, align 4
-// PATTERN: @__const.test_semivolatileinit_braces.braces = private unnamed_addr constant %struct.semivolatileinit { i32 -1431655766, i32 -1431655766 }, align 4
-// PATTERN: @__const.test_semivolatileinit_custom.custom = private unnamed_addr constant %struct.semivolatileinit { i32 -1431655766, i32 -1431655766 }, align 4
-// ZERO: @__const.test_semivolatile_custom.custom = private unnamed_addr constant %struct.semivolatile { i32 1145324612, i32 1145324612 }, align 4
+// PATTERN-O0: @__const.test_semivolatileinit_uninit.uninit = private unnamed_addr constant %struct.semivolatileinit { i32 -1431655766, i32 -1431655766 }, align 4
+// PATTERN-O1-NOT: @__const.test_semivolatileinit_uninit.uninit
+// PATTERN-O0: @__const.test_semivolatileinit_braces.braces = private unnamed_addr constant %struct.semivolatileinit { i32 -1431655766, i32 -1431655766 }, align 4
+// PATTERN-O1-NOT: @__const.test_semivolatileinit_braces.braces
+// PATTERN-O0: @__const.test_semivolatileinit_custom.custom = private unnamed_addr constant %struct.semivolatileinit { i32 -1431655766, i32 -1431655766 }, align 4
+// PATTERN-O1-NOT: @__const.test_semivolatileinit_custom.custom = private unnamed_addr constant %struct.semivolatileinit { i32 -1431655766, i32 -1431655766 }, align 4
+// ZERO-O0: @__const.test_semivolatile_custom.custom = private unnamed_addr constant %struct.semivolatile { i32 1145324612, i32 1145324612 }, align 4
+// ZERO-O1-NOT: @__const.test_semivolatile_custom.custom
 struct semivolatileinit { int i = 0x11111111; volatile int vi = 0x11111111; };
-// PATTERN: @__const.test_base_uninit.uninit = private unnamed_addr constant %struct.base { i32 (...)** inttoptr (i64 -6148914691236517206 to i32 (...)**) }, align 8
-// PATTERN: @__const.test_base_braces.braces = private unnamed_addr constant %struct.base { i32 (...)** inttoptr (i64 -6148914691236517206 to i32 (...)**) }, align 8
+// PATTERN-O0: @__const.test_base_uninit.uninit = private unnamed_addr constant %struct.base { i32 (...)** inttoptr (i64 -6148914691236517206 to i32 (...)**) }, align 8
+// PATTERN-O1-NOT: @__const.test_base_uninit.uninit
+// PATTERN-O0: @__const.test_base_braces.braces = private unnamed_addr constant %struct.base { i32 (...)** inttoptr (i64 -6148914691236517206 to i32 (...)**) }, align 8
+// PATTERN-O1-NOT: @__const.test_base_braces.braces
 struct base { virtual ~base(); };
-// PATTERN: @__const.test_derived_uninit.uninit = private unnamed_addr constant %struct.derived { %struct.base { i32 (...)** inttoptr (i64 -6148914691236517206 to i32 (...)**) } }, align 8
-// PATTERN: @__const.test_derived_braces.braces = private unnamed_addr constant %struct.derived { %struct.base { i32 (...)** inttoptr (i64 -6148914691236517206 to i32 (...)**) } }, align 8
+// PATTERN-O0: @__const.test_derived_uninit.uninit = private unnamed_addr constant %struct.derived { %struct.base { i32 (...)** inttoptr (i64 -6148914691236517206 to i32 (...)**) } }, align 8
+// PATTERN-O1-NOT: @__const.test_derived_uninit.uninit
+// PATTERN-O0: @__const.test_derived_braces.braces = private unnamed_addr constant %struct.derived { %struct.base { i32 (...)** inttoptr (i64 -6148914691236517206 to i32 (...)**) } }, align 8
+// PATTERN-O1-NOT: @__const.test_derived_braces.braces
 struct derived : public base {};
-// PATTERN: @__const.test_virtualderived_uninit.uninit = private unnamed_addr constant %struct.virtualderived { %struct.base { i32 (...)** inttoptr (i64 -6148914691236517206 to i32 (...)**) }, %struct.derived { %struct.base { i32 (...)** inttoptr (i64 -6148914691236517206 to i32 (...)**) } } }, align 8
-// PATTERN: @__const.test_virtualderived_braces.braces = private unnamed_addr constant %struct.virtualderived { %struct.base { i32 (...)** inttoptr (i64 -6148914691236517206 to i32 (...)**) }, %struct.derived { %struct.base { i32 (...)** inttoptr (i64 -6148914691236517206 to i32 (...)**) } } }, align 8
+// PATTERN-O0: @__const.test_virtualderived_uninit.uninit = private unnamed_addr constant %struct.virtualderived { %struct.base { i32 (...)** inttoptr (i64 -6148914691236517206 to i32 (...)**) }, %struct.derived { %struct.base { i32 (...)** inttoptr (i64 -6148914691236517206 to i32 (...)**) } } }, align 8
+// PATTERN-O1-NOT: @__const.test_virtualderived_uninit.uninit
+// PATTERN-O0: @__const.test_virtualderived_braces.braces = private unnamed_addr constant %struct.virtualderived { %struct.base { i32 (...)** inttoptr (i64 -6148914691236517206 to i32 (...)**) }, %struct.derived { %struct.base { i32 (...)** inttoptr (i64 -6148914691236517206 to i32 (...)**) } } }, align 8
+// PATTERN-O1-NOT: @__const.test_virtualderived_braces.braces
 struct virtualderived : public virtual base, public virtual derived {};
-// PATTERN: @__const.test_matching_uninit.uninit = private unnamed_addr constant %union.matching { i32 -1431655766 }, align 4
-// PATTERN: @__const.test_matching_custom.custom = private unnamed_addr constant { float } { float 6.145500e+04 }, align 4
-// ZERO: @__const.test_matching_custom.custom = private unnamed_addr constant { float } { float 6.145500e+04 }, align 4
+// PATTERN-O0: @__const.test_matching_uninit.uninit = private unnamed_addr constant %union.matching { i32 -1431655766 }, align 4
+// PATTERN-O1-NOT: @__const.test_matching_uninit.uninit
+// PATTERN-O0: @__const.test_matching_custom.custom = private unnamed_addr constant { float } { float 6.145500e+04 }, align 4
+// PATTERN-O1-NOT: @__const.test_matching_custom.custom
+// ZERO-O0: @__const.test_matching_custom.custom = private unnamed_addr constant { float } { float 6.145500e+04 }, align 4
+// ZERO-O1-NOT: @__const.test_matching_custom.custom
 union matching { int i; float f; };
-// PATTERN: @__const.test_matchingreverse_uninit.uninit = private unnamed_addr constant %union.matchingreverse { float 0xFFFFFFFFE0000000 }, align 4
-// PATTERN: @__const.test_matchingreverse_custom.custom = private unnamed_addr constant { i32 } { i32 61455 }, align 4
-// ZERO: @__const.test_matchingreverse_custom.custom = private unnamed_addr constant { i32 } { i32 61455 }, align 4
+// PATTERN-O0: @__const.test_matchingreverse_uninit.uninit = private unnamed_addr constant %union.matchingreverse { float 0xFFFFFFFFE0000000 }, align 4
+// PATTERN-O1-NOT: @__const.test_matchingreverse_uninit.uninit
+// PATTERN-O0: @__const.test_matchingreverse_custom.custom = private unnamed_addr constant { i32 } { i32 61455 }, align 4
+// PATTERN-O1-NOT: @__const.test_matchingreverse_custom.custom
+// ZERO-O0: @__const.test_matchingreverse_custom.custom = private unnamed_addr constant { i32 } { i32 61455 }, align 4
+// ZERO-O1-NOT: @__const.test_matchingreverse_custom.custom
 union matchingreverse { float f; int i; };
-// PATTERN: @__const.test_unmatched_uninit.uninit = private unnamed_addr constant %union.unmatched { i32 -1431655766 }, align 4
-// PATTERN: @__const.test_unmatched_custom.custom = private unnamed_addr constant %union.unmatched { i32 1001242351 }, align 4
-// ZERO: @__const.test_unmatched_custom.custom = private unnamed_addr constant %union.unmatched { i32 1001242351 }, align 4
+// PATTERN-O0: @__const.test_unmatched_uninit.uninit = private unnamed_addr constant %union.unmatched { i32 -1431655766 }, align 4
+// PATTERN-O1-NOT: @__const.test_unmatched_uninit.uninit
+// PATTERN-O0: @__const.test_unmatched_custom.custom = private unnamed_addr constant %union.unmatched { i32 1001242351 }, align 4
+// PATTERN-O1-NOT: @__const.test_unmatched_custom.custom
+// ZERO-O0: @__const.test_unmatched_custom.custom = private unnamed_addr constant %union.unmatched { i32 1001242351 }, align 4
+// ZERO-O1-NOT: @__const.test_unmatched_custom.custom
 union unmatched { char c; int i; };
-// PATTERN: @__const.test_unmatchedreverse_uninit.uninit = private unnamed_addr constant %union.unmatchedreverse { i32 -1431655766 }, align 4
-// PATTERN: @__const.test_unmatchedreverse_custom.custom = private unnamed_addr constant { i8, [3 x i8] } { i8 42, [3 x i8] c"\AA\AA\AA" }, align 4
-// ZERO: @__const.test_unmatchedreverse_custom.custom = private unnamed_addr constant { i8, [3 x i8] } { i8 42, [3 x i8] zeroinitializer }, align 4
+// PATTERN-O0: @__const.test_unmatchedreverse_uninit.uninit = private unnamed_addr constant %union.unmatchedreverse { i32 -1431655766 }, align 4
+// PATTERN-O1-NOT: @__const.test_unmatchedreverse_uninit.uninit
+// PATTERN-O0: @__const.test_unmatchedreverse_custom.custom = private unnamed_addr constant { i8, [3 x i8] } { i8 42, [3 x i8] c"\AA\AA\AA" }, align 4
+// PATTERN-O1-NOT: @__const.test_unmatchedreverse_custom.custom
+// ZERO-O0: @__const.test_unmatchedreverse_custom.custom = private unnamed_addr constant { i8, [3 x i8] } { i8 42, [3 x i8] zeroinitializer }, align 4
+// ZERO-O1-NOT: @__const.test_unmatchedreverse_custom.custom
 union unmatchedreverse { int i; char c; };
-// PATTERN: @__const.test_unmatchedfp_uninit.uninit = private unnamed_addr constant %union.unmatchedfp { double 0xFFFFFFFFFFFFFFFF }, align 8
-// PATTERN: @__const.test_unmatchedfp_custom.custom = private unnamed_addr constant %union.unmatchedfp { double 0x400921FB54442D18 }, align 8
-// ZERO: @__const.test_unmatchedfp_custom.custom = private unnamed_addr constant %union.unmatchedfp { double 0x400921FB54442D18 }, align 8
+// PATTERN-O0: @__const.test_unmatchedfp_uninit.uninit = private unnamed_addr constant %union.unmatchedfp { double 0xFFFFFFFFFFFFFFFF }, align 8
+// PATTERN-O1-NOT: @__const.test_unmatchedfp_uninit.uninit
+// PATTERN-O0: @__const.test_unmatchedfp_custom.custom = private unnamed_addr constant %union.unmatchedfp { double 0x400921FB54442D18 }, align 8
+// PATTERN-O1-NOT: @__const.test_unmatchedfp_custom.custom
+// ZERO-O0: @__const.test_unmatchedfp_custom.custom = private unnamed_addr constant %union.unmatchedfp { double 0x400921FB54442D18 }, align 8
+// ZERO-O1-NOT: @__const.test_unmatchedfp_custom.custom
 union unmatchedfp { float f; double d; };
 enum emptyenum {};
 enum smallenum { VALUE };
 
 extern "C" {
 
 TEST_UNINIT(char, char);
 // CHECK-LABEL: @test_char_uninit()

 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 
 TEST_UNINIT(empty, empty);
 // CHECK-LABEL: @test_empty_uninit()
 // CHECK:       %uninit = alloca %struct.empty, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_empty_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_empty_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_empty_uninit.uninit
+// PATTERN-O1: store i8 -86, {{.*}} align 1
 // ZERO-LABEL: @test_empty_uninit()
-// ZERO: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O1: store i8 0, {{.*}} align 1
 
 TEST_BRACES(empty, empty);
 // CHECK-LABEL: @test_empty_braces()
 // CHECK:       %braces = alloca %struct.empty, align
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memcpy
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 TEST_UNINIT(small, small);
 // CHECK-LABEL: @test_small_uninit()
 // CHECK:       %uninit = alloca %struct.small, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_small_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_small_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_small_uninit.uninit
+// PATTERN-O1: store i8 -86, {{.*}} align 1
 // ZERO-LABEL: @test_small_uninit()
-// ZERO: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O1: store i8 0, {{.*}} align 1
 
 TEST_BRACES(small, small);
 // CHECK-LABEL: @test_small_braces()
 // CHECK:       %braces = alloca %struct.small, align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 1, i1 false)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 

 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%custom)
 
 TEST_UNINIT(smallpartinit, smallpartinit);
 // CHECK-LABEL: @test_smallpartinit_uninit()
 // CHECK:       %uninit = alloca %struct.smallpartinit, align
 // CHECK-NEXT:  call void @{{.*}}smallpartinit{{.*}}%uninit)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_smallpartinit_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_smallpartinit_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_smallpartinit_uninit.uninit
+// PATTERN-O1: store i8 -86, {{.*}} align 1
+// PATTERN-O1: store i8 42, {{.*}} align 1
 // ZERO-LABEL: @test_smallpartinit_uninit()
-// ZERO: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O1: store i16 0, i16* %uninit, align 2
 
 TEST_BRACES(smallpartinit, smallpartinit);
 // CHECK-LABEL: @test_smallpartinit_braces()
 // CHECK:       %braces = alloca %struct.smallpartinit, align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  %[[C:[^ ]*]] = getelementptr inbounds %struct.smallpartinit, %struct.smallpartinit* %braces, i32 0, i32 0
 // CHECK-NEXT:  store i8 42, i8* %[[C]], align [[ALIGN]]
 // CHECK-NEXT:  %[[D:[^ ]*]] = getelementptr inbounds %struct.smallpartinit, %struct.smallpartinit* %braces, i32 0, i32 1
 // CHECK-NEXT:  store i8 0, i8* %[[D]], align [[ALIGN]]

 // CHECK-NEXT:  store i8* getelementptr inbounds {{.*}}, i8** %[[N]], align [[ALIGN]]
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%custom)
 
 TEST_UNINIT(padded, padded);
 // CHECK-LABEL: @test_padded_uninit()
 // CHECK:       %uninit = alloca %struct.padded, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_padded_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_padded_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_padded_uninit.uninit
+// PATTERN-O1: store i64 -6148914691236517206, i64* %uninit, align 8
 // ZERO-LABEL: @test_padded_uninit()
-// ZERO: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O1: store i64 0, i64* %uninit, align 8
 
 TEST_BRACES(padded, padded);
 // CHECK-LABEL: @test_padded_braces()
 // CHECK:       %braces = alloca %struct.padded, align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 8, i1 false)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 TEST_CUSTOM(padded, padded, { 42, 13371337 });
 // CHECK-LABEL: @test_padded_custom()
 // CHECK:       %custom = alloca %struct.padded, align
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memcpy
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%custom)
 
 TEST_UNINIT(paddednullinit, paddednullinit);
 // CHECK-LABEL: @test_paddednullinit_uninit()
 // CHECK:       %uninit = alloca %struct.paddednullinit, align
 // CHECK-NEXT:  call void @{{.*}}paddednullinit{{.*}}%uninit)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_paddednullinit_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_paddednullinit_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_paddednullinit_uninit.uninit
+// PATTERN-O1: store i64 -6148914691236517206, i64* %uninit, align 8
 // ZERO-LABEL: @test_paddednullinit_uninit()
-// ZERO: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O1: store i64 0, i64* %uninit, align 8
 
 TEST_BRACES(paddednullinit, paddednullinit);
 // CHECK-LABEL: @test_paddednullinit_braces()
 // CHECK:       %braces = alloca %struct.paddednullinit, align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  %[[C:[^ ]*]] = getelementptr inbounds %struct.paddednullinit, %struct.paddednullinit* %braces, i32 0, i32 0
 // CHECK-NEXT:  store i8 0, i8* %[[C]], align [[ALIGN]]
 // CHECK-NEXT:  %[[I:[^ ]*]] = getelementptr inbounds %struct.paddednullinit, %struct.paddednullinit* %braces, i32 0, i32 1
 // CHECK-NEXT:  store i32 0, i32* %[[I]], align [[ALIGN]]
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 TEST_CUSTOM(paddednullinit, paddednullinit, { 42, 13371337 });
 // CHECK-LABEL: @test_paddednullinit_custom()
 // CHECK:       %custom = alloca %struct.paddednullinit, align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  %[[C:[^ ]*]] = getelementptr inbounds %struct.paddednullinit, %struct.paddednullinit* %custom, i32 0, i32 0
 // CHECK-NEXT:  store i8 42, i8* %[[C]], align [[ALIGN]]
 // CHECK-NEXT:  %[[I:[^ ]*]] = getelementptr inbounds %struct.paddednullinit, %struct.paddednullinit* %custom, i32 0, i32 1
 // CHECK-NEXT:  store i32 13371337, i32* %[[I]], align [[ALIGN]]
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%custom)
 
 TEST_UNINIT(paddedpacked, paddedpacked);
 // CHECK-LABEL: @test_paddedpacked_uninit()
 // CHECK:       %uninit = alloca %struct.paddedpacked, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_paddedpacked_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_paddedpacked_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_paddedpacked_uninit.uninit
+// PATTERN-O1:  %[[C:[^ ]*]] = getelementptr inbounds {{.*}}%uninit, i64 0, i32 0
+// PATTERN-O1  store i8 -86, i8* %[[C]], align
+// PATTERN-O1:  %[[I:[^ ]*]] = getelementptr inbounds {{.*}}%uninit, i64 0, i32 1
+// PATTERN-O1: store i32 -1431655766, i32* %[[I]], align
+
 // ZERO-LABEL: @test_paddedpacked_uninit()
 // ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(paddedpacked, paddedpacked);
 // CHECK-LABEL: @test_paddedpacked_braces()
 // CHECK:       %braces = alloca %struct.paddedpacked, align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 5, i1 false)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 TEST_CUSTOM(paddedpacked, paddedpacked, { 42, 13371337 });
 // CHECK-LABEL: @test_paddedpacked_custom()
 // CHECK:       %custom = alloca %struct.paddedpacked, align
 // CHECK-NEXT:  bitcast
-// CHECK-NEXT:  call void @llvm.memcpy
+// CHECK-NEXT:  call void @llvm.memcpy{{.*}}({{.*}}@__const.test_paddedpacked_custom.custom
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%custom)
 
 TEST_UNINIT(paddedpackedarray, paddedpackedarray);
 // CHECK-LABEL: @test_paddedpackedarray_uninit()
 // CHECK:       %uninit = alloca %struct.paddedpackedarray, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_paddedpackedarray_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_paddedpackedarray_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_paddedpackedarray_uninit.uninit
+// PATTERN-O1: getelementptr
+// PATTERN-O1: call void @llvm.memset{{.*}}({{.*}}i8 -86, i64 10
 // ZERO-LABEL: @test_paddedpackedarray_uninit()
 // ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(paddedpackedarray, paddedpackedarray);
 // CHECK-LABEL: @test_paddedpackedarray_braces()
 // CHECK:       %braces = alloca %struct.paddedpackedarray, align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 10, i1 false)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 TEST_CUSTOM(paddedpackedarray, paddedpackedarray, { {{ 42, 13371337 }, { 43, 13371338 }} });
 // CHECK-LABEL: @test_paddedpackedarray_custom()
 // CHECK:       %custom = alloca %struct.paddedpackedarray, align
 // CHECK-NEXT:  bitcast
-// CHECK-NEXT:  call void @llvm.memcpy
+// CHECK-NEXT:  call void @llvm.memcpy{{.*}}({{.*}}@__const.test_paddedpackedarray_custom.custom
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%custom)
 
 TEST_UNINIT(paddednested, paddednested);
 // CHECK-LABEL: @test_paddednested_uninit()
 // CHECK:       %uninit = alloca %struct.paddednested, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_paddednested_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_paddednested_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_paddednested_uninit.uninit
+// PATTERN-O1: getelementptr
+// PATTERN-O1: call void @llvm.memset{{.*}}({{.*}}, i8 -86, i64 16
 // ZERO-LABEL: @test_paddednested_uninit()
 // ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(paddednested, paddednested);
 // CHECK-LABEL: @test_paddednested_braces()
 // CHECK:       %braces = alloca %struct.paddednested, align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 16, i1 false)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 TEST_CUSTOM(paddednested, paddednested, { { 42, 13371337 }, { 43, 13371338 } });
 // CHECK-LABEL: @test_paddednested_custom()
 // CHECK:       %custom = alloca %struct.paddednested, align
 // CHECK-NEXT:  bitcast
-// CHECK-NEXT:  call void @llvm.memcpy
+// CHECK-NEXT:  call void @llvm.memcpy{{.*}}({{.*}}@__const.test_paddednested_custom.custom
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%custom)
 
 TEST_UNINIT(paddedpackednested, paddedpackednested);
 // CHECK-LABEL: @test_paddedpackednested_uninit()
 // CHECK:       %uninit = alloca %struct.paddedpackednested, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_paddedpackednested_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_paddedpackednested_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_paddedpackednested_uninit.uninit
+// PATTERN-O1: getelementptr
+// PATTERN-O1: call void @llvm.memset.p0i8.i64(i8* nonnull align 1 %0, i8 -86, i64 10, i1 false
 // ZERO-LABEL: @test_paddedpackednested_uninit()
 // ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(paddedpackednested, paddedpackednested);
 // CHECK-LABEL: @test_paddedpackednested_braces()
 // CHECK:       %braces = alloca %struct.paddedpackednested, align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 10, i1 false)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 TEST_CUSTOM(paddedpackednested, paddedpackednested, { { 42, 13371337 }, { 43, 13371338 } });
 // CHECK-LABEL: @test_paddedpackednested_custom()
 // CHECK:       %custom = alloca %struct.paddedpackednested, align
 // CHECK-NEXT:  bitcast
-// CHECK-NEXT:  call void @llvm.memcpy
+// CHECK-NEXT:  call void @llvm.memcpy{{.*}}({{.*}}@__const.test_paddedpackednested_custom.custom
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%custom)
 
 TEST_UNINIT(bitfield, bitfield);
 // CHECK-LABEL: @test_bitfield_uninit()
 // CHECK:       %uninit = alloca %struct.bitfield, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_bitfield_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_bitfield_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_bitfield_uninit.uninit
+// PATTERN-O1: store i32 -1431655766, i32* %uninit, align 4
 // ZERO-LABEL: @test_bitfield_uninit()
-// ZERO: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O1: store i32 0, i32* %uninit, align 4
 
 TEST_BRACES(bitfield, bitfield);
 // CHECK-LABEL: @test_bitfield_braces()
 // CHECK:       %braces = alloca %struct.bitfield, align
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memcpy
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 TEST_CUSTOM(bitfield, bitfield, { 4, 1 });
 // CHECK-LABEL: @test_bitfield_custom()
 // CHECK:       %custom = alloca %struct.bitfield, align
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memcpy
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%custom)
 
 TEST_UNINIT(bitfieldaligned, bitfieldaligned);
 // CHECK-LABEL: @test_bitfieldaligned_uninit()
 // CHECK:       %uninit = alloca %struct.bitfieldaligned, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_bitfieldaligned_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_bitfieldaligned_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_bitfieldaligned_uninit.uninit
+// PATTERN-O1: store i64 -6148914691236517206, i64* %uninit, align 8
 // ZERO-LABEL: @test_bitfieldaligned_uninit()
-// ZERO: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O1: store i64 0, i64* %uninit, align 8
 
 TEST_BRACES(bitfieldaligned, bitfieldaligned);
 // CHECK-LABEL: @test_bitfieldaligned_braces()
 // CHECK:       %braces = alloca %struct.bitfieldaligned, align
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memcpy
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 

 // CHECK-NEXT:  call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 -86, i64 104, i1 false)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%custom)
 
 TEST_UNINIT(arraytail, arraytail);
 // CHECK-LABEL: @test_arraytail_uninit()
 // CHECK:       %uninit = alloca %struct.arraytail, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_arraytail_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_arraytail_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_arraytail_uninit.uninit
+// PATTERN-O1: store i32 -1431655766, {{.*}} align 4
 // ZERO-LABEL: @test_arraytail_uninit()
-// ZERO: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O1: store i32 0, {{.*}} align 4
 
 TEST_BRACES(arraytail, arraytail);
 // CHECK-LABEL: @test_arraytail_braces()
 // CHECK:       %braces = alloca %struct.arraytail, align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 4, i1 false)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 TEST_CUSTOM(arraytail, arraytail, { 0xdead });
 // CHECK-LABEL: @test_arraytail_custom()
 // CHECK:       %custom = alloca %struct.arraytail, align
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memcpy
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%custom)
 
 
 TEST_UNINIT(int0, int[0]);
 // CHECK-LABEL: @test_int0_uninit()
 // CHECK:       %uninit = alloca [0 x i32], align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_int0_uninit()
 // PATTERN:       %uninit = alloca [0 x i32], align
-// PATTERN-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-O0-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // ZERO-LABEL: @test_int0_uninit()
 // ZERO:       %uninit = alloca [0 x i32], align
-// ZERO-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// ZERO-O0-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 
 TEST_BRACES(int0, int[0]);
 // CHECK-LABEL: @test_int0_braces()
 // CHECK:       %braces = alloca [0 x i32], align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  bitcast
-// CHECK-NEXT:  call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 0, i1 false)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 TEST_UNINIT(int1, int[1]);
 // CHECK-LABEL: @test_int1_uninit()
 // CHECK:       %uninit = alloca [1 x i32], align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_int1_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_int1_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_int1_uninit.uninit
+// PATTERN-O1: store i32 -1431655766, {{.*}} align 4
 // ZERO-LABEL: @test_int1_uninit()
-// ZERO: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O1: store i32 0, {{.*}} align 4
 
 TEST_BRACES(int1, int[1]);
 // CHECK-LABEL: @test_int1_braces()
 // CHECK:       %braces = alloca [1 x i32], align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 4, i1 false)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 

 // CHECK-NEXT:  call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 17, i64 256, i1 false)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%custom)
 
 TEST_UNINIT(bool4, bool[4]);
 // CHECK-LABEL: @test_bool4_uninit()
 // CHECK:       %uninit = alloca [4 x i8], align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_bool4_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_bool4_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_bool4_uninit.uninit
+// PATTERN-O1: store i32 -1431655766, i32* %uninit, align 4
 // ZERO-LABEL: @test_bool4_uninit()
-// ZERO: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O1: store i32 0, i32* %uninit, align 4
 
 TEST_BRACES(bool4, bool[4]);
 // CHECK-LABEL: @test_bool4_braces()
 // CHECK:       %braces = alloca [4 x i8], align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 4, i1 false)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 

 // CHECK-NEXT:  call void @llvm.memcpy
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%custom)
 
 TEST_UNINIT(tailpad4, tailpad[4]);
 // CHECK-LABEL: @test_tailpad4_uninit()
 // CHECK:       %uninit = alloca [4 x %struct.tailpad], align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_tailpad4_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_tailpad4_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_tailpad4_uninit.uninit
+// PATTERN-O1: bitcast
+// PATTERN-O1: call void @llvm.memset{{.*}}({{.*}}0, i8 -86, i64 16
 // ZERO-LABEL: @test_tailpad4_uninit()
 // ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(tailpad4, tailpad[4]);
 // CHECK-LABEL: @test_tailpad4_braces()
 // CHECK:       %braces = alloca [4 x %struct.tailpad], align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 16, i1 false)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 TEST_CUSTOM(tailpad4, tailpad[4], { {257, 1}, {257, 1}, {257, 1}, {257, 1} });
 // CHECK-LABEL: @test_tailpad4_custom()
 // CHECK:       %custom = alloca [4 x %struct.tailpad], align
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memcpy
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%custom)
 
 TEST_UNINIT(tailpad9, tailpad[9]);
 // CHECK-LABEL: @test_tailpad9_uninit()
 // CHECK:       %uninit = alloca [9 x %struct.tailpad], align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_tailpad9_uninit()
-// PATTERN: call void @llvm.memset{{.*}}, i8 -86,
+// PATTERN-O0: call void @llvm.memset{{.*}}, i8 -86,
 // ZERO-LABEL: @test_tailpad9_uninit()
 // ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(tailpad9, tailpad[9]);
 // CHECK-LABEL: @test_tailpad9_braces()
 // CHECK:       %braces = alloca [9 x %struct.tailpad], align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 36, i1 false)

 // ZERO-LABEL: @test_atomicdouble_uninit()
 // ZERO: store double 0.000000e+00, double* %uninit, align 8
 
 TEST_UNINIT(atomicnotlockfree, _Atomic(notlockfree));
 // CHECK-LABEL: @test_atomicnotlockfree_uninit()
 // CHECK:       %uninit = alloca %struct.notlockfree, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_atomicnotlockfree_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_atomicnotlockfree_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_atomicnotlockfree_uninit.uninit
+// PATTERN-O1: bitcast
+// PATTERN-O1: call void @llvm.memset{{.*}}({{.*}}0, i8 -86, i64 32
 // ZERO-LABEL: @test_atomicnotlockfree_uninit()
 // ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_UNINIT(atomicpadded, _Atomic(padded));
 // CHECK-LABEL: @test_atomicpadded_uninit()
 // CHECK:       %uninit = alloca %struct.padded, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_atomicpadded_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_atomicpadded_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_atomicpadded_uninit.uninit
+// PATTERN-O1: store i64 -6148914691236517206, i64* %uninit, align 8
 // ZERO-LABEL: @test_atomicpadded_uninit()
-// ZERO: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O1: store i64 0, i64* %uninit, align 8
 
 TEST_UNINIT(atomictailpad, _Atomic(tailpad));
 // CHECK-LABEL: @test_atomictailpad_uninit()
 // CHECK:       %uninit = alloca %struct.tailpad, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_atomictailpad_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_atomictailpad_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_atomictailpad_uninit.uninit
 // ZERO-LABEL: @test_atomictailpad_uninit()
-// ZERO: call void @llvm.memset{{.*}}, i8 0,
-
+// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O1: store i32 0, i32* %uninit, align 4
 
 TEST_UNINIT(complexfloat, _Complex float);
 // CHECK-LABEL: @test_complexfloat_uninit()
 // CHECK:       %uninit = alloca { float, float }, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_complexfloat_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_complexfloat_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_complexfloat_uninit.uninit
+// PATTERN-O1:  %[[F1:[^ ]*]] = getelementptr inbounds {{.*}}%uninit, i64 0, i32 0
+// PATTERN-O1  store float 0xFFFFFFFFE0000000, float* %[[F1]], align
+// PATTERN-O1:  %[[F2:[^ ]*]] = getelementptr inbounds {{.*}}%uninit, i64 0, i32 1
+// PATTERN-O1: store float 0xFFFFFFFFE0000000, float* %[[F2]], align
+
 // ZERO-LABEL: @test_complexfloat_uninit()
-// ZERO: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O1: store i64 0, i64* %uninit, align 8
 
 TEST_BRACES(complexfloat, _Complex float);
 // CHECK-LABEL: @test_complexfloat_braces()
 // CHECK:       %braces = alloca { float, float }, align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  %[[R:[^ ]*]] = getelementptr inbounds { float, float }, { float, float }* %braces, i32 0, i32 0
 // CHECK-NEXT:  %[[I:[^ ]*]] = getelementptr inbounds { float, float }, { float, float }* %braces, i32 0, i32 1
 // CHECK-NEXT:  store float 0.000000e+00, float* %[[R]], align [[ALIGN]]
 // CHECK-NEXT:  store float 0.000000e+00, float* %[[I]], align [[ALIGN]]
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 TEST_CUSTOM(complexfloat, _Complex float, { 3.1415926535897932384626433, 3.1415926535897932384626433 });
 // CHECK-LABEL: @test_complexfloat_custom()
 // CHECK:       %custom = alloca { float, float }, align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  %[[R:[^ ]*]] = getelementptr inbounds { float, float }, { float, float }* %custom, i32 0, i32 0
 // CHECK-NEXT:  %[[I:[^ ]*]] = getelementptr inbounds { float, float }, { float, float }* %custom, i32 0, i32 1
 // CHECK-NEXT:  store float 0x400921FB60000000, float* %[[R]], align [[ALIGN]]
 // CHECK-NEXT:  store float 0x400921FB60000000, float* %[[I]], align [[ALIGN]]
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%custom)
 
 TEST_UNINIT(complexdouble, _Complex double);
 // CHECK-LABEL: @test_complexdouble_uninit()
 // CHECK:       %uninit = alloca { double, double }, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_complexdouble_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_complexdouble_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_complexdouble_uninit.uninit
 // ZERO-LABEL: @test_complexdouble_uninit()
 // ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(complexdouble, _Complex double);
 // CHECK-LABEL: @test_complexdouble_braces()
 // CHECK:       %braces = alloca { double, double }, align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  %[[R:[^ ]*]] = getelementptr inbounds { double, double }, { double, double }* %braces, i32 0, i32 0
 // CHECK-NEXT:  %[[I:[^ ]*]] = getelementptr inbounds { double, double }, { double, double }* %braces, i32 0, i32 1

 // CHECK-NEXT:  store volatile i32 0, i32* %braces, align [[ALIGN]]
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 TEST_UNINIT(semivolatile, semivolatile);
 // CHECK-LABEL: @test_semivolatile_uninit()
 // CHECK:       %uninit = alloca %struct.semivolatile, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_semivolatile_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_semivolatile_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_semivolatile_uninit.uninit
 // ZERO-LABEL: @test_semivolatile_uninit()
-// ZERO: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O1: store i64 0, i64* %uninit, align 8
 
 TEST_BRACES(semivolatile, semivolatile);
 // CHECK-LABEL: @test_semivolatile_braces()
 // CHECK:       %braces = alloca %struct.semivolatile, align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 8, i1 false)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 TEST_CUSTOM(semivolatile, semivolatile, { 0x44444444, 0x44444444 });
 // CHECK-LABEL: @test_semivolatile_custom()
 // CHECK:       %custom = alloca %struct.semivolatile, align
-// CHECK-NEXT:  bitcast
-// CHECK-NEXT:  call void @llvm.memcpy
-// CHECK-NEXT:  call void @{{.*}}used{{.*}}%custom)
+// CHECK-O0:  bitcast
+// CHECK-O0:  call void @llvm.memcpy
+// CHECK-O0:  call void @{{.*}}used{{.*}}%custom)
+// CHECK-O1:  store i64 4919131752989213764, i64* %custom, align 8
 
 TEST_UNINIT(semivolatileinit, semivolatileinit);
 // CHECK-LABEL: @test_semivolatileinit_uninit()
 // CHECK:       %uninit = alloca %struct.semivolatileinit, align
 // CHECK-NEXT:  call void @{{.*}}semivolatileinit{{.*}}%uninit)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 
 TEST_BRACES(semivolatileinit, semivolatileinit);

 
 
 TEST_UNINIT(base, base);
 // CHECK-LABEL: @test_base_uninit()
 // CHECK:       %uninit = alloca %struct.base, align
 // CHECK-NEXT:  call void @{{.*}}base{{.*}}%uninit)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_base_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_base_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_base_uninit.uninit
 // ZERO-LABEL: @test_base_uninit()
-// ZERO: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O1: store i64 0, {{.*}} align 8
 
 TEST_BRACES(base, base);
 // CHECK-LABEL: @test_base_braces()
 // CHECK:       %braces = alloca %struct.base, align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 8, i1 false)
 // CHECK-NEXT:  call void @{{.*}}base{{.*}}%braces)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 TEST_UNINIT(derived, derived);
 // CHECK-LABEL: @test_derived_uninit()
 // CHECK:       %uninit = alloca %struct.derived, align
 // CHECK-NEXT:  call void @{{.*}}derived{{.*}}%uninit)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_derived_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_derived_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_derived_uninit.uninit
 // ZERO-LABEL: @test_derived_uninit()
-// ZERO: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O1: store i64 0, {{.*}} align 8
 
 TEST_BRACES(derived, derived);
 // CHECK-LABEL: @test_derived_braces()
 // CHECK:       %braces = alloca %struct.derived, align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 8, i1 false)
 // CHECK-NEXT:  call void @{{.*}}derived{{.*}}%braces)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 TEST_UNINIT(virtualderived, virtualderived);
 // CHECK-LABEL: @test_virtualderived_uninit()
 // CHECK:       %uninit = alloca %struct.virtualderived, align
 // CHECK-NEXT:  call void @{{.*}}virtualderived{{.*}}%uninit)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_virtualderived_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_virtualderived_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_virtualderived_uninit.uninit
 // ZERO-LABEL: @test_virtualderived_uninit()
 // ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(virtualderived, virtualderived);
 // CHECK-LABEL: @test_virtualderived_braces()
 // CHECK:       %braces = alloca %struct.virtualderived, align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 16, i1 false)
 // CHECK-NEXT:  call void @{{.*}}virtualderived{{.*}}%braces)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 
 TEST_UNINIT(matching, matching);
 // CHECK-LABEL: @test_matching_uninit()
 // CHECK:       %uninit = alloca %union.matching, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_matching_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_matching_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_matching_uninit.uninit
 // ZERO-LABEL: @test_matching_uninit()
-// ZERO: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O1: store i32 0, {{.*}} align 4
 
 TEST_BRACES(matching, matching);
 // CHECK-LABEL: @test_matching_braces()
 // CHECK:       %braces = alloca %union.matching, align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 4, i1 false)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 TEST_CUSTOM(matching, matching, { .f = 0xf00f });
 // CHECK-LABEL: @test_matching_custom()
 // CHECK:       %custom = alloca %union.matching, align
-// CHECK-NEXT:  bitcast
-// CHECK-NEXT:  call void @llvm.memcpy
-// CHECK-NEXT:  call void @{{.*}}used{{.*}}%custom)
+// CHECK-O0:  bitcast
+// CHECK-O0:  call void @llvm.memcpy
+// CHECK-O0:  call void @{{.*}}used{{.*}}%custom)
+// CHECK-O1:  getelementptr
+// CHECK-O1:  store i32 1198526208, i32* {{.*}}, align 4
 
 TEST_UNINIT(matchingreverse, matchingreverse);
 // CHECK-LABEL: @test_matchingreverse_uninit()
 // CHECK:       %uninit = alloca %union.matchingreverse, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_matchingreverse_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_matchingreverse_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_matchingreverse_uninit.uninit
+// PATTERN-O1: store float 0xFFFFFFFFE0000000
 // ZERO-LABEL: @test_matchingreverse_uninit()
-// ZERO: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O1: store i32 0, {{.*}} align 4
 
 TEST_BRACES(matchingreverse, matchingreverse);
 // CHECK-LABEL: @test_matchingreverse_braces()
 // CHECK:       %braces = alloca %union.matchingreverse, align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 4, i1 false)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 TEST_CUSTOM(matchingreverse, matchingreverse, { .i = 0xf00f });
 // CHECK-LABEL: @test_matchingreverse_custom()
 // CHECK:       %custom = alloca %union.matchingreverse, align
-// CHECK-NEXT:  bitcast
-// CHECK-NEXT:  call void @llvm.memcpy
-// CHECK-NEXT:  call void @{{.*}}used{{.*}}%custom)
+// CHECK-O0:    bitcast
+// CHECK-O0:    call void @llvm.memcpy
+// CHECK-O0:    call void @{{.*}}used{{.*}}%custom)
+// CHECK-O1:    store i32 61455, i32* %1, align 4
 
 TEST_UNINIT(unmatched, unmatched);
 // CHECK-LABEL: @test_unmatched_uninit()
 // CHECK:       %uninit = alloca %union.unmatched, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_unmatched_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_unmatched_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_unmatched_uninit.uninit
 // ZERO-LABEL: @test_unmatched_uninit()
-// ZERO: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O1: store i32 0, {{.*}} align 4
 
 TEST_BRACES(unmatched, unmatched);
 // CHECK-LABEL: @test_unmatched_braces()
 // CHECK:       %braces = alloca %union.unmatched, align
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memcpy
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 TEST_CUSTOM(unmatched, unmatched, { .i = 0x3badbeef });
 // CHECK-LABEL: @test_unmatched_custom()
 // CHECK:       %custom = alloca %union.unmatched, align
-// CHECK-NEXT:  bitcast
-// CHECK-NEXT:  call void @llvm.memcpy
-// CHECK-NEXT:  call void @{{.*}}used{{.*}}%custom)
+// CHECK-O0:    bitcast
+// CHECK-O0:    call void @llvm.memcpy
+// CHECK-O0:    call void @{{.*}}used{{.*}}%custom)
+// CHECK-O1:    store i32 1001242351, i32* {{.*}}, align 4
 
 TEST_UNINIT(unmatchedreverse, unmatchedreverse);
 // CHECK-LABEL: @test_unmatchedreverse_uninit()
 // CHECK:       %uninit = alloca %union.unmatchedreverse, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_unmatchedreverse_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_unmatchedreverse_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_unmatchedreverse_uninit.uninit
 // ZERO-LABEL: @test_unmatchedreverse_uninit()
-// ZERO: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O1:  store i32 0, {{.*}} align 4
 
 TEST_BRACES(unmatchedreverse, unmatchedreverse);
 // CHECK-LABEL: @test_unmatchedreverse_braces()
 // CHECK:       %braces = alloca %union.unmatchedreverse, align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 4, i1 false)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 TEST_CUSTOM(unmatchedreverse, unmatchedreverse, { .c = 42  });
 // CHECK-LABEL: @test_unmatchedreverse_custom()
 // CHECK:       %custom = alloca %union.unmatchedreverse, align
-// CHECK-NEXT:  bitcast
-// CHECK-NEXT:  call void @llvm.memcpy
-// CHECK-NEXT:  call void @{{.*}}used{{.*}}%custom)
+// CHECK-O0:    bitcast
+// CHECK-O0:    call void @llvm.memcpy
+// CHECK-O0:    call void @{{.*}}used{{.*}}%custom)
+// PATTERN-O1:  store i32 -1431655894, i32* {{.*}}, align 4
+// ZERO-O1:     store i32 42, i32* {{.*}}, align 4
 
 TEST_UNINIT(unmatchedfp, unmatchedfp);
 // CHECK-LABEL: @test_unmatchedfp_uninit()
 // CHECK:       %uninit = alloca %union.unmatchedfp, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_unmatchedfp_uninit()
-// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_unmatchedfp_uninit.uninit
+// PATTERN-O0: call void @llvm.memcpy{{.*}} @__const.test_unmatchedfp_uninit.uninit
 // ZERO-LABEL: @test_unmatchedfp_uninit()
-// ZERO: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O0: call void @llvm.memset{{.*}}, i8 0,
+// ZERO-O1: store i64 0, {{.*}} align 8
 
 TEST_BRACES(unmatchedfp, unmatchedfp);
 // CHECK-LABEL: @test_unmatchedfp_braces()
 // CHECK:       %braces = alloca %union.unmatchedfp, align
 // CHECK-NEXT:  bitcast
 // CHECK-NEXT:  call void @llvm.memcpy
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 TEST_CUSTOM(unmatchedfp, unmatchedfp, { .d = 3.1415926535897932384626433 });
 // CHECK-LABEL: @test_unmatchedfp_custom()
 // CHECK:       %custom = alloca %union.unmatchedfp, align
-// CHECK-NEXT:  bitcast
-// CHECK-NEXT:  call void @llvm.memcpy
-// CHECK-NEXT:  call void @{{.*}}used{{.*}}%custom)
-
+// CHECK-O0:    bitcast
+// CHECK-O0:    call void @llvm.memcpy
+// CHECK-O0:    call void @{{.*}}used{{.*}}%custom)
+// CHECK-O1:    store i64 4614256656552045848, i64* %1, align 8
 
 TEST_UNINIT(emptyenum, emptyenum);
 // CHECK-LABEL: @test_emptyenum_uninit()
 // CHECK:       %uninit = alloca i32, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_emptyenum_uninit()
-// PATTERN: store i32 -1431655766, i32* %braces, align 4
+// PATTERN: store i32 -1431655766, i32* %uninit, align 4
 // ZERO-LABEL: @test_emptyenum_uninit()
-// ZERO: store i32 0, i32* %braces, align 4
+// ZERO: store i32 0, i32* %uninit, align 4
 
 TEST_BRACES(emptyenum, emptyenum);
 // CHECK-LABEL: @test_emptyenum_braces()
 // CHECK:       %braces = alloca i32, align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  store i32 0, i32* %braces, align [[ALIGN]]
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 TEST_CUSTOM(emptyenum, emptyenum, { (emptyenum)42 });
 // CHECK-LABEL: @test_emptyenum_custom()
 // CHECK:       %custom = alloca i32, align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  store i32 42, i32* %custom, align [[ALIGN]]
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%custom)
 
 TEST_UNINIT(smallenum, smallenum);
 // CHECK-LABEL: @test_smallenum_uninit()
 // CHECK:       %uninit = alloca i32, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 // PATTERN-LABEL: @test_smallenum_uninit()
-// PATTERN: store i32 -1431655766, i32* %braces, align 4
+// PATTERN: store i32 -1431655766, i32* %uninit, align 4
 // ZERO-LABEL: @test_smallenum_uninit()
-// ZERO: store i32 0, i32* %braces, align 4
+// ZERO: store i32 0, i32* %uninit, align 4
 
 TEST_BRACES(smallenum, smallenum);
 // CHECK-LABEL: @test_smallenum_braces()
 // CHECK:       %braces = alloca i32, align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  store i32 0, i32* %braces, align [[ALIGN]]
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
 TEST_CUSTOM(smallenum, smallenum, { (smallenum)42 });