[SelDag] Add FREEZE

[aqjune]

Summary

• Add FREEZE node to SelDag
• Lower FreezeInst (in IR) to FREEZE node
• Add Legalization for FREEZE node

[mkuper]

include/llvm/CodeGen/ISDOpcodes.h

180 ↗

(On Diff #85346)

Please add more documentation here.
In particular, a very short form of the explanation from SelectionDAGISel::Select_FREEZE() about the difference in semantics between FREEZE and UNDEF would be appropriate.

lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp

2238 ↗

(On Diff #85346)

I'm a bit confused about this. The function above selects UNDEF to IMPLICIT_DEF.
Where do we do the duplication? If we duplicate UNDEFs on the DAG level, then everything's already fine, and you can just select a FREEZE directly to IMPLICIT_DEF.
If we're duplicating IMPLICIT_DEFs, then this is a problem regardless of what you do here, if the freeze ends up being an IMPLICIT_DEF. Or are you saying we're duplicating IMPLICIT_DEFs in the DAG, but not in MI? I'd find that surprising.

(Note that the documentation for IMPLICIT_DEF just says that "this is the MachineInstr-level equivalent of undef." If we're not allowed to duplicate IMPLICIT_DEFs and they have a single defined value, that should be corrected.)

[mkuper]

Also, I didn't see a corresponding patch for FastISel. Did I miss one?

[nlopes]

lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp

2238 ↗

(On Diff #85346)

If you have e.g.:
mul undef, undef

this eventually may get translated into:

%v1 = IMPLICIT_DEF
%v2 = IMPLICIT_DEF
%v3 = mul %v1, %v2

Even if it doesn't get translated to this explicitly, the semantics of UNDEF/IMPLICIT_DEF are that each read may see a different value. For example, IMPLICIT_DEF can be sank into a loop and each iteration may see a different value, and known-bits analyses will also take advantage of the fact that each bit can take any value it wants.
Therefore we use this "hack" of making a copy of the vreg where IMPLICIT_DEF is assigned to. This seems to guarantee that all uses see the same value, since there isn't a pass propagating IMPLICIT_DEF to copies.

[aqjune]

• FastISel for FREEZE added.
• Updated description of FREEZE.

[mkuper]

include/llvm/CodeGen/ISDOpcodes.h

180 ↗

(On Diff #85743)

random -> arbitrary (sorry, it's a pet peeve :-) )

lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp

2238 ↗

(On Diff #85346)

If I understand correctly, the freeze.ll test shows you do end up with just an IMPLICIT_DEF eventually.
So I'm not sure where the benefit is? You rely on backend phase ordering to elide the copy only after the last time IMPLICIT_DEF may be duplicated? This seems a bit fishy.

Anyway, this is just a drive-by, I defer to Quentin and the other if they think this is reasonable.

[aqjune]

Rebased to svn commit 308173

[aqjune]

Rebased to svn commit 331585

[lebedev.ri]

All these new llc tests - have you considered using utils/update_llc_test_checks.py ?

[aqjune]

Rebase to trunk@369887 (August 26th, 2019)

[lebedev.ri]

I guess this is the next patch in the queue?
This needs rebasing, and more tests to match the langref wording.

[aqjune]

• Support floating points, pointers, aggregate types, add tests for those types

[lebedev.ri]

I believe vector legalization tests are missing.

Cost model handling might be missing (i would guess they should be treated as free)

This looks reasonable-ish, but i don't have much knowledge on this part, sadly.
I'm wondering if @RKSimon / @spatel / @craig.topper might want to help review this?

include/llvm/CodeGen/ISDOpcodes.h

179 ↗

(On Diff #224861)

s/integer/value/

[craig.topper]

lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

385 ↗

(On Diff #224861)

Just use ISD::FREEZE instead of N->getOpcode()

4021 ↗

(On Diff #224861)

I think this should be with the other PromoteIntOp functions? I think that's how this file is sorted.

4021 ↗

(On Diff #224861)

Does this function even get called? The input type and the output type are the same right? So PromoteIntRes should have been called first.

4025 ↗

(On Diff #224861)

Same with this. We should expand the result and then we'll never have to expand the input separately.

lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

3105 ↗

(On Diff #224861)

Would we better off just handling freeze on its own instead of adding all this aggregate code to the generic visitUnary?

[lebedev.ri]

(as per inline comments)

[aqjune]

• Remove DAGTypeLegalizer::PromoteIntOp_FREEZE, DAGTypeLegalizer::ExpandIntOp_FREEZE as they are never called
• Add cost model of freeze
• Add vector legalization test to freeze-legalize.ll

[aqjune]

lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

3105 ↗

(On Diff #224861)

I put it here as visitBinary() was also handling opcodes, but I have no special preference.

[lebedev.ri]

@qcolombet - ping as per inline comment?
I'm also not really fully sold that the SelectionDAGISel::Select_FREEZE()
approach is fully future-proof/correct.

Other than that, nothing major stands out to me here.

include/llvm/Analysis/TargetTransformInfoImpl.h

853 ↗

(On Diff #225827)

Also here
And in TargetTransformInfo::getInstructionThroughput()

lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

4055 ↗

(On Diff #225827)

Spurious newline change

lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

3106 ↗

(On Diff #225827)

assert(Opcode == ISD::FREEZE && "Can only get here for freeze` nodes");`

3110 ↗

(On Diff #225827)

for (unsigned i = 0, NumOps = Op.getNumOperands(); i < NumOps; ++i)

lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp

2238 ↗

(On Diff #85346)

@qcolombet ping, please?

[aqjune]

I was looking through MachineInstr's optimizations to recheck validity of using IMPLICIT_DEF, and found that it caused problems at two cases.

1. When a function call is involved, its value is not preserved.

define i32 @foo()
  %y1 = freeze i32 undef
  %k = add i32 %y1, 10
  call i32 @g()
  %k2 = add i32 %y1, 20
  ..

->

_foo:
  addl  $10, %ebx
  callq _g
  addl  $20, %eax <- this is wrong because %eax and %ebx may differ
  ..

2. When PHI node is involved, it is folded into an incorrect value.

  br i1 %cond, label %BB1, label %BB2
BB1:
  %y1 = freeze i32 undef
  %k1 = sub i32 %y1, 1
  br label %END
BB2:
  %y2 = freeze i32 undef
  br label %END
END:
  %p = phi i32 [%y1, %BB1], [%y2, %BB2]
  %p2 = phi i32 [%k1, %BB1], [0, %BB2]
  store i32 %p, i32* @x
  store i32 %p2, i32* @y

->

LBB0_3:                                 ## %END
  movl  %eax, _x(%rip)
  movl  %eax, _y(%rip) <- should be "%eax - 1" if %cond is true

But, other than these two (function call and phi), other operations seemed okay.
Arithmetic operations including add / mul didn't fold IMPLICIT_DEF in a undef-like way.

What would be a good way to address the two cases?
The latter seemed to be done by ProcessImplicitDefs.cpp / PHIElimination.cpp. The former one seems more complicated; it seems to require preservation of information about the original IMPLICIT_DEF before it is removed by later pipeline.

[arsenm]

lib/CodeGen/SelectionDAG/FastISel.cpp

1573 ↗

(On Diff #225827)

s/unsigned/Register

lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

3109 ↗

(On Diff #225827)

1 seems a bit small for the size

lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp

2302 ↗

(On Diff #225827)

s/unsigned/Register/

2238 ↗

(On Diff #85346)

I see one potential problem with using a COPY of IMPLICIT_DEF. Suppose an instruction has two registers operands with different register classes. A codegen pass may reasonably try to rematerialize one of the implicit_defs to the natural operand register class to avoid the copy to satisfy operand constraints

test/CodeGen/X86/freeze.ll

89 ↗

(On Diff #225827)

I'm not sure where Phabricator gets it syntax highlighting from, but it should add the freeze keyword

[aqjune]

• Add FREEZE pseudoinstruction for MachineIR
• Let ExpandPostRAPseudos expand FREEZE
• Address reviewers' comments
• Add tests for function call / phi
• Add a comment to TargetPassConfig stating that after ExpandPostRAPseudos IMPLICIT_DEF should not be optimized to different registers/constants

[aqjune]

I added FREEZE pseudoinstruction to MachineIR, as it seemed to be the succinct way to make it correct.
I made ExpandPostRAPseudos expand the FREEZE to register-copy assembly instruction. ExpandPostRAPseudos pass is done after register allocation (which, or at least a pass relevant to it, seems to replace uses of IMPLICIT_DEF with different registers) as well as ProcessImplicitDef (which is the culprit of the incorrect PHI optimization example) and PHIElimination. I left a commit that explicitly states that IMPLICIT_DEF cannot be used in an undef-y way after the pass.

[craig.topper]

llvm/include/llvm/Support/TargetOpcodes.def

59 ↗

(On Diff #228705)

Comment says IMPLICIT_DEF

llvm/lib/CodeGen/SelectionDAG/FastISel.cpp

1571

Why doesn't this TargetOpcode::Freeze?

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

3106

Separate this into just visitFreeze.

[aqjune]

• Let FastISel emit FREEZE
• Let SelectionDAGBuilder::visitFreeze lower freeze IR instructions
• Minor fixes

[craig.topper]

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

3104

Drop this change?

[lebedev.ri]

Do you also want a MIR test that shows IR freeze -> MIR freeze lowering?

[aqjune]

• Add freeze IR -> MIR test

[aqjune]

llvm/test/CodeGen/X86/freeze-mir.ll

1 ↗

(On Diff #229230)

I made a separate freeze-mir.ll file for mir test because update_mir_test_checks.py did not work after freeze.ll is processed with update_llc_test_checks.py

[lebedev.ri]

Hmm, thanks, i think this now looks good to me.
@craig.topper / @qcolombet / @arsenm ?

llvm/test/CodeGen/X86/freeze-call.ll

7–12 ↗

(On Diff #229230)

You can avoid cfi noise via define i32 @foo() nounwind {

[aqjune]

• Add nounwind to tests

[arsenm]

llvm/include/llvm/Target/Target.td

1067 ↗

(On Diff #229443)

Why isNotDuplicable? I don't think this is the best maintained instruction property

[aqjune]

llvm/include/llvm/Target/Target.td

1067 ↗

(On Diff #229443)

It is because different freeze defs can yield different values. For example, consider following transformation:

%undef = IMPLICIT_DEF
%x = FREEZE %undef
use(%x)
use(%x) // these two uses should see the same freezed value
->
%undef = IMPLICIT_DEF
%x = FREEZE %undef
use(%x)
%x' = FREEZE %undef
use(%x') // It is possible that %x and %x' are assigned differently freezed values.

This transformation is incorrect, because use()s after the transformation can see different values.
To prevent this class of optimizations, isNotDuplicable is set to 1.

[arsenm]

llvm/include/llvm/Target/Target.td

1067 ↗

(On Diff #229443)

But these both are using the same undef vreg? The second freeze is still using the original undef, so this should be fine?

[arsenm]

llvm/include/llvm/Target/Target.td

1067 ↗

(On Diff #229443)

To clarify, I think the property you are really looking for is not the property given to you by isNotDuplicable. What you really care about is the input operand not using a new instance of undef. The "nonduplicability" is a function of the input value and not the instruction itself

[aqjune]

llvm/include/llvm/Target/Target.td

1067 ↗

(On Diff #229443)

The second use of the undef register can be assigned a different physical register at (perhaps) register allocation, after seeing that its definition is IMPLICIT_DEF.

%undef = IMPLICIT_DEF
%x = FREEZE %undef
use(%x)
%x' = FREEZE %undef
use(%x')
=>
%x = FREEZE $eax
use(%x)
%x' = FREEZE $ebx
use(%x')

This transformation can happen if there is a function call between them. test/CodeGen/X86/freeze-call.ll checks that it never happens.

[arsenm]

llvm/include/llvm/Target/Target.td

1067 ↗

(On Diff #229443)

I think handling this correctly requires changing to how IMPLICIT_DEF is handled in register allocation. I don't think noduplicate is really going to model this constraint sufficiently

[aqjune]

llvm/include/llvm/Target/Target.td

1067 ↗

(On Diff #229443)

It would make complexity of this patch (lowering of freeze IR instruction) easier, but would the change in register allocation need performance tests?
For example, if all uses of IMPLICIT_DEF should see a same caller-save register, the register should be spilled whenever its liveness crosses a function call.
Currently IMPLICIT_DEF is commented as MachineInstr-level equivalent of undef, and undef`may evaluate to different values as well, so I wonder whether there are passes other than regalloc that use that semantics too.

isNotDuplicable's comment says

class MCInstrDesc {
...
  /// Return true if this instruction cannot be safely
  /// duplicated.  For example, if the instruction has a unique labels attached
  /// to it, duplicating it would cause multiple definition errors.
  bool isNotDuplicable() const { return Flags & (1ULL << MCID::NotDuplicable); }

so I thought non-duplicability was a property of the freeze instruction itself.

[arsenm]

llvm/include/llvm/Target/Target.td

1067 ↗

(On Diff #229443)

I mean the definition of IMPLICIT_DEF needs to change to support this. I think we may actually need two different flavors of IMPLICIT_DEF. noduplicate is a stronger barrier to useful transforms, and also doesn't model the real problem here.

If you started out with two separate freeze instructions reading the same IMPLICIT_DEF register, you would still see the same problem in your example.

[aqjune]

llvm/include/llvm/Target/Target.td

1067 ↗

(On Diff #229443)

I mean the definition of IMPLICIT_DEF needs to change to support this. I think we may actually need two different flavors of IMPLICIT_DEF.

Understood. So we can have two kinds of IMPLICIT_DEFs; the first one is an instruction that can be optimized to different values whenever used. The second one has the same value whenever it is used.

I think the second IMPLICIT_DEF is equivalent to FREEZE (IMPLICIT_DEF of the first kind), because freeze is an instruction that guarantees all uses of the value see the same value even though the operand is an undef-like value.

If you started out with two separate freeze instructions reading the same IMPLICIT_DEF register, you would still see the same problem in your example.

If the program originally contained two separated freezes, then it is okay, because the program is originally written in that way. The programmer must have written down two freezes in LLVM IR bitcode, and they are lowered into MIR.
The problematic case is when a program with a single freeze is optimized to a program with several freezes. Different freezes can return different values.

%undef = IMPLICIT_DEF
%x = FREEZE %undef
use(%x)
use(%x) // these two uses see the same value with help of freeze.
->
%undef = IMPLICIT_DEF
%x = FREEZE %undef
use(%x)
%x' = FREEZE %undef
use(%x') // Now %x and %x' can be assigned different values, so this optimization is wrong.

[arsenm]

llvm/include/llvm/Target/Target.td

1067 ↗

(On Diff #229443)

I think it's important to define the rules here clearly for MIR, and not rely on expectations from the original IR. Some legalization for example may end up inserting multiple freezes of the same input value, and that should work correctly. Multiple freezes of the same input register should produce the same value.

[aqjune]

llvm/include/llvm/Target/Target.td

1067 ↗

(On Diff #229443)

I agree, I'll write the rule for freeze.
Regarding legalization, in which case can it be copied? The case that I was aware of was splitting a register of illegal size, which copies freeze but does not increase # of uses per definition:

%x = ... // invalid type, say i50
%y = freeze %x
  =>
%x1 = ... // i32
%x2 = ... // i18
%y1 = freeze %x1
%y2 = freeze %x2

[lebedev.ri]

Looks like this is waiting on an update.
Really happy to see this progressing.

[aqjune]

I added description about FREEZE and IMPLICIT_DEF to TargetOpcodes.def.
To address the legalization issue, updated the semantics so FREEZEs return the same value if placed consecutively. Would this properly address the issue?

[lebedev.ri]

I'm sorry, it seems the track of this patch was lost.
We really need to get this last bit of freeze support landed.
Can anyone spot any remaining issues with the patch?

[craig.topper]

llvm/include/llvm/Support/TargetOpcodes.def

61 ↗

(On Diff #230899)

constraint -> constrain

[nlopes]

The patch & semantics look good to me, but I'm not a backend expert. I'll leave the final LGTM to someone else.
It would be awesome if we could get this in for 10.0 so that we have complete support for freeze.

[aqjune]

• Rebase
• Address a comment

[aqjune]

To synchronize - the remaining issue was about duplicability of the freeze instruction in MIR.

In IR, a duplicated freeze may result in a different value due to the nature of undef. However, in MIR, this may be problematic in some cases.

My suggestion is to allow freeze in MIR to yield the same value when they are consecutively arranged.
This involved clarificaton of when the value of IMPLICIT_DEF could change during execution, and the updated comments of TargetOpcodes.def reflects that.

Currently, isNotDuplicable is still set to 1 to leave it as conservative & copiers deal with freeze specially in the future.

[aqjune]

BTW, there was a discussion about IMPLICIT_DEF as well, about whether it should return consistent value for every use vs. it can return different values per use (as IR's undef).

Currently IMPLICIT_DEF can yieid different values, and there are a few cases where that happens (such as using IMPLICIT_DEF across function calls).

Making it return consistent value will allow FREEZE to be duplicable, but it would require fixing several transformations that deal with IMPLICIT_DEF.

Before discussion about IMPLICIT_DEF is further made and it is changed to return concrete value, I'd like to suggest leaving FREEZE as semi-duplicable (as suggested by this patch), but not fully duplicable.

[efriedma]

llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp

1135 ↗

(On Diff #237090)

If we expand FREEZE in ExpandPostRAPseudos, how can we reach this code?

[aqjune]

• Rebase
• Remove AsmPrinter::emitFreeze

[aqjune]

llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp

1135 ↗

(On Diff #237090)

It was a legacy code to see the freeze comment from assembly before updating ExpandPostRAPseudos. I removed it.

[aqjune]

For the next step, what can I do?

If duplicability is the main concern, I can split this patch so (1) FREEZE is defined to be duplicable first, and (2) when there happens a practical case where duplication matters, the next one is created.

It will reduce diffs in TargetOpcodes.def as well because the comment update at IMPLICIT_DEF is not needed anymore.

[aqjune]

Rebase

[aqjune]

ping

[xbolva00]

Re-ping

[aqjune]

ping

If adding FREEZE to MachineIR causes maintenance cost, another possible solution is to lower SelDag's FREEZE(UNDEF) to IMPLICIT_DEF directly.
In this option, FREEZE only exist in SelDag, not in MIR.
Instead, now IMPLICIT_DEF in MIR should always yield the same value. This requires a few transformations regarding IMPLICIT_DEF in MIR to be fixed.
This may cause suboptimal codegen in a few cases, but I believe we have a solution for the cases, because a single UNDEF from SelDag can be always lowered into multiple IMPLICIT_DEFs (per each use) in MIR.

[lebedev.ri]

This seems good to me, but you want to wait for someone else more knowledgeable..
ping @arsenm @qcolombet @craig.topper

llvm/include/llvm/Analysis/TargetTransformInfoImpl.h

65–68 ↗

(On Diff #241401)

This is unrelated, please feel free to commit this right away.

[aqjune]

• Rebase
• Make the update in getOperationCost as a separate commit (4f71252)

[aqjune]

Leave the SelDag patch only, MachineIR part will be made as a separate patch (as discussed at D76483)

[spatel]

llvm/test/CodeGen/X86/freeze-legalize.ll

9–13

Do we need to add basic simplify / constant folding for SDAG ?
freeze ( Constant ) --> Constant

[aqjune]

llvm/test/CodeGen/X86/freeze-legalize.ll

9–13

Yes, I agree it will be great. Do you want to make this patch contain the change as well?

[aqjune]

llvm/test/CodeGen/X86/freeze-legalize.ll

9–13

Or I can land this first, and add the simplify / constant folding for SDAG.
I prefer incrementally making things because this patch itself is a big change.

[spatel]

llvm/test/CodeGen/X86/freeze-legalize.ll

9–13

I prefer smaller patches too. Let's make that a follow-up.

Is it correct that there is very little chance that this patch will create a visible performance regression (because there should be almost no freeze instruction creation in IR yet)?

[aqjune]

llvm/test/CodeGen/X86/freeze-legalize.ll

9–13

Yes, it is.
Currently there is only one place where freeze is introduced - https://reviews.llvm.org/D76179
I checked that from assembly outputs of LLVM test-suite , only 3 / 5239 files are affected by this patch.

[spatel]

llvm/test/CodeGen/X86/freeze-legalize.ll

9–13

That sounds good then. But can we avoid those 3 regressions cases before or within this patch? Ideally, we don't want to knowingly regress anything.

[aqjune]

llvm/test/CodeGen/X86/freeze-legalize.ll

9–13

Among 3 files, two were simple regressions that had a bit more verbose assembly:

...
sete  %al    
orb %bpl, %al
jne .LBB9_1  
=>
...
sete  %al    
orb %bpl, %al
testb $1, %al
jne .LBB9_1

...
testb $1, %al
je  .LBB0_2  
=>
...
movl  %eax, %ecx 
andl  $1, %ecx   
je  .LBB0_2

Case 3's assembly diff was bigger, so needs inspection. I can visit it after the simpler two cases are resolved.

[JamesNagurne]

@aqjune

@bkramer committed a fix for a crash in LegalizeFloatTypes where this operator appeared in SoftPromoteHalfResult

Our downstream ARM compiler is crashing in regressions in a similar manner in SoftenFloatResult in the same file. Is it not conceivable that ISD::FREEZE might make it to this function? If that's the case, we can look into it and start up a new review if necessary. The test causing this crash is select-cc.ll.

[aqjune]

In D29014#1946763, @JamesNagurne wrote:

@aqjune

@bkramer committed a fix for a crash in LegalizeFloatTypes where this operator appeared in SoftPromoteHalfResult

Our downstream ARM compiler is crashing in regressions in a similar manner in SoftenFloatResult in the same file. Is it not conceivable that ISD::FREEZE might make it to this function? If that's the case, we can look into it and start up a new review if necessary. The test causing this crash is select-cc.ll.

Hi, I saw @bkramer 's fix too (commit id 0019c2f194a5e1f4cd65c5284e204328cc40ab3d ), and I think it is a valid fix. Similar thing can be applied to the SoftenFloatResult function. I'll make a patch for this.

llvm/include/llvm/CodeGen/FastISel.h

protected:
   bool selectBinaryOp(const User *I, unsigned ISDOpcode);
   bool selectFNeg(const User *I, const Value *In);
   bool selectGetElementPtr(const User *I);
   bool selectStackmap(const CallInst *I);
   bool selectPatchpoint(const CallInst *I);
   bool selectCall(const User *I);
   bool selectIntrinsicCall(const IntrinsicInst *II);
   bool selectBitCast(const User *I);
+  bool selectFreeze(const User *I);
   bool selectCast(const User *I, unsigned Opcode);
   bool selectExtractValue(const User *U);
   bool selectInsertValue(const User *I);
   bool selectXRayCustomEvent(const CallInst *II);
   bool selectXRayTypedEvent(const CallInst *II);
 
   bool shouldOptForSize(const MachineFunction *MF) const {
     // TODO: Implement PGSO.

llvm/include/llvm/CodeGen/ISDOpcodes.h

enum NodeType {
     /// CopyFromReg - This node indicates that the input value is a virtual or
     /// physical register that is defined outside of the scope of this
     /// SelectionDAG.  The register is available from the RegisterSDNode object.
     CopyFromReg,
 
     /// UNDEF - An undefined node.
     UNDEF,
 
+    // FREEZE - FREEZE(VAL) returns an arbitrary value if VAL is UNDEF (or
+    // is evaluated to UNDEF), or returns VAL otherwise. Note that each
+    // read of UNDEF can yield different value, but FREEZE(UNDEF) cannot.
+    FREEZE,
+
     /// EXTRACT_ELEMENT - This is used to get the lower or upper (determined by
     /// a Constant, which is required to be operand #1) half of the integer or
     /// float value specified as operand #0.  This is only for use before
     /// legalization, for values that will be broken into multiple registers.
     EXTRACT_ELEMENT,
 
     /// BUILD_PAIR - This is the opposite of EXTRACT_ELEMENT in some ways.
     /// Given two values of the same integer value type, this produces a value

llvm/include/llvm/CodeGen/SelectionDAGISel.h

private:
 
   // Calls to these functions are generated by tblgen.
   void Select_INLINEASM(SDNode *N, bool Branch);
   void Select_READ_REGISTER(SDNode *Op);
   void Select_WRITE_REGISTER(SDNode *Op);
   void Select_UNDEF(SDNode *N);
   void CannotYetSelect(SDNode *N);
 
+  void Select_FREEZE(SDNode *N);
+
 private:
   void DoInstructionSelection();
   SDNode *MorphNode(SDNode *Node, unsigned TargetOpc, SDVTList VTList,
                     ArrayRef<SDValue> Ops, unsigned EmitNodeInfo);
 
   SDNode *MutateStrictFPToFP(SDNode *Node, unsigned NewOpc);
 
   /// Prepares the landing pad to take incoming values or do other EH

llvm/lib/CodeGen/SelectionDAG/FastISel.cpp

bool FastISel::selectBitCast(const User *I) {
 
   // If the reg-reg copy failed, select a BITCAST opcode.
   if (!ResultReg)
     ResultReg = fastEmit_r(SrcVT, DstVT, ISD::BITCAST, Op0, Op0IsKill);
 
   if (!ResultReg)
     return false;
 
   updateValueMap(I, ResultReg);
   return true;
 }
 
+bool FastISel::selectFreeze(const User *I) {
+  Register Reg = getRegForValue(I->getOperand(0));
+  if (!Reg)
+    // Unhandled operand.
+    return false;
+
+  EVT ETy = TLI.getValueType(DL, I->getOperand(0)->getType());
+  if (ETy == MVT::Other || !TLI.isTypeLegal(ETy))
+    // Unhandled type, bail out.
+    return false;
+
+  MVT Ty = ETy.getSimpleVT();
+  const TargetRegisterClass *TyRegClass = TLI.getRegClassFor(Ty);
+  Register ResultReg = createResultReg(TyRegClass);
+  BuildMI(*FuncInfo.MBB, FuncInfo.InsertPt, DbgLoc,
+          TII.get(TargetOpcode::COPY), ResultReg).addReg(Reg);
+
+  updateValueMap(I, ResultReg);
+  return true;
+}
+
 // Remove local value instructions starting from the instruction after
 // SavedLastLocalValue to the current function insert point.
 void FastISel::removeDeadLocalValueCode(MachineInstr *SavedLastLocalValue)
 {
   MachineInstr *CurLastLocalValue = getLastLocalValue();
   if (CurLastLocalValue != SavedLastLocalValue) {
     // Find the first local value instruction to be deleted.
     // This is the instruction after SavedLastLocalValue if it is non-NULL.
if (!Reg)
       return false;
     updateValueMap(I, Reg);
     return true;
   }
 
   case Instruction::ExtractValue:
     return selectExtractValue(I);
 
+  case Instruction::Freeze:
+    return selectFreeze(I);
+
   case Instruction::PHI:
     llvm_unreachable("FastISel shouldn't visit PHI nodes!");
 
   default:
     // Unhandled instruction. Halt "fast" selection and bail.
     return false;
   }
 }

llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

#endif
   case ISD::VECREDUCE_OR:
   case ISD::VECREDUCE_XOR:
   case ISD::VECREDUCE_SMAX:
   case ISD::VECREDUCE_SMIN:
   case ISD::VECREDUCE_UMAX:
   case ISD::VECREDUCE_UMIN:
     Res = PromoteIntRes_VECREDUCE(N);
     break;
+
+  case ISD::FREEZE:
+    Res = PromoteIntRes_FREEZE(N);
+    break;
   }
 
   // If the result is null then the sub-method took care of registering it.
   if (Res.getNode())
     SetPromotedInteger(SDValue(N, ResNo), Res);
 }
 
 SDValue DAGTypeLegalizer::PromoteIntRes_MERGE_VALUES(SDNode *N,
static EVT getShiftAmountTyForConstant(EVT VT, const TargetLowering &TLI,
   // If any possible shift value won't fit in the prefered type, just use
   // something safe. It will be legalized when the shift is expanded.
   if (!ShiftVT.isVector() &&
       ShiftVT.getSizeInBits() < Log2_32_Ceil(VT.getSizeInBits()))
     ShiftVT = MVT::i32;
   return ShiftVT;
 }
 
+SDValue DAGTypeLegalizer::PromoteIntRes_FREEZE(SDNode *N) {
+  SDValue V = GetPromotedInteger(N->getOperand(0));
+  return DAG.getNode(ISD::FREEZE, SDLoc(N),
+                     V.getValueType(), V);
+}
+
 SDValue DAGTypeLegalizer::PromoteIntRes_BSWAP(SDNode *N) {
   SDValue Op = GetPromotedInteger(N->getOperand(0));
   EVT OVT = N->getValueType(0);
   EVT NVT = Op.getValueType();
   SDLoc dl(N);
 
   unsigned DiffBits = NVT.getScalarSizeInBits() - OVT.getScalarSizeInBits();
   EVT ShiftVT = getShiftAmountTyForConstant(NVT, TLI, DAG);

 #endif
     report_fatal_error("Do not know how to expand the result of this "
                        "operator!");
 
   case ISD::MERGE_VALUES: SplitRes_MERGE_VALUES(N, ResNo, Lo, Hi); break;
   case ISD::SELECT:       SplitRes_SELECT(N, Lo, Hi); break;
   case ISD::SELECT_CC:    SplitRes_SELECT_CC(N, Lo, Hi); break;
   case ISD::UNDEF:        SplitRes_UNDEF(N, Lo, Hi); break;
+  case ISD::FREEZE:       SplitRes_FREEZE(N, Lo, Hi); break;
 
   case ISD::BITCAST:            ExpandRes_BITCAST(N, Lo, Hi); break;
   case ISD::BUILD_PAIR:         ExpandRes_BUILD_PAIR(N, Lo, Hi); break;
   case ISD::EXTRACT_ELEMENT:    ExpandRes_EXTRACT_ELEMENT(N, Lo, Hi); break;
   case ISD::EXTRACT_VECTOR_ELT: ExpandRes_EXTRACT_VECTOR_ELT(N, Lo, Hi); break;
   case ISD::VAARG:              ExpandRes_VAARG(N, Lo, Hi); break;
 
   case ISD::ANY_EXTEND:  ExpandIntRes_ANY_EXTEND(N, Lo, Hi); break;

llvm/lib/CodeGen/SelectionDAG/LegalizeTypes.h

private:
   SDValue PromoteIntRes_BUILD_PAIR(SDNode *N);
   SDValue PromoteIntRes_Constant(SDNode *N);
   SDValue PromoteIntRes_CTLZ(SDNode *N);
   SDValue PromoteIntRes_CTPOP(SDNode *N);
   SDValue PromoteIntRes_CTTZ(SDNode *N);
   SDValue PromoteIntRes_EXTRACT_VECTOR_ELT(SDNode *N);
   SDValue PromoteIntRes_FP_TO_XINT(SDNode *N);
   SDValue PromoteIntRes_FP_TO_FP16(SDNode *N);
+  SDValue PromoteIntRes_FREEZE(SDNode *N);
   SDValue PromoteIntRes_INT_EXTEND(SDNode *N);
   SDValue PromoteIntRes_LOAD(LoadSDNode *N);
   SDValue PromoteIntRes_MLOAD(MaskedLoadSDNode *N);
   SDValue PromoteIntRes_MGATHER(MaskedGatherSDNode *N);
   SDValue PromoteIntRes_Overflow(SDNode *N);
   SDValue PromoteIntRes_SADDSUBO(SDNode *N, unsigned ResNo);
   SDValue PromoteIntRes_SELECT(SDNode *N);
   SDValue PromoteIntRes_VSELECT(SDNode *N);
private:
   void GetPairElements(SDValue Pair, SDValue &Lo, SDValue &Hi);
 
   // Generic Result Splitting.
   void SplitRes_MERGE_VALUES(SDNode *N, unsigned ResNo,
                              SDValue &Lo, SDValue &Hi);
   void SplitRes_SELECT      (SDNode *N, SDValue &Lo, SDValue &Hi);
   void SplitRes_SELECT_CC   (SDNode *N, SDValue &Lo, SDValue &Hi);
   void SplitRes_UNDEF       (SDNode *N, SDValue &Lo, SDValue &Hi);
+  void SplitRes_FREEZE      (SDNode *N, SDValue &Lo, SDValue &Hi);
 
   void SplitVSETCC(const SDNode *N);
 
   //===--------------------------------------------------------------------===//
   // Generic Expansion: LegalizeTypesGeneric.cpp
   //===--------------------------------------------------------------------===//
 
   // Legalization methods which only use that the illegal type is split into two

llvm/lib/CodeGen/SelectionDAG/LegalizeTypesGeneric.cpp

 }
 
 void DAGTypeLegalizer::SplitRes_UNDEF(SDNode *N, SDValue &Lo, SDValue &Hi) {
   EVT LoVT, HiVT;
   std::tie(LoVT, HiVT) = DAG.GetSplitDestVTs(N->getValueType(0));
   Lo = DAG.getUNDEF(LoVT);
   Hi = DAG.getUNDEF(HiVT);
 }
+
+void DAGTypeLegalizer::SplitRes_FREEZE(SDNode *N, SDValue &Lo, SDValue &Hi) {
+  SDValue L, H;
+  SDLoc dl(N);
+  GetSplitOp(N->getOperand(0), L, H);
+
+  Lo = DAG.getNode(ISD::FREEZE, dl, L.getValueType(), L);
+  Hi = DAG.getNode(ISD::FREEZE, dl, H.getValueType(), H);
+}

llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp

#endif
   case ISD::FEXP:
   case ISD::FEXP2:
   case ISD::FFLOOR:
   case ISD::FLOG:
   case ISD::FLOG10:
   case ISD::FLOG2:
   case ISD::FNEARBYINT:
   case ISD::FNEG:
+  case ISD::FREEZE:
   case ISD::FP_EXTEND:
   case ISD::FP_TO_SINT:
   case ISD::FP_TO_UINT:
   case ISD::FRINT:
   case ISD::FROUND:
   case ISD::FSIN:
   case ISD::FSQRT:
   case ISD::FTRUNC:
#endif
   case ISD::FEXP:
   case ISD::FEXP2:
   case ISD::FFLOOR:
   case ISD::FLOG:
   case ISD::FLOG10:
   case ISD::FLOG2:
   case ISD::FNEARBYINT:
   case ISD::FNEG:
+  case ISD::FREEZE:
   case ISD::FP_EXTEND:
   case ISD::FP_ROUND:
   case ISD::FP_TO_SINT:
   case ISD::FP_TO_UINT:
   case ISD::FRINT:
   case ISD::FROUND:
   case ISD::FSIN:
   case ISD::FSQRT:
#include "llvm/IR/ConstrainedOps.def"
   case ISD::BITREVERSE:
   case ISD::BSWAP:
   case ISD::CTLZ:
   case ISD::CTLZ_ZERO_UNDEF:
   case ISD::CTPOP:
   case ISD::CTTZ:
   case ISD::CTTZ_ZERO_UNDEF:
   case ISD::FNEG:
+  case ISD::FREEZE:
   case ISD::FCANONICALIZE:
     Res = WidenVecRes_Unary(N);
     break;
   case ISD::FMA:
     Res = WidenVecRes_Ternary(N);
     break;
   }
 

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

for (const auto *U : User->users()) {
         // must be a reduction operation.
         if (!U->hasOneUse())
           return false;
 
         auto U2 = dyn_cast<Instruction>(*U->user_begin());
         if (!U2 || U2->getOpcode() != OpCode)
           return false;
 
         // Check operands of the reduction operation.
         if ((U2->getOperand(0) == U->getOperand(0) && U2->getOperand(1) == U) ||
             (U2->getOperand(1) == U->getOperand(0) && U2->getOperand(0) == U)) {
           UsersToVisit.push_back(U2);
           ElemNumToReduce /= 2;
         } else
           return false;
       } else if (isa<ExtractElementInst>(U)) {
         // At this moment we should have reduced all elements in the vector.
         if (ElemNumToReduce != 1)
           return false;
if (NumClusters > 3 && TM.getOptLevel() != CodeGenOpt::None &&
       continue;
     }
 
     lowerWorkItem(W, SI.getCondition(), SwitchMBB, DefaultMBB);
   }
 }
 
 void SelectionDAGBuilder::visitFreeze(const FreezeInst &I) {
-  SDValue N = getValue(I.getOperand(0));
-  setValue(&I, N);
+  SDNodeFlags Flags;
+
+  SDValue Op = getValue(I.getOperand(0));
+  if (I.getOperand(0)->getType()->isAggregateType()) {
+    EVT VT = Op.getValueType();
+    SmallVector<SDValue, 1> Values;
+    for (unsigned i = 0; i < Op.getNumOperands(); ++i) {
+      SDValue Arg(Op.getNode(), i);
+      SDValue UnNodeValue = DAG.getNode(ISD::FREEZE, getCurSDLoc(), VT, Arg, Flags);
+      Values.push_back(UnNodeValue);
+    }
+    SDValue MergedValue = DAG.getMergeValues(Values, getCurSDLoc());
+    setValue(&I, MergedValue);
+  } else {
+    SDValue UnNodeValue = DAG.getNode(ISD::FREEZE, getCurSDLoc(), Op.getValueType(),
+                                      Op, Flags);
+    setValue(&I, UnNodeValue);
+  }
 }

llvm/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp

#endif
   case ISD::STACKRESTORE:               return "stackrestore";
   case ISD::TRAP:                       return "trap";
   case ISD::DEBUGTRAP:                  return "debugtrap";
   case ISD::LIFETIME_START:             return "lifetime.start";
   case ISD::LIFETIME_END:               return "lifetime.end";
   case ISD::GC_TRANSITION_START:        return "gc_transition.start";
   case ISD::GC_TRANSITION_END:          return "gc_transition.end";
   case ISD::GET_DYNAMIC_AREA_OFFSET:    return "get.dynamic.area.offset";
+  case ISD::FREEZE:                     return "freeze";
 
   // Bit manipulation
   case ISD::ABS:                        return "abs";
   case ISD::BITREVERSE:                 return "bitreverse";
   case ISD::BSWAP:                      return "bswap";
   case ISD::CTPOP:                      return "ctpop";
   case ISD::CTTZ:                       return "cttz";
   case ISD::CTTZ_ZERO_UNDEF:            return "cttz_zero_undef";

llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp

void SelectionDAGISel::Select_WRITE_REGISTER(SDNode *Op) {
   ReplaceUses(Op, New.getNode());
   CurDAG->RemoveDeadNode(Op);
 }
 
 void SelectionDAGISel::Select_UNDEF(SDNode *N) {
   CurDAG->SelectNodeTo(N, TargetOpcode::IMPLICIT_DEF, N->getValueType(0));
 }
 
+void SelectionDAGISel::Select_FREEZE(SDNode *N) {
+  // TODO: We don't have FREEZE pseudo-instruction in MachineInstr-level now.
+  // If FREEZE instruction is added later, the code below must be changed as
+  // well.
+  CurDAG->SelectNodeTo(N, TargetOpcode::COPY, N->getValueType(0),
+                       N->getOperand(0));
+}
+
 /// GetVBR - decode a vbr encoding whose top bit is set.
 LLVM_ATTRIBUTE_ALWAYS_INLINE static inline uint64_t
 GetVBR(uint64_t Val, const unsigned char *MatcherTable, unsigned &Idx) {
   assert(Val >= 128 && "Not a VBR");
   Val &= 127;  // Remove first vbr bit.
 
   unsigned Shift = 7;
   uint64_t NextBits;
case ISD::READ_REGISTER:
     Select_READ_REGISTER(NodeToMatch);
     return;
   case ISD::WRITE_REGISTER:
     Select_WRITE_REGISTER(NodeToMatch);
     return;
   case ISD::UNDEF:
     Select_UNDEF(NodeToMatch);
     return;
+  case ISD::FREEZE:
+    Select_FREEZE(NodeToMatch);
+    return;
   }
 
   assert(!NodeToMatch->isMachineOpcode() && "Node already selected!");
 
   // Set up the node stack with NodeToMatch as the only node on the stack.
   SmallVector<SDValue, 8> NodeStack;
   SDValue N = SDValue(NodeToMatch, 0);
   NodeStack.push_back(N);

llvm/lib/CodeGen/TargetLoweringBase.cpp

#include "llvm/IR/Instruction.def"
   case UserOp2:        return 0;
   case VAArg:          return 0;
   case ExtractElement: return ISD::EXTRACT_VECTOR_ELT;
   case InsertElement:  return ISD::INSERT_VECTOR_ELT;
   case ShuffleVector:  return ISD::VECTOR_SHUFFLE;
   case ExtractValue:   return ISD::MERGE_VALUES;
   case InsertValue:    return ISD::MERGE_VALUES;
   case LandingPad:     return 0;
-  case Freeze:         return 0;
+  case Freeze:         return ISD::FREEZE;
   }
 
   llvm_unreachable("Unknown instruction type encountered!");
 }
 
 std::pair<int, MVT>
 TargetLoweringBase::getTypeLegalizationCost(const DataLayout &DL,
                                             Type *Ty) const {

llvm/test/CodeGen/X86/fast-isel-freeze.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc < %s                               -mtriple=x86_64-unknown-linux | FileCheck %s --check-prefix=SDAG
+; RUN: llc < %s -fast-isel -fast-isel-abort=1 -mtriple=x86_64-unknown-linux | FileCheck %s --check-prefix=FAST
+
+define i32 @freeze(i32 %t) {
+; SDAG-LABEL: freeze:
+; SDAG:       # %bb.0:
+; SDAG-NEXT:    movl $10, %eax
+; SDAG-NEXT:    xorl %edi, %eax
+; SDAG-NEXT:    retq
+;
+; FAST-LABEL: freeze:
+; FAST:       # %bb.0:
+; FAST-NEXT:    movl $10, %eax
+; FAST-NEXT:    xorl %edi, %eax
+; FAST-NEXT:    retq
+  %1 = freeze i32 %t
+  %2 = freeze i32 10
+  %3 = xor i32 %1, %2
+  ret i32 %3
+}

llvm/test/CodeGen/X86/fast-isel.ll

 }
 
 define void @load_store_i1(i1* %p, i1* %q) nounwind {
   %t = load i1, i1* %p
   store i1 %t, i1* %q
   ret void
 }
 
+define void @freeze_i32(i32 %x) {
+  %t = freeze i32 %x
+  ret void
+}
+
 @crash_test1x = external global <2 x i32>, align 8
 
 define void @crash_test1() nounwind ssp {
   %tmp = load <2 x i32>, <2 x i32>* @crash_test1x, align 8
   %neg = xor <2 x i32> %tmp, <i32 -1, i32 -1>
   ret void
 }
 

llvm/test/CodeGen/X86/freeze-legalize.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; Make sure that seldag legalization works correctly for freeze instruction.
+; RUN: llc -mtriple=i386-apple-darwin < %s 2>&1 | FileCheck %s
+
+define i64 @expand(i32 %x) {
+; CHECK-LABEL: expand:
+; CHECK:       ## %bb.0:
+; CHECK-NEXT:    movl $303174162, %eax ## imm = 0x12121212
+; CHECK-NEXT:    movl $875836468, %ecx ## imm = 0x34343434
+; CHECK-NEXT:    movl $1448498774, %edx ## imm = 0x56565656
+; CHECK-NEXT:    xorl %eax, %edx
+; CHECK-NEXT:    movl $2021161080, %eax ## imm = 0x78787878
+; CHECK-NEXT:    xorl %ecx, %eax
+; CHECK-NEXT:    retl
+  %y1 = freeze i64 1302123111658042420 ; 0x1212121234343434
+  %y2 = freeze i64 6221254864647256184 ; 0x5656565678787878
+  %t2 = xor i64 %y1, %y2
+  ret i64 %t2
+}
+
+
+define <2 x i64> @expand_vec(i32 %x) nounwind {
+; CHECK-LABEL: expand_vec:
+; CHECK:       ## %bb.0:
+; CHECK-NEXT:    pushl %ebx
+; CHECK-NEXT:    pushl %edi
+; CHECK-NEXT:    pushl %esi
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-NEXT:    movl $16843009, %ecx ## imm = 0x1010101
+; CHECK-NEXT:    movl $589505315, %edx ## imm = 0x23232323
+; CHECK-NEXT:    movl $303174162, %esi ## imm = 0x12121212
+; CHECK-NEXT:    movl $875836468, %edi ## imm = 0x34343434
+; CHECK-NEXT:    movl $1162167621, %ebx ## imm = 0x45454545
+; CHECK-NEXT:    xorl %ecx, %ebx
+; CHECK-NEXT:    movl $1734829927, %ecx ## imm = 0x67676767
+; CHECK-NEXT:    xorl %edx, %ecx
+; CHECK-NEXT:    movl $1448498774, %edx ## imm = 0x56565656
+; CHECK-NEXT:    xorl %esi, %edx
+; CHECK-NEXT:    movl $2021161080, %esi ## imm = 0x78787878
+; CHECK-NEXT:    xorl %edi, %esi
+; CHECK-NEXT:    movl %ebx, 12(%eax)
+; CHECK-NEXT:    movl %ecx, 8(%eax)
+; CHECK-NEXT:    movl %edx, 4(%eax)
+; CHECK-NEXT:    movl %esi, (%eax)
+; CHECK-NEXT:    popl %esi
+; CHECK-NEXT:    popl %edi
+; CHECK-NEXT:    popl %ebx
+; CHECK-NEXT:    retl $4
+  ; <0x1212121234343434, 0x101010123232323>
+  %y1 = freeze <2 x i64> <i64 1302123111658042420, i64 72340173410738979>
+  ; <0x5656565678787878, 0x4545454567676767>
+  %y2 = freeze <2 x i64> <i64 6221254864647256184, i64 4991471926399952743>
+  %t2 = xor <2 x i64> %y1, %y2
+  ret <2 x i64> %t2
+}
+
+define i10 @promote() {
+; CHECK-LABEL: promote:
+; CHECK:       ## %bb.0:
+; CHECK-NEXT:    movw $682, %cx ## imm = 0x2AA
+; CHECK-NEXT:    movw $992, %ax ## imm = 0x3E0
+; CHECK-NEXT:    addl %ecx, %eax
+; CHECK-NEXT:    ## kill: def $ax killed $ax killed $eax
+; CHECK-NEXT:    retl
+  %a = freeze i10 682
+  %b = freeze i10 992
+  %res = add i10 %a, %b
+  ret i10 %res
+}
+
+define <2 x i10> @promote_vec() {
+; CHECK-LABEL: promote_vec:
+; CHECK:       ## %bb.0:
+; CHECK-NEXT:    movw $125, %ax
+; CHECK-NEXT:    movw $682, %cx ## imm = 0x2AA
+; CHECK-NEXT:    movw $393, %dx ## imm = 0x189
+; CHECK-NEXT:    addl %eax, %edx
+; CHECK-NEXT:    movw $992, %ax ## imm = 0x3E0
+; CHECK-NEXT:    addl %ecx, %eax
+; CHECK-NEXT:    ## kill: def $ax killed $ax killed $eax
+; CHECK-NEXT:    ## kill: def $dx killed $dx killed $edx
+; CHECK-NEXT:    retl
+  %a = freeze <2 x i10> <i10 682, i10 125>
+  %b = freeze <2 x i10> <i10 992, i10 393>
+  %res = add <2 x i10> %a, %b
+  ret <2 x i10> %res
+}

llvm/test/CodeGen/X86/freeze.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=x86_64-unknown-linux-gnu < %s 2>&1 | FileCheck %s --check-prefix=X86ASM
+
+%struct.T = type { i32, i32 }
+
+define i32 @freeze_int() {
+; X86ASM-LABEL: freeze_int:
+; X86ASM:       # %bb.0:
+; X86ASM-NEXT:    imull %eax, %eax
+; X86ASM-NEXT:    retq
+  %y1 = freeze i32 undef
+  %t1 = mul i32 %y1, %y1
+  ret i32 %t1
+}
+
+define i5 @freeze_int2() {
+; X86ASM-LABEL: freeze_int2:
+; X86ASM:       # %bb.0:
+; X86ASM-NEXT:    mulb %al
+; X86ASM-NEXT:    retq
+  %y1 = freeze i5 undef
+  %t1 = mul i5 %y1, %y1
+  ret i5 %t1
+}
+
+define float @freeze_float() {
+; X86ASM-LABEL: freeze_float:
+; X86ASM:       # %bb.0:
+; X86ASM-NEXT:    addss %xmm0, %xmm0
+; X86ASM-NEXT:    retq
+  %y1 = freeze float undef
+  %t1 = fadd float %y1, %y1
+  ret float %t1
+}
+
+define <2 x i32> @freeze_ivec() {
+; X86ASM-LABEL: freeze_ivec:
+; X86ASM:       # %bb.0:
+; X86ASM-NEXT:    paddd %xmm0, %xmm0
+; X86ASM-NEXT:    retq
+  %y1 = freeze <2 x i32> undef
+  %t1 = add <2 x i32> %y1, %y1
+  ret <2 x i32> %t1
+}
+
+define i8* @freeze_ptr() {
+; X86ASM-LABEL: freeze_ptr:
+; X86ASM:       # %bb.0:
+; X86ASM-NEXT:    addq $4, %rax
+; X86ASM-NEXT:    retq
+  %y1 = freeze i8* undef
+  %t1 = getelementptr i8, i8* %y1, i64 4
+  ret i8* %t1
+}
+
+define i32 @freeze_struct() {
+; X86ASM-LABEL: freeze_struct:
+; X86ASM:       # %bb.0:
+; X86ASM-NEXT:    addl %eax, %eax
+; X86ASM-NEXT:    retq
+  %y1 = freeze %struct.T undef
+  %v1 = extractvalue %struct.T %y1, 0
+  %v2 = extractvalue %struct.T %y1, 1
+  %t1 = add i32 %v1, %v2
+  ret i32 %t1
+}
+
+define i32 @freeze_anonstruct() {
+; X86ASM-LABEL: freeze_anonstruct:
+; X86ASM:       # %bb.0:
+; X86ASM-NEXT:    addl %eax, %eax
+; X86ASM-NEXT:    retq
+  %y1 = freeze {i32, i32} undef
+  %v1 = extractvalue {i32, i32} %y1, 0
+  %v2 = extractvalue {i32, i32} %y1, 1
+  %t1 = add i32 %v1, %v2
+  ret i32 %t1
+}
+
+define i64 @freeze_array() {
+; X86ASM-LABEL: freeze_array:
+; X86ASM:       # %bb.0:
+; X86ASM-NEXT:    addq %rax, %rax
+; X86ASM-NEXT:    retq
+  %y1 = freeze [2 x i64] undef
+  %v1 = extractvalue [2 x i64] %y1, 0
+  %v2 = extractvalue [2 x i64] %y1, 1
+  %t1 = add i64 %v1, %v2
+  ret i64 %t1
+}