[X86] Adjust Keylocker store register num for encodekey128/256

[xiangzhangllvm]

Summary

We plan let encodekey128/256 not store "0" value xmm* regs into handle mem.

XMM0 -> Handle[127:0]; // AAD
XMM1 -> Handle[255:128]; // Integrity Tag
XMM2 -> Handle[383:256]; // CipherText
XMM4 := 0; // Reserved for future usage
XMM5 := 0; // Reserved for future usage
XMM6 := 0; // Reserved for future usage

[xiangzhangllvm]

Draft

[craig.topper]

These stores aren’t going to handle memory. The encode instruction read from handle memory. The stores are to an array provided by the caller of the intrinsic that only exists because we can’t return multiple values in C. If you don’t store them then those values in the user array are unitialized.

[LuoYuanke]

In D109488#2991661, @craig.topper wrote:

These stores aren’t going to handle memory. The encode instruction read from handle memory. The stores are to an array provided by the caller of the intrinsic that only exists because we can’t return multiple values in C. If you don’t store them then those values in the user array are unitialized.

The encodekey instruction generate key handler. User expect the buffer size of handler is 3*16 bytes for _mm_encodekey128_u32 and 4*16 for _mm_encodekey2568_u32. Storing extra data to the buffer either need extra instruction or need extra memory space for user.

[LuoYuanke]

Here (https://godbolt.org/z/EzPcsnz6f) is the difference codegen for clang and gcc. xmm4~xmm6 are zero value and are not part of key handler.

[craig.topper]

You need to update the doxygen comment in the intrinsic header too.

[craig.topper]

The pseudocode for _mm256_encodekey256_u32 is incorrect on the Intel Intrinsics Guide

__h[383:0] := WrapKey256(__key_lo[127:0], __key_hi[127:0], __htype)
dst[0] := IWKey.NoBackup
dst[4:1] := IWKey.KeySource[3:0]

That should be __h[511:0]

[xiangzhangllvm]

Yes, For encodekey256, That should be __h[511:0] , thanks!

[craig.topper]

In D109488#2993090, @xiangzhangllvm wrote:

Yes, For encodekey256, That should be __h[511:0] , thanks!

Is there a real place to file Intrinsic Guide issues? The "Questions? Issues?" takes you here https://community.intel.com/t5/Software/ct-p/software-products/topic/363747 but doesn't look like an obvious place to make sure the right people see it.

[xiangzhangllvm]

+ @FreddyYe , do you know Craig's question ?

[pengfei]

In D109488#2993123, @craig.topper wrote:

In D109488#2993090, @xiangzhangllvm wrote:

Yes, For encodekey256, That should be __h[511:0] , thanks!

Is there a real place to file Intrinsic Guide issues? The "Questions? Issues?" takes you here https://community.intel.com/t5/Software/ct-p/software-products/topic/363747 but doesn't look like an obvious place to make sure the right people see it.

Yes, we haven't found a good place for reporting bugs. Will try to fix the problem. Currently, @FreddyYe and I will handle it internally if we notice the intrinsics guide issues.

[xiangzhangllvm]

I am appling the access to modify the intrinsic guide, it may need more than 24h.

Could we let this patch in first? They are 2 independent jobs.

[LuoYuanke]

As Craig mentions we need to change the comments of header file keylockerintrin.h. For example change ((__m128i*)__h) to ((__m128i*)__h) + 5 to ((__m128i*)__h) to ((__m128i*)__h) + 2.

[craig.topper]

In D109488#2996636, @LuoYuanke wrote:

As Craig mentions we need to change the comments of header file keylockerintrin.h. For example change ((__m128i*)__h) to ((__m128i*)__h) + 5 to ((__m128i*)__h) to ((__m128i*)__h) + 2.

Also

/ MEM[h+511:h+384] := 0 Reserved for future usage

124 /// MEM[__h+639:__h+512] := 0 // Reserved for future usage
125 /// MEM[__h+767:__h+640] := 0 // Reserved for future usage

[xiangzhangllvm]

Update related comments for intrinsic API.

[xiangzhangllvm]

Done, thanks!

[craig.topper]

LGTM

clang/lib/CodeGen/CGBuiltin.cpp

// AVX512 bf16 intrinsics
   case X86::BI__builtin_ia32_psubusb128:
   case X86::BI__builtin_ia32_psubusw128:
     return EmitX86BinaryIntrinsic(*this, Ops, Intrinsic::usub_sat);
   case X86::BI__builtin_ia32_encodekey128_u32: {
     Intrinsic::ID IID = Intrinsic::x86_encodekey128;
 
     Value *Call = Builder.CreateCall(CGM.getIntrinsic(IID), {Ops[0], Ops[1]});
 
-    for (int i = 0; i < 6; ++i) {
+    for (int i = 0; i < 3; ++i) {
       Value *Extract = Builder.CreateExtractValue(Call, i + 1);
       Value *Ptr = Builder.CreateConstGEP1_32(Int8Ty, Ops[2], i * 16);
       Ptr = Builder.CreateBitCast(
           Ptr, llvm::PointerType::getUnqual(Extract->getType()));
       Builder.CreateAlignedStore(Extract, Ptr, Align(1));
     }
 
     return Builder.CreateExtractValue(Call, 0);
   }
   case X86::BI__builtin_ia32_encodekey256_u32: {
     Intrinsic::ID IID = Intrinsic::x86_encodekey256;
 
     Value *Call =
         Builder.CreateCall(CGM.getIntrinsic(IID), {Ops[0], Ops[1], Ops[2]});
 
-    for (int i = 0; i < 7; ++i) {
+    for (int i = 0; i < 4; ++i) {
       Value *Extract = Builder.CreateExtractValue(Call, i + 1);
       Value *Ptr = Builder.CreateConstGEP1_32(Int8Ty, Ops[3], i * 16);
       Ptr = Builder.CreateBitCast(
           Ptr, llvm::PointerType::getUnqual(Extract->getType()));
       Builder.CreateAlignedStore(Extract, Ptr, Align(1));
     }
 
     return Builder.CreateExtractValue(Call, 0);

clang/lib/Headers/keylockerintrin.h

  * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
  * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
  * THE SOFTWARE.
  *
  *===-----------------------------------------------------------------------===
  */
 
 #ifndef __IMMINTRIN_H
 #error "Never use <keylockerintrin.h> directly; include <immintrin.h> instead."
 #endif
 
 #ifndef _KEYLOCKERINTRIN_H
 #define _KEYLOCKERINTRIN_H
 
 #if !(defined(_MSC_VER) || defined(__SCE__)) || __has_feature(modules) ||      \
     defined(__KL__)
 

 /// dst := ZF
 /// OF := 0
 /// SF := 0
 /// AF := 0
 /// PF := 0
 /// CF := 0
 /// \endoperation
 static __inline__ void __DEFAULT_FN_ATTRS
 _mm_loadiwkey (unsigned int __ctl, __m128i __intkey,
                __m128i __enkey_lo, __m128i __enkey_hi) {
   __builtin_ia32_loadiwkey (__intkey, __enkey_lo, __enkey_hi, __ctl);
 }
 
 /// Wrap a 128-bit AES key from __key into a key handle and output in
-/// ((__m128i*)__h) to ((__m128i*)__h) + 5  and a 32-bit value as return.
+/// ((__m128i*)__h) to ((__m128i*)__h) + 2  and a 32-bit value as return.
 /// The explicit source operand __htype specifies handle restrictions.
 ///
 /// \headerfile <x86intrin.h>
 ///
 /// This intrinsic corresponds to the <c> ENCODEKEY128 </c> instructions.
 ///
 /// \operation
 /// InputKey[127:0] := __key[127:0]
 /// KeyMetadata[2:0] := __htype[2:0]
 /// KeyMetadata[23:3] := 0 // Reserved for future usage
 /// KeyMetadata[27:24] := 0 // KeyType is AES-128 (value of 0)
 /// KeyMetadata[127:28] := 0 // Reserved for future usage
 /// Handle[383:0] := WrapKey128(InputKey[127:0], KeyMetadata[127:0],
 ///                  IWKey.Integrity Key[127:0], IWKey.Encryption Key[255:0])
 /// dst[0] := IWKey.NoBackup
 /// dst[4:1] := IWKey.KeySource[3:0]
 /// dst[31:5] := 0
 /// MEM[__h+127:__h] := Handle[127:0]   // AAD
 /// MEM[__h+255:__h+128] := Handle[255:128] // Integrity Tag
 /// MEM[__h+383:__h+256] := Handle[383:256] // CipherText
-/// MEM[__h+511:__h+384] := 0 // Reserved for future usage
-/// MEM[__h+639:__h+512] := 0 // Reserved for future usage
-/// MEM[__h+767:__h+640] := 0 // Reserved for future usage
 /// OF := 0
 /// SF := 0
 /// ZF := 0
 /// AF := 0
 /// PF := 0
 /// CF := 0
 /// \endoperation
 static __inline__ unsigned int __DEFAULT_FN_ATTRS
 _mm_encodekey128_u32(unsigned int __htype, __m128i __key, void *__h) {
   return __builtin_ia32_encodekey128_u32(__htype, (__v2di)__key, __h);
 }
 
 /// Wrap a 256-bit AES key from __key_hi:__key_lo into a key handle, then
-/// output handle in ((__m128i*)__h) to ((__m128i*)__h) + 6 and
+/// output handle in ((__m128i*)__h) to ((__m128i*)__h) + 3 and
 /// a 32-bit value as return.
 /// The explicit source operand __htype specifies handle restrictions.
 ///
 /// \headerfile <x86intrin.h>
 ///
 /// This intrinsic corresponds to the <c> ENCODEKEY256 </c> instructions.
 ///
 /// \operation
 /// InputKey[127:0] := __key_lo[127:0]
 /// InputKey[255:128] := __key_hi[255:128]
 /// KeyMetadata[2:0] := __htype[2:0]
 /// KeyMetadata[23:3] := 0 // Reserved for future usage
 /// KeyMetadata[27:24] := 1 // KeyType is AES-256 (value of 1)
 /// KeyMetadata[127:28] := 0 // Reserved for future usage
 /// Handle[511:0] := WrapKey256(InputKey[255:0], KeyMetadata[127:0],
 ///                  IWKey.Integrity Key[127:0], IWKey.Encryption Key[255:0])
 /// dst[0] := IWKey.NoBackup
 /// dst[4:1] := IWKey.KeySource[3:0]
 /// dst[31:5] := 0
 /// MEM[__h+127:__h]   := Handle[127:0] // AAD
 /// MEM[__h+255:__h+128] := Handle[255:128] // Tag
 /// MEM[__h+383:__h+256] := Handle[383:256] // CipherText[127:0]
 /// MEM[__h+511:__h+384] := Handle[511:384] // CipherText[255:128]
-/// MEM[__h+639:__h+512] := 0 // Reserved for future usage
-/// MEM[__h+767:__h+640] := 0 // Reserved for future usage
-/// MEM[__h+895:__h+768] := 0 Integrity// Reserved for future usage
 /// OF := 0
 /// SF := 0
 /// ZF := 0
 /// AF := 0
 /// PF := 0
 /// CF := 0
 /// \endoperation
 static __inline__ unsigned int __DEFAULT_FN_ATTRS
 _mm_encodekey256_u32(unsigned int __htype, __m128i __key_lo, __m128i __key_hi,
                      void *__h) {
   return __builtin_ia32_encodekey256_u32(__htype, (__v2di)__key_lo,
                                          (__v2di)__key_hi, __h);
 }
 
 /// The AESENC128KL performs 10 rounds of AES to encrypt the __idata using
 /// the 128-bit key in the handle from the __h. It stores the result in the
 /// __odata. And return the affected ZF flag status.
 ///
 /// \headerfile <x86intrin.h>
 ///

 /// dst := ZF
 /// OF := 0
 /// SF := 0
 /// AF := 0
 /// PF := 0
 /// CF := 0
 /// \endoperation
 static __inline__ unsigned char __DEFAULT_FN_ATTRS
 _mm_aesenc128kl_u8(__m128i* __odata, __m128i __idata, const void *__h) {
   return __builtin_ia32_aesenc128kl_u8((__v2di *)__odata, (__v2di)__idata, __h);
 }
 
 /// The AESENC256KL performs 14 rounds of AES to encrypt the __idata using
 /// the 256-bit key in the handle from the __h. It stores the result in the
 /// __odata. And return the affected ZF flag status.
 ///
 /// \headerfile <x86intrin.h>
 ///

 /// dst := ZF
 /// OF := 0
 /// SF := 0
 /// AF := 0
 /// PF := 0
 /// CF := 0
 /// \endoperation
 static __inline__ unsigned char __DEFAULT_FN_ATTRS
 _mm_aesenc256kl_u8(__m128i* __odata, __m128i __idata, const void *__h) {
   return __builtin_ia32_aesenc256kl_u8((__v2di *)__odata, (__v2di)__idata, __h);
 }
 
 /// The AESDEC128KL performs 10 rounds of AES to decrypt the __idata using
 /// the 128-bit key in the handle from the __h. It stores the result in the
 /// __odata. And return the affected ZF flag status.
 ///
 /// \headerfile <x86intrin.h>
 ///

clang/test/CodeGen/X86/keylocker.c

 // CHECK64-NEXT:    [[TMP9:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP6]], 2
 // CHECK64-NEXT:    [[TMP10:%.*]] = getelementptr i8, i8* [[TMP5]], i32 16
 // CHECK64-NEXT:    [[TMP11:%.*]] = bitcast i8* [[TMP10]] to <2 x i64>*
 // CHECK64-NEXT:    store <2 x i64> [[TMP9]], <2 x i64>* [[TMP11]], align 1
 // CHECK64-NEXT:    [[TMP12:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP6]], 3
 // CHECK64-NEXT:    [[TMP13:%.*]] = getelementptr i8, i8* [[TMP5]], i32 32
 // CHECK64-NEXT:    [[TMP14:%.*]] = bitcast i8* [[TMP13]] to <2 x i64>*
 // CHECK64-NEXT:    store <2 x i64> [[TMP12]], <2 x i64>* [[TMP14]], align 1
-// CHECK64-NEXT:    [[TMP15:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP6]], 4
-// CHECK64-NEXT:    [[TMP16:%.*]] = getelementptr i8, i8* [[TMP5]], i32 48
-// CHECK64-NEXT:    [[TMP17:%.*]] = bitcast i8* [[TMP16]] to <2 x i64>*
-// CHECK64-NEXT:    store <2 x i64> [[TMP15]], <2 x i64>* [[TMP17]], align 1
-// CHECK64-NEXT:    [[TMP18:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP6]], 5
-// CHECK64-NEXT:    [[TMP19:%.*]] = getelementptr i8, i8* [[TMP5]], i32 64
-// CHECK64-NEXT:    [[TMP20:%.*]] = bitcast i8* [[TMP19]] to <2 x i64>*
-// CHECK64-NEXT:    store <2 x i64> [[TMP18]], <2 x i64>* [[TMP20]], align 1
-// CHECK64-NEXT:    [[TMP21:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP6]], 6
-// CHECK64-NEXT:    [[TMP22:%.*]] = getelementptr i8, i8* [[TMP5]], i32 80
-// CHECK64-NEXT:    [[TMP23:%.*]] = bitcast i8* [[TMP22]] to <2 x i64>*
-// CHECK64-NEXT:    store <2 x i64> [[TMP21]], <2 x i64>* [[TMP23]], align 1
-// CHECK64-NEXT:    [[TMP24:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP6]], 0
-// CHECK64-NEXT:    ret i32 [[TMP24]]
+// CHECK64-NEXT:    [[TMP15:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP6]], 0
+// CHECK64-NEXT:    ret i32 [[TMP15]]
 //
 // CHECK32-LABEL: @test_encodekey128_u32(
 // CHECK32-NEXT:  entry:
 // CHECK32-NEXT:    [[__HTYPE_ADDR_I:%.*]] = alloca i32, align 4
 // CHECK32-NEXT:    [[__KEY_ADDR_I:%.*]] = alloca <2 x i64>, align 16
 // CHECK32-NEXT:    [[__H_ADDR_I:%.*]] = alloca i8*, align 4
 // CHECK32-NEXT:    [[HTYPE_ADDR:%.*]] = alloca i32, align 4
 // CHECK32-NEXT:    [[KEY_ADDR:%.*]] = alloca <2 x i64>, align 16

 // CHECK32-NEXT:    [[TMP9:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP6]], 2
 // CHECK32-NEXT:    [[TMP10:%.*]] = getelementptr i8, i8* [[TMP5]], i32 16
 // CHECK32-NEXT:    [[TMP11:%.*]] = bitcast i8* [[TMP10]] to <2 x i64>*
 // CHECK32-NEXT:    store <2 x i64> [[TMP9]], <2 x i64>* [[TMP11]], align 1
 // CHECK32-NEXT:    [[TMP12:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP6]], 3
 // CHECK32-NEXT:    [[TMP13:%.*]] = getelementptr i8, i8* [[TMP5]], i32 32
 // CHECK32-NEXT:    [[TMP14:%.*]] = bitcast i8* [[TMP13]] to <2 x i64>*
 // CHECK32-NEXT:    store <2 x i64> [[TMP12]], <2 x i64>* [[TMP14]], align 1
-// CHECK32-NEXT:    [[TMP15:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP6]], 4
-// CHECK32-NEXT:    [[TMP16:%.*]] = getelementptr i8, i8* [[TMP5]], i32 48
-// CHECK32-NEXT:    [[TMP17:%.*]] = bitcast i8* [[TMP16]] to <2 x i64>*
-// CHECK32-NEXT:    store <2 x i64> [[TMP15]], <2 x i64>* [[TMP17]], align 1
-// CHECK32-NEXT:    [[TMP18:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP6]], 5
-// CHECK32-NEXT:    [[TMP19:%.*]] = getelementptr i8, i8* [[TMP5]], i32 64
-// CHECK32-NEXT:    [[TMP20:%.*]] = bitcast i8* [[TMP19]] to <2 x i64>*
-// CHECK32-NEXT:    store <2 x i64> [[TMP18]], <2 x i64>* [[TMP20]], align 1
-// CHECK32-NEXT:    [[TMP21:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP6]], 6
-// CHECK32-NEXT:    [[TMP22:%.*]] = getelementptr i8, i8* [[TMP5]], i32 80
-// CHECK32-NEXT:    [[TMP23:%.*]] = bitcast i8* [[TMP22]] to <2 x i64>*
-// CHECK32-NEXT:    store <2 x i64> [[TMP21]], <2 x i64>* [[TMP23]], align 1
-// CHECK32-NEXT:    [[TMP24:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP6]], 0
-// CHECK32-NEXT:    ret i32 [[TMP24]]
+// CHECK32-NEXT:    [[TMP15:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP6]], 0
+// CHECK32-NEXT:    ret i32 [[TMP15]]
 //
 unsigned int test_encodekey128_u32(unsigned int htype, __m128i key, void *h) {
   return _mm_encodekey128_u32(htype, key, h);
 }
 
 // CHECK64-LABEL: @test_encodekey256_u32(
 // CHECK64-NEXT:  entry:
 // CHECK64-NEXT:    [[__HTYPE_ADDR_I:%.*]] = alloca i32, align 4

 // CHECK64-NEXT:    [[TMP14:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP8]], 3
 // CHECK64-NEXT:    [[TMP15:%.*]] = getelementptr i8, i8* [[TMP7]], i32 32
 // CHECK64-NEXT:    [[TMP16:%.*]] = bitcast i8* [[TMP15]] to <2 x i64>*
 // CHECK64-NEXT:    store <2 x i64> [[TMP14]], <2 x i64>* [[TMP16]], align 1
 // CHECK64-NEXT:    [[TMP17:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP8]], 4
 // CHECK64-NEXT:    [[TMP18:%.*]] = getelementptr i8, i8* [[TMP7]], i32 48
 // CHECK64-NEXT:    [[TMP19:%.*]] = bitcast i8* [[TMP18]] to <2 x i64>*
 // CHECK64-NEXT:    store <2 x i64> [[TMP17]], <2 x i64>* [[TMP19]], align 1
-// CHECK64-NEXT:    [[TMP20:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP8]], 5
-// CHECK64-NEXT:    [[TMP21:%.*]] = getelementptr i8, i8* [[TMP7]], i32 64
-// CHECK64-NEXT:    [[TMP22:%.*]] = bitcast i8* [[TMP21]] to <2 x i64>*
-// CHECK64-NEXT:    store <2 x i64> [[TMP20]], <2 x i64>* [[TMP22]], align 1
-// CHECK64-NEXT:    [[TMP23:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP8]], 6
-// CHECK64-NEXT:    [[TMP24:%.*]] = getelementptr i8, i8* [[TMP7]], i32 80
-// CHECK64-NEXT:    [[TMP25:%.*]] = bitcast i8* [[TMP24]] to <2 x i64>*
-// CHECK64-NEXT:    store <2 x i64> [[TMP23]], <2 x i64>* [[TMP25]], align 1
-// CHECK64-NEXT:    [[TMP26:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP8]], 7
-// CHECK64-NEXT:    [[TMP27:%.*]] = getelementptr i8, i8* [[TMP7]], i32 96
-// CHECK64-NEXT:    [[TMP28:%.*]] = bitcast i8* [[TMP27]] to <2 x i64>*
-// CHECK64-NEXT:    store <2 x i64> [[TMP26]], <2 x i64>* [[TMP28]], align 1
-// CHECK64-NEXT:    [[TMP29:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP8]], 0
-// CHECK64-NEXT:    ret i32 [[TMP29]]
+// CHECK64-NEXT:    [[TMP20:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP8]], 0
+// CHECK64-NEXT:    ret i32 [[TMP20]]
 //
 // CHECK32-LABEL: @test_encodekey256_u32(
 // CHECK32-NEXT:  entry:
 // CHECK32-NEXT:    [[__HTYPE_ADDR_I:%.*]] = alloca i32, align 4
 // CHECK32-NEXT:    [[__KEY_LO_ADDR_I:%.*]] = alloca <2 x i64>, align 16
 // CHECK32-NEXT:    [[__KEY_HI_ADDR_I:%.*]] = alloca <2 x i64>, align 16
 // CHECK32-NEXT:    [[__H_ADDR_I:%.*]] = alloca i8*, align 4
 // CHECK32-NEXT:    [[HTYPE_ADDR:%.*]] = alloca i32, align 4

 // CHECK32-NEXT:    [[TMP14:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP8]], 3
 // CHECK32-NEXT:    [[TMP15:%.*]] = getelementptr i8, i8* [[TMP7]], i32 32
 // CHECK32-NEXT:    [[TMP16:%.*]] = bitcast i8* [[TMP15]] to <2 x i64>*
 // CHECK32-NEXT:    store <2 x i64> [[TMP14]], <2 x i64>* [[TMP16]], align 1
 // CHECK32-NEXT:    [[TMP17:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP8]], 4
 // CHECK32-NEXT:    [[TMP18:%.*]] = getelementptr i8, i8* [[TMP7]], i32 48
 // CHECK32-NEXT:    [[TMP19:%.*]] = bitcast i8* [[TMP18]] to <2 x i64>*
 // CHECK32-NEXT:    store <2 x i64> [[TMP17]], <2 x i64>* [[TMP19]], align 1
-// CHECK32-NEXT:    [[TMP20:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP8]], 5
-// CHECK32-NEXT:    [[TMP21:%.*]] = getelementptr i8, i8* [[TMP7]], i32 64
-// CHECK32-NEXT:    [[TMP22:%.*]] = bitcast i8* [[TMP21]] to <2 x i64>*
-// CHECK32-NEXT:    store <2 x i64> [[TMP20]], <2 x i64>* [[TMP22]], align 1
-// CHECK32-NEXT:    [[TMP23:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP8]], 6
-// CHECK32-NEXT:    [[TMP24:%.*]] = getelementptr i8, i8* [[TMP7]], i32 80
-// CHECK32-NEXT:    [[TMP25:%.*]] = bitcast i8* [[TMP24]] to <2 x i64>*
-// CHECK32-NEXT:    store <2 x i64> [[TMP23]], <2 x i64>* [[TMP25]], align 1
-// CHECK32-NEXT:    [[TMP26:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP8]], 7
-// CHECK32-NEXT:    [[TMP27:%.*]] = getelementptr i8, i8* [[TMP7]], i32 96
-// CHECK32-NEXT:    [[TMP28:%.*]] = bitcast i8* [[TMP27]] to <2 x i64>*
-// CHECK32-NEXT:    store <2 x i64> [[TMP26]], <2 x i64>* [[TMP28]], align 1
-// CHECK32-NEXT:    [[TMP29:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP8]], 0
-// CHECK32-NEXT:    ret i32 [[TMP29]]
+// CHECK32-NEXT:    [[TMP20:%.*]] = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } [[TMP8]], 0
+// CHECK32-NEXT:    ret i32 [[TMP20]]
 //
 unsigned int test_encodekey256_u32(unsigned int htype, __m128i key_lo, __m128i key_hi, void *h) {
   return _mm_encodekey256_u32(htype, key_lo, key_hi, h);
 }
 
 // CHECK64-LABEL: @test_mm_aesenc256kl_u8(
 // CHECK64-NEXT:  entry:
 // CHECK64-NEXT:    [[__ODATA_ADDR_I:%.*]] = alloca <2 x i64>*, align 8

llvm/test/CodeGen/X86/keylocker-intrinsics.ll

 entry:
   tail call void @llvm.x86.loadiwkey(<2 x i64> %intkey, <2 x i64> %enkey_lo, <2 x i64> %enkey_hi, i32 %ctl)
   ret void
 }
 
 define i32 @test_encodekey128_u32(i32 %htype, <2 x i64> %key, <2 x i64>* nocapture %h0, <2 x i64>* nocapture %h1, <2 x i64>* nocapture %h2, <2 x i64>* nocapture %h3, <2 x i64>* nocapture %h4, <2 x i64>* nocapture %h5) nounwind {
 ; X64-LABEL: test_encodekey128_u32:
 ; X64:       # %bb.0: # %entry
-; X64-NEXT:    movq {{[0-9]+}}(%rsp), %r10
 ; X64-NEXT:    encodekey128 %edi, %eax
 ; X64-NEXT:    movaps %xmm0, (%rsi)
 ; X64-NEXT:    movaps %xmm1, (%rdx)
 ; X64-NEXT:    movaps %xmm2, (%rcx)
-; X64-NEXT:    movaps %xmm4, (%r8)
-; X64-NEXT:    movaps %xmm5, (%r9)
-; X64-NEXT:    movaps %xmm6, (%r10)
 ; X64-NEXT:    retq
 ;
 ; X32-LABEL: test_encodekey128_u32:
 ; X32:       # %bb.0: # %entry
-; X32-NEXT:    pushl %ebp
-; X32-NEXT:    pushl %ebx
-; X32-NEXT:    pushl %edi
 ; X32-NEXT:    pushl %esi
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %edx
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; X32-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; X32-NEXT:    movl {{[0-9]+}}(%esp), %ebx
-; X32-NEXT:    movl {{[0-9]+}}(%esp), %ebp
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X32-NEXT:    encodekey128 %eax, %eax
-; X32-NEXT:    vmovaps %xmm0, (%ebp)
-; X32-NEXT:    vmovaps %xmm1, (%ebx)
-; X32-NEXT:    vmovaps %xmm2, (%edi)
-; X32-NEXT:    vmovaps %xmm4, (%esi)
-; X32-NEXT:    vmovaps %xmm5, (%edx)
-; X32-NEXT:    vmovaps %xmm6, (%ecx)
+; X32-NEXT:    vmovaps %xmm0, (%esi)
+; X32-NEXT:    vmovaps %xmm1, (%edx)
+; X32-NEXT:    vmovaps %xmm2, (%ecx)
 ; X32-NEXT:    popl %esi
-; X32-NEXT:    popl %edi
-; X32-NEXT:    popl %ebx
-; X32-NEXT:    popl %ebp
 ; X32-NEXT:    retl
 entry:
   %0 = tail call { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } @llvm.x86.encodekey128(i32 %htype, <2 x i64> %key)
   %1 = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } %0, 1
   store <2 x i64> %1, <2 x i64>* %h0, align 16
   %2 = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } %0, 2
   store <2 x i64> %2, <2 x i64>* %h1, align 16
   %3 = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } %0, 3
   store <2 x i64> %3, <2 x i64>* %h2, align 16
-  %4 = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } %0, 4
-  store <2 x i64> %4, <2 x i64>* %h3, align 16
-  %5 = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } %0, 5
-  store <2 x i64> %5, <2 x i64>* %h4, align 16
-  %6 = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } %0, 6
-  store <2 x i64> %6, <2 x i64>* %h5, align 16
-  %7 = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } %0, 0
-  ret i32 %7
+  %4 = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } %0, 0
+  ret i32 %4
 }
 
 define i32 @test_encodekey256_u32(i32 %htype, <2 x i64> %key_lo, <2 x i64> %key_hi, <2 x i64>* nocapture %h0, <2 x i64>* nocapture %h1, <2 x i64>* nocapture %h2, <2 x i64>* nocapture %h3, <2 x i64>* nocapture %h4, <2 x i64>* nocapture %h5, <2 x      i64>* nocapture readnone %h6) nounwind {
 ; X64-LABEL: test_encodekey256_u32:
 ; X64:       # %bb.0: # %entry
-; X64-NEXT:    movq {{[0-9]+}}(%rsp), %r10
 ; X64-NEXT:    encodekey256 %edi, %eax
 ; X64-NEXT:    movaps %xmm0, (%rsi)
 ; X64-NEXT:    movaps %xmm1, (%rdx)
 ; X64-NEXT:    movaps %xmm2, (%rcx)
 ; X64-NEXT:    movaps %xmm3, (%r8)
-; X64-NEXT:    movaps %xmm4, (%r9)
-; X64-NEXT:    movaps %xmm5, (%r10)
 ; X64-NEXT:    retq
 ;
 ; X32-LABEL: test_encodekey256_u32:
 ; X32:       # %bb.0: # %entry
-; X32-NEXT:    pushl %ebp
-; X32-NEXT:    pushl %ebx
 ; X32-NEXT:    pushl %edi
 ; X32-NEXT:    pushl %esi
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %edx
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %esi
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %edi
-; X32-NEXT:    movl {{[0-9]+}}(%esp), %ebx
-; X32-NEXT:    movl {{[0-9]+}}(%esp), %ebp
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X32-NEXT:    encodekey256 %eax, %eax
-; X32-NEXT:    vmovaps %xmm0, (%ebp)
-; X32-NEXT:    vmovaps %xmm1, (%ebx)
-; X32-NEXT:    vmovaps %xmm2, (%edi)
-; X32-NEXT:    vmovaps %xmm3, (%esi)
-; X32-NEXT:    vmovaps %xmm4, (%edx)
-; X32-NEXT:    vmovaps %xmm5, (%ecx)
+; X32-NEXT:    vmovaps %xmm0, (%edi)
+; X32-NEXT:    vmovaps %xmm1, (%esi)
+; X32-NEXT:    vmovaps %xmm2, (%edx)
+; X32-NEXT:    vmovaps %xmm3, (%ecx)
 ; X32-NEXT:    popl %esi
 ; X32-NEXT:    popl %edi
-; X32-NEXT:    popl %ebx
-; X32-NEXT:    popl %ebp
 ; X32-NEXT:    retl
 entry:
   %0 = tail call { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } @llvm.x86.encodekey256(i32 %htype, <2 x i64> %key_lo, <2 x i64> %key_hi)
   %1 = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } %0, 1
   store <2 x i64> %1, <2 x i64>* %h0, align 16
   %2 = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } %0, 2
   store <2 x i64> %2, <2 x i64>* %h1, align 16
   %3 = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } %0, 3
   store <2 x i64> %3, <2 x i64>* %h2, align 16
   %4 = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } %0, 4
   store <2 x i64> %4, <2 x i64>* %h3, align 16
-  %5 = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } %0, 5
-  store <2 x i64> %5, <2 x i64>* %h4, align 16
-  %6 = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } %0, 6
-  store <2 x i64> %6, <2 x i64>* %h5, align 16
-  %7 = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } %0, 0
-  ret i32 %7
+  %5 = extractvalue { i32, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } %0, 0
+  ret i32 %5
 }
 
 define i8 @test_mm_aesenc128kl_u8(<2 x i64> %data, i8* %h, <2 x i64>* %out) {
 ; X64-LABEL: test_mm_aesenc128kl_u8:
 ; X64:       # %bb.0: # %entry
 ; X64-NEXT:    aesenc128kl (%rdi), %xmm0
 ; X64-NEXT:    sete %al
 ; X64-NEXT:    movaps %xmm0, (%rsi)