This is an archive of the discontinued LLVM Phabricator instance.

Differential D89105

[X86] Support Intel avxvnni
ClosedPublic

Authored by LiuChen3 on Oct 9 2020, 2:04 AM.

Details

Reviewers
craig.topper
pengfei
xiangzhangllvm
LuoYuanke
RKSimon
annita.zhang
Commits
rG756f59784108: [X86] Support Intel avxvnni
Summary

This patch mainly made the following changes:

  1. Support AVX-VNNI instructions;
  2. Support more complex FeatureList. By default, ',' is prior than '|'. And parentheses is supported;
  3. Introduce ExplicitVEXPrefix flag so that vpdpbusd/vpdpbusds/vpdpbusds/vpdpbusds instructions only use vex-encoding when user explicity add {vex} prefix.

Diff Detail

Event Timeline

LiuChen3 created this revision.Oct 9 2020, 2:04 AM
Herald added a project: Restricted Project. · View Herald TranscriptOct 9 2020, 2:04 AM
Herald added subscribers: llvm-commits, dang, hiraditya and 2 others. · View Herald Transcript
LiuChen3 requested review of this revision.Oct 9 2020, 2:04 AM
RKSimon added a comment.Oct 9 2020, 2:38 AM

Please can you add an entry to the 12.0 release notes?

clang/include/clang/Basic/BuiltinsNVPTX.def
46

Pull this out? This and the CodeGenFunction.* changes seem to be worth a separate patch

clang/lib/Headers/avxvnniintrin.h
31

Is having a commonvnniintrin.h header the approach gcc/icc are taking?

39

Please can you add doxygen descriptions to each intrinsic.

LiuChen3 updated this revision to Diff 297392.Oct 10 2020, 12:56 AM

Address comments.

LiuChen3 marked an inline comment as done.Oct 10 2020, 1:17 AM
clang/include/clang/Basic/BuiltinsNVPTX.def
46

Thanks for your review.
I will separate this part. I want to keep this part here for now until another patch is merged in.

clang/lib/Headers/avxvnniintrin.h
31

GCC doesn't have commonvnniintrin.h headr, they only have avxvnniintrin.h. For ICC, I will check with this and give you answer later.

LiuChen3 added a comment.Oct 12 2020, 1:37 AM
clang/lib/Headers/avxvnniintrin.h
31

GCC and ICC doesn't break intrinsics into multiple headers.

craig.topper mentioned this in D89184: Support complex target features combinations.Oct 13 2020, 10:50 AM
craig.topper added a parent revision: D89184: Support complex target features combinations.
LiuChen3 updated this revision to Diff 298920.Oct 18 2020, 8:05 PM

Rebase.

RKSimon added inline comments.Oct 22 2020, 2:14 AM
clang/lib/Headers/avxvnniintrin.h
31

Is there anyway that we can avoid this as well then? I get worried when header layouts start to diverge...

Herald added a subscriber: dexonsmith. · View Herald TranscriptOct 22 2020, 2:14 AM
dexonsmith removed a subscriber: dexonsmith.Oct 22 2020, 8:03 AM
craig.topper added a comment.Oct 22 2020, 10:21 AM

We also need to add avxvnni versions of the patterns from this commit along with appropriate testing. Maybe just split the 128/256 bit out of avx512vnni.ll and into a separate test with multiple RUN lines. That should all be done in a separate patch.

{code}
commit cc4b0596b1b0e58672e4151396c7b804eccaf273
Author: Craig Topper <craig.topper@intel.com>
Date: Sat Aug 24 23:14:57 2019 +0000

[X86] Add isel patterns to match vpdpwssd avx512vnni instruction from add+pmaddwd nodes.

{code}

pengfei added inline comments.Oct 22 2020, 6:06 PM
clang/lib/Headers/avxvnniintrin.h
31

Hi Simon, why we need to keep the same layout with GCC and ICC?
We always create new header files for new instructions. We don't allow user include these headers directly. So I think it doesn't matter if GCC and ICC create the same headers or not.

craig.topper added inline comments.Oct 22 2020, 6:23 PM
clang/lib/Headers/avxvnniintrin.h
31

As far as I know our header layout is largely the same as gcc for existing intrinsics. Can we just keep them the same?

clang/lib/Headers/commonvnniintrin.h
49

Arguments to macros shouldn't start with __

50

Because this is a macro this needs to have parentheses around S, A, and B.

pengfei added inline comments.Oct 22 2020, 10:35 PM
clang/lib/Headers/avxvnniintrin.h
33

Maybe we can move commonvnniintrin.h to immintrin.h. Then we don't have a different layout.

#if !(defined(_MSC_VER) || defined(__SCE__)) || __has_feature(modules) ||      \
     defined(__AVX512VNNI__) || (defined(__AVX512VL__) && defined(__AVX512VNNI__))
#include <commonvnniintrin.h>
#endif
craig.topper added inline comments.Oct 22 2020, 10:39 PM
clang/lib/Headers/avxvnniintrin.h
33

gcc appears to have just put the common intrinsics in avxvnniintrin.h. Why can't we do that?

pengfei added inline comments.Oct 22 2020, 10:52 PM
clang/lib/Headers/avxvnniintrin.h
33

I asked GCC guys. They don't use macro like defined(__AVXVNNI__). If we put it in avxvnniintrin.h, we need handle macro AVX512VNNI in avxvnniintrin.h. It's bit confused.

@LiuChen3 I meant __AVXVNNI__ for the first __AVX512VNNI__ in last comment.

pengfei added inline comments.Oct 22 2020, 11:36 PM
clang/lib/Headers/avxvnniintrin.h
33

Or maybe we can put the common intrinsics in avx512vlvnniintrin.h and not to handle macro __AVXVNNI__ int it.
I think it's reasonable. For Linux, we always include all headers. For Windows, we can only allow user use the _avx_ intrinscis for AVXVNNI, which is compatible with MSVC.

craig.topper added inline comments.Oct 22 2020, 11:40 PM
clang/lib/Headers/avxvnniintrin.h
33

Minor clarification. It's not Linux vs Windows. Its -ms-compatibility that controls it. You can use clang on Windows without enabling MSVC compatibility. In which case MSC_VER won't be defined.

pengfei added inline comments.Oct 22 2020, 11:42 PM
clang/lib/Headers/avxvnniintrin.h
33

Right. Thank @craig.topper

RKSimon added inline comments.Oct 23 2020, 1:19 AM
clang/test/CodeGen/avxvnni-builtins.c
2

Please can you move this test file under the X86 subdirectory?

LiuChen3 updated this revision to Diff 300564.Oct 25 2020, 6:35 PM
  1. move the commonvnniintrin.h to the avx512vlvnniintrin.h.
  2. move the testcase avxvnni-builtins.c to X86 subdirectory.
craig.topper added a subscriber: bkramer.Oct 25 2020, 7:03 PM
craig.topper added inline comments.
llvm/lib/Support/X86TargetParser.cpp
200–202

Add here too.

llvm/lib/Target/X86/X86.td
762

Please add this to Alderlake which was added to X86.td by @bkramer yesterday

craig.topper added a comment.Oct 25 2020, 7:07 PM

Also need clang/test/Preprocessor/predefined-arch-macros.c updates for sapphirerapids and alderlake to check that AVXVNNI is defined.

LiuChen3 updated this revision to Diff 300569.Oct 25 2020, 7:58 PM

Adding avxvnni to Alder Lake.

pengfei added inline comments.Oct 25 2020, 8:04 PM
clang/test/Preprocessor/predefined-arch-macros.c
1637 ↗(On Diff #300569)

You need to add to sapphirerapids too.

1707 ↗(On Diff #300569)

You need to add to sapphirerapids too.

llvm/docs/ReleaseNotes.rst
124 ↗(On Diff #300569)

You also need to add it to Clang release notes.

LiuChen3 updated this revision to Diff 300570.Oct 25 2020, 8:24 PM

Address comments.

pengfei added inline comments.Oct 26 2020, 12:17 AM
clang/lib/Headers/avxvnniintrin.h
31

I think we may still need add some information for intrinsics that can be used for AVXVNNI but not declared here. A proposal might be:

/* Below intrinsics defined in avx512vlvnniintrin.h can be used for */
/* AVXVNNI except used for compatibility with msvc.                 */
// #define _mm256_dpbusd_epi32(S, A, B)
// #define _mm256_dpbusds_epi32(S, A, B)
// #define _mm256_dpwssd_epi32(S, A, B)
...
pengfei added inline comments.Oct 29 2020, 7:38 PM
clang/lib/Headers/avxvnniintrin.h
31

I had a look at the Doxygen grammar. Maybe we can use like below:

/* Below intrinsics defined in avx512vlvnniintrin.h can be used for */
/* AVXVNNI except used for compatibility with msvc.                 */
/// \fn __m256i _mm256_dpbusd_epi32(__m256i __S, __m256i __A, __m256i __B)
/// \fn __m256i _mm256_dpbusds_epi32(__m256i __S, __m256i __A, __m256i __B)
/// \fn __m256i _mm256_dpwssd_epi32(__m256i __S, __m256i __A, __m256i __B)
...
LiuChen3 updated this revision to Diff 301814.Oct 29 2020, 10:02 PM
pengfei added inline comments.Oct 29 2020, 10:21 PM
clang/lib/Headers/avxvnniintrin.h
32

Missing one line comment?

clang/lib/Headers/immintrin.h
142–147

Change back the spaces.

llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp
3739

Missing check for VEX2?

LiuChen3 updated this revision to Diff 301816.Oct 29 2020, 10:50 PM

Address comments

pengfei accepted this revision.Oct 29 2020, 11:06 PM

LGTM with little nit:

llvm/test/CodeGen/X86/avx-vnni/avx_vnni-intrinsics.ll
2

Do we need new directory for the tests?

3

I don't see difference between AVX-X86 and AVX-X64, maybe you can use AVX for both, or AVX,AVX-X86 and AVX,AVX-X64 if they do have some differences. And remove CHECK since it never been used.

This revision is now accepted and ready to land.Oct 29 2020, 11:06 PM
LiuChen3 updated this revision to Diff 301825.Oct 30 2020, 12:39 AM
  1. Move the testcase from avx-vnni/ to test/CodeGen/X86/.
  2. Refine the Run line in avx_vnni-intrinsics.ll
pengfei accepted this revision.Oct 30 2020, 12:47 AM

LGTM. Thanks. Better to wait one day or two to see if others object.

LiuChen3 added a comment.Oct 30 2020, 12:51 AM
In D89105#2363927, @pengfei wrote:

LGTM. Thanks. Better to wait one day or two to see if others object.

Sure. Thanks for your review.

RKSimon accepted this revision.Oct 30 2020, 2:39 AM

LGTM cheers

craig.topper accepted this revision.Oct 30 2020, 11:33 AM

LGTM to me with a couple minors that can be fixed on commit.

clang/test/Preprocessor/predefined-arch-macros.c
1828 ↗(On Diff #301825)

doesn't this NOT need to be above AVXNNI to be effective since that's where AVX512 would go alphabetically?

llvm/lib/Target/X86/MCTargetDesc/X86InstPrinterCommon.cpp
348–353

Put parentheses around the & parts. Similar to lock and notrack above. I don't want to think about operator precedence.

Closed by commit rG756f59784108: [X86] Support Intel avxvnni (authored by LiuChen3). · Explain WhyOct 30 2020, 10:18 PM
This revision was automatically updated to reflect the committed changes.
LiuChen3 added a commit: rG756f59784108: [X86] Support Intel avxvnni.
Herald added a project: Restricted Project. · View Herald TranscriptOct 30 2020, 10:18 PM
Herald added a subscriber: cfe-commits. · View Herald Transcript
LiuChen3 marked 2 inline comments as done.Oct 30 2020, 10:20 PM

Thanks for all of your review!

RKSimon added inline comments.Jan 24 2022, 2:51 PM
clang/lib/Headers/cpuid.h
198

@LiuChen3 https://github.com/llvm/llvm-project/issues/53392 reports this should be 0x00000010?

craig.topper added inline comments.Jan 24 2022, 3:28 PM
clang/lib/Headers/cpuid.h
198

Maybe we should start writing these as 1 << 4. This is not the first shifted by one issue we've had in this file.

LiuChen3 added inline comments.Jan 24 2022, 10:16 PM
clang/lib/Headers/cpuid.h
198

It my fault. I have a fix here: https://reviews.llvm.org/D118103 .

Revision Contents

PathSize
clang/
docs/
2 lines
include/
clang/
Basic/
2 lines
16 lines
Driver/
2 lines
lib/
Basic/
Targets/
1 line
6 lines
CodeGen/
56 lines
55 lines
Headers/
2 lines
58 lines
89 lines
183 lines
1 line
7 lines
test/
CodeGen/
4 lines
99 lines
470 lines
264 lines
Driver/
5 lines
Preprocessor/
14 lines
unittests/
CodeGen/
1 line
25 lines
llvm/
include/
llvm/
Support/
1 line
lib/
Support/
1 line
4 lines
Target/
X86/
AsmParser/
6 lines
MCTargetDesc/
6 lines
4 lines
4 lines
29 lines
8 lines
5 lines
4 lines
2 lines
42 lines
4 lines
test/
CodeGen/
X86/
avx-vnni/
213 lines
242 lines
MC/
Disassembler/
X86/
170 lines
170 lines
170 lines
170 lines
X86/
226 lines
226 lines
226 lines
226 lines

Diff 297148

clang/docs/ClangCommandLineReference.rst

Show First 20 Lines • Show All 3,177 Lines • ▼ Show 20 Lines
.. option:: -mavx512vl, -mno-avx512vl .. option:: -mavx512vl, -mno-avx512vl
.. option:: -mavx512vnni, -mno-avx512vnni .. option:: -mavx512vnni, -mno-avx512vnni
.. option:: -mavx512vp2intersect, -mno-avx512vp2intersect .. option:: -mavx512vp2intersect, -mno-avx512vp2intersect
.. option:: -mavx512vpopcntdq, -mno-avx512vpopcntdq .. option:: -mavx512vpopcntdq, -mno-avx512vpopcntdq
.. option:: -mavxvnni, -mno-avxvnni
.. option:: -mbmi, -mno-bmi .. option:: -mbmi, -mno-bmi
.. option:: -mbmi2, -mno-bmi2 .. option:: -mbmi2, -mno-bmi2
.. option:: -mcldemote, -mno-cldemote .. option:: -mcldemote, -mno-cldemote
.. option:: -mclflushopt, -mno-clflushopt .. option:: -mclflushopt, -mno-clflushopt
▲ Show 20 Lines • Show All 482 Lines • Show Last 20 Lines

clang/include/clang/Basic/BuiltinsNVPTX.def

Show All 37 Lines
#define PTX70 "ptx70" #define PTX70 "ptx70"
#define PTX65 "ptx65|" PTX70 #define PTX65 "ptx65|" PTX70
#define PTX64 "ptx64|" PTX65 #define PTX64 "ptx64|" PTX65
#define PTX63 "ptx63|" PTX64 #define PTX63 "ptx63|" PTX64
#define PTX61 "ptx61|" PTX63 #define PTX61 "ptx61|" PTX63
#define PTX60 "ptx60|" PTX61 #define PTX60 "ptx60|" PTX61
#pragma push_macro("AND") #pragma push_macro("AND")
#define AND(a, b) a "," b #define AND(a, b) "(" a "),(" b ")"
RKSimonUnsubmitted
Not Done
ReplyInline Actions

Pull this out? This and the CodeGenFunction.* changes seem to be worth a separate patch

RKSimon: Pull this out? This and the CodeGenFunction.* changes seem to be worth a separate patch
LiuChen3AuthorUnsubmitted
Done
ReplyInline Actions

Thanks for your review.
I will separate this part. I want to keep this part here for now until another patch is merged in.

LiuChen3: Thanks for your review. I will separate this part. I want to keep this part here for now until…
// Special Registers // Special Registers
BUILTIN(__nvvm_read_ptx_sreg_tid_x, "i", "nc") BUILTIN(__nvvm_read_ptx_sreg_tid_x, "i", "nc")
BUILTIN(__nvvm_read_ptx_sreg_tid_y, "i", "nc") BUILTIN(__nvvm_read_ptx_sreg_tid_y, "i", "nc")
BUILTIN(__nvvm_read_ptx_sreg_tid_z, "i", "nc") BUILTIN(__nvvm_read_ptx_sreg_tid_z, "i", "nc")
BUILTIN(__nvvm_read_ptx_sreg_tid_w, "i", "nc") BUILTIN(__nvvm_read_ptx_sreg_tid_w, "i", "nc")
▲ Show 20 Lines • Show All 682 Lines • Show Last 20 Lines

clang/include/clang/Basic/BuiltinsX86.def

Show First 20 Lines • Show All 954 Lines • ▼ Show 20 Lines
TARGET_BUILTIN(__builtin_ia32_alignd512, "V16iV16iV16iIi", "ncV:512:", "avx512f") TARGET_BUILTIN(__builtin_ia32_alignd512, "V16iV16iV16iIi", "ncV:512:", "avx512f")
TARGET_BUILTIN(__builtin_ia32_alignd128, "V4iV4iV4iIi", "ncV:128:", "avx512vl") TARGET_BUILTIN(__builtin_ia32_alignd128, "V4iV4iV4iIi", "ncV:128:", "avx512vl")
TARGET_BUILTIN(__builtin_ia32_alignd256, "V8iV8iV8iIi", "ncV:256:", "avx512vl") TARGET_BUILTIN(__builtin_ia32_alignd256, "V8iV8iV8iIi", "ncV:256:", "avx512vl")
TARGET_BUILTIN(__builtin_ia32_alignq128, "V2OiV2OiV2OiIi", "ncV:128:", "avx512vl") TARGET_BUILTIN(__builtin_ia32_alignq128, "V2OiV2OiV2OiIi", "ncV:128:", "avx512vl")
TARGET_BUILTIN(__builtin_ia32_alignq256, "V4OiV4OiV4OiIi", "ncV:256:", "avx512vl") TARGET_BUILTIN(__builtin_ia32_alignq256, "V4OiV4OiV4OiIi", "ncV:256:", "avx512vl")
TARGET_BUILTIN(__builtin_ia32_extractf64x4_mask, "V4dV8dIiV4dUc", "ncV:512:", "avx512f") TARGET_BUILTIN(__builtin_ia32_extractf64x4_mask, "V4dV8dIiV4dUc", "ncV:512:", "avx512f")
TARGET_BUILTIN(__builtin_ia32_extractf32x4_mask, "V4fV16fIiV4fUc", "ncV:512:", "avx512f") TARGET_BUILTIN(__builtin_ia32_extractf32x4_mask, "V4fV16fIiV4fUc", "ncV:512:", "avx512f")
TARGET_BUILTIN(__builtin_ia32_vpdpbusd128, "V4iV4iV4iV4i", "ncV:128:", "avx512vl,avx512vnni") TARGET_BUILTIN(__builtin_ia32_vpdpbusd128, "V4iV4iV4iV4i", "ncV:128:", "avx512vl,avx512vnni|avxvnni")
TARGET_BUILTIN(__builtin_ia32_vpdpbusd256, "V8iV8iV8iV8i", "ncV:256:", "avx512vl,avx512vnni") TARGET_BUILTIN(__builtin_ia32_vpdpbusd256, "V8iV8iV8iV8i", "ncV:256:", "avx512vl,avx512vnni|avxvnni")
TARGET_BUILTIN(__builtin_ia32_vpdpbusd512, "V16iV16iV16iV16i", "ncV:512:", "avx512vnni") TARGET_BUILTIN(__builtin_ia32_vpdpbusd512, "V16iV16iV16iV16i", "ncV:512:", "avx512vnni")
TARGET_BUILTIN(__builtin_ia32_vpdpbusds128, "V4iV4iV4iV4i", "ncV:128:", "avx512vl,avx512vnni") TARGET_BUILTIN(__builtin_ia32_vpdpbusds128, "V4iV4iV4iV4i", "ncV:128:", "avx512vl,avx512vnni|avxvnni")
TARGET_BUILTIN(__builtin_ia32_vpdpbusds256, "V8iV8iV8iV8i", "ncV:256:", "avx512vl,avx512vnni") TARGET_BUILTIN(__builtin_ia32_vpdpbusds256, "V8iV8iV8iV8i", "ncV:256:", "avx512vl,avx512vnni|avxvnni")
TARGET_BUILTIN(__builtin_ia32_vpdpbusds512, "V16iV16iV16iV16i", "ncV:512:", "avx512vnni") TARGET_BUILTIN(__builtin_ia32_vpdpbusds512, "V16iV16iV16iV16i", "ncV:512:", "avx512vnni")
TARGET_BUILTIN(__builtin_ia32_vpdpwssd128, "V4iV4iV4iV4i", "ncV:128:", "avx512vl,avx512vnni") TARGET_BUILTIN(__builtin_ia32_vpdpwssd128, "V4iV4iV4iV4i", "ncV:128:", "avx512vl,avx512vnni|avxvnni")
TARGET_BUILTIN(__builtin_ia32_vpdpwssd256, "V8iV8iV8iV8i", "ncV:256:", "avx512vl,avx512vnni") TARGET_BUILTIN(__builtin_ia32_vpdpwssd256, "V8iV8iV8iV8i", "ncV:256:", "avx512vl,avx512vnni|avxvnni")
TARGET_BUILTIN(__builtin_ia32_vpdpwssd512, "V16iV16iV16iV16i", "ncV:512:", "avx512vnni") TARGET_BUILTIN(__builtin_ia32_vpdpwssd512, "V16iV16iV16iV16i", "ncV:512:", "avx512vnni")
TARGET_BUILTIN(__builtin_ia32_vpdpwssds128, "V4iV4iV4iV4i", "ncV:128:", "avx512vl,avx512vnni") TARGET_BUILTIN(__builtin_ia32_vpdpwssds128, "V4iV4iV4iV4i", "ncV:128:", "avx512vl,avx512vnni|avxvnni")
TARGET_BUILTIN(__builtin_ia32_vpdpwssds256, "V8iV8iV8iV8i", "ncV:256:", "avx512vl,avx512vnni") TARGET_BUILTIN(__builtin_ia32_vpdpwssds256, "V8iV8iV8iV8i", "ncV:256:", "avx512vl,avx512vnni|avxvnni")
TARGET_BUILTIN(__builtin_ia32_vpdpwssds512, "V16iV16iV16iV16i", "ncV:512:", "avx512vnni") TARGET_BUILTIN(__builtin_ia32_vpdpwssds512, "V16iV16iV16iV16i", "ncV:512:", "avx512vnni")
TARGET_BUILTIN(__builtin_ia32_gather3div2df, "V2dV2dvC*V2OiUcIi", "nV:128:", "avx512vl") TARGET_BUILTIN(__builtin_ia32_gather3div2df, "V2dV2dvC*V2OiUcIi", "nV:128:", "avx512vl")
TARGET_BUILTIN(__builtin_ia32_gather3div2di, "V2OiV2OivC*V2OiUcIi", "nV:128:", "avx512vl") TARGET_BUILTIN(__builtin_ia32_gather3div2di, "V2OiV2OivC*V2OiUcIi", "nV:128:", "avx512vl")
TARGET_BUILTIN(__builtin_ia32_gather3div4df, "V4dV4dvC*V4OiUcIi", "nV:256:", "avx512vl") TARGET_BUILTIN(__builtin_ia32_gather3div4df, "V4dV4dvC*V4OiUcIi", "nV:256:", "avx512vl")
TARGET_BUILTIN(__builtin_ia32_gather3div4di, "V4OiV4OivC*V4OiUcIi", "nV:256:", "avx512vl") TARGET_BUILTIN(__builtin_ia32_gather3div4di, "V4OiV4OivC*V4OiUcIi", "nV:256:", "avx512vl")
TARGET_BUILTIN(__builtin_ia32_gather3div4sf, "V4fV4fvC*V2OiUcIi", "nV:128:", "avx512vl") TARGET_BUILTIN(__builtin_ia32_gather3div4sf, "V4fV4fvC*V2OiUcIi", "nV:128:", "avx512vl")
TARGET_BUILTIN(__builtin_ia32_gather3div4si, "V4iV4ivC*V2OiUcIi", "nV:128:", "avx512vl") TARGET_BUILTIN(__builtin_ia32_gather3div4si, "V4iV4ivC*V2OiUcIi", "nV:128:", "avx512vl")
▲ Show 20 Lines • Show All 980 Lines • Show Last 20 Lines

clang/include/clang/Driver/Options.td

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 3,218 Lines • ▼ Show 20 Lines
def mavx512vl : Flag<["-"], "mavx512vl">, Group<m_x86_Features_Group>; def mavx512vl : Flag<["-"], "mavx512vl">, Group<m_x86_Features_Group>;
def mno_avx512vl : Flag<["-"], "mno-avx512vl">, Group<m_x86_Features_Group>; def mno_avx512vl : Flag<["-"], "mno-avx512vl">, Group<m_x86_Features_Group>;
def mavx512vnni : Flag<["-"], "mavx512vnni">, Group<m_x86_Features_Group>; def mavx512vnni : Flag<["-"], "mavx512vnni">, Group<m_x86_Features_Group>;
def mno_avx512vnni : Flag<["-"], "mno-avx512vnni">, Group<m_x86_Features_Group>; def mno_avx512vnni : Flag<["-"], "mno-avx512vnni">, Group<m_x86_Features_Group>;
def mavx512vpopcntdq : Flag<["-"], "mavx512vpopcntdq">, Group<m_x86_Features_Group>; def mavx512vpopcntdq : Flag<["-"], "mavx512vpopcntdq">, Group<m_x86_Features_Group>;
def mno_avx512vpopcntdq : Flag<["-"], "mno-avx512vpopcntdq">, Group<m_x86_Features_Group>; def mno_avx512vpopcntdq : Flag<["-"], "mno-avx512vpopcntdq">, Group<m_x86_Features_Group>;
def mavx512vp2intersect : Flag<["-"], "mavx512vp2intersect">, Group<m_x86_Features_Group>; def mavx512vp2intersect : Flag<["-"], "mavx512vp2intersect">, Group<m_x86_Features_Group>;
def mno_avx512vp2intersect : Flag<["-"], "mno-avx512vp2intersect">, Group<m_x86_Features_Group>; def mno_avx512vp2intersect : Flag<["-"], "mno-avx512vp2intersect">, Group<m_x86_Features_Group>;
def mavxvnni : Flag<["-"], "mavxvnni">, Group<m_x86_Features_Group>;
def mno_avxvnni : Flag<["-"], "mno-avxvnni">, Group<m_x86_Features_Group>;
def madx : Flag<["-"], "madx">, Group<m_x86_Features_Group>; def madx : Flag<["-"], "madx">, Group<m_x86_Features_Group>;
def mno_adx : Flag<["-"], "mno-adx">, Group<m_x86_Features_Group>; def mno_adx : Flag<["-"], "mno-adx">, Group<m_x86_Features_Group>;
def maes : Flag<["-"], "maes">, Group<m_x86_Features_Group>; def maes : Flag<["-"], "maes">, Group<m_x86_Features_Group>;
def mno_aes : Flag<["-"], "mno-aes">, Group<m_x86_Features_Group>; def mno_aes : Flag<["-"], "mno-aes">, Group<m_x86_Features_Group>;
def mbmi : Flag<["-"], "mbmi">, Group<m_x86_Features_Group>; def mbmi : Flag<["-"], "mbmi">, Group<m_x86_Features_Group>;
def mno_bmi : Flag<["-"], "mno-bmi">, Group<m_x86_Features_Group>; def mno_bmi : Flag<["-"], "mno-bmi">, Group<m_x86_Features_Group>;
def mbmi2 : Flag<["-"], "mbmi2">, Group<m_x86_Features_Group>; def mbmi2 : Flag<["-"], "mbmi2">, Group<m_x86_Features_Group>;
def mno_bmi2 : Flag<["-"], "mno-bmi2">, Group<m_x86_Features_Group>; def mno_bmi2 : Flag<["-"], "mno-bmi2">, Group<m_x86_Features_Group>;
▲ Show 20 Lines • Show All 1,718 Lines • Show Last 20 Lines

clang/lib/Basic/Targets/X86.h

Show First 20 Lines • Show All 123 Lines • ▼ Show 20 Linesclass LLVM_LIBRARY_VISIBILITY X86TargetInfo : public TargetInfo {
bool HasWAITPKG = false; bool HasWAITPKG = false;
bool HasMOVDIRI = false; bool HasMOVDIRI = false;
bool HasMOVDIR64B = false; bool HasMOVDIR64B = false;
bool HasPTWRITE = false; bool HasPTWRITE = false;
bool HasINVPCID = false; bool HasINVPCID = false;
bool HasENQCMD = false; bool HasENQCMD = false;
bool HasKL = false; // For key locker bool HasKL = false; // For key locker
bool HasWIDEKL = false; // For wide key locker bool HasWIDEKL = false; // For wide key locker
bool HasAVXVNNI = false;
bool HasAMXTILE = false; bool HasAMXTILE = false;
bool HasAMXINT8 = false; bool HasAMXINT8 = false;
bool HasAMXBF16 = false; bool HasAMXBF16 = false;
bool HasSERIALIZE = false; bool HasSERIALIZE = false;
bool HasTSXLDTRK = false; bool HasTSXLDTRK = false;
protected: protected:
llvm::X86::CPUKind CPU = llvm::X86::CK_None; llvm::X86::CPUKind CPU = llvm::X86::CK_None;
▲ Show 20 Lines • Show All 774 Lines • Show Last 20 Lines

clang/lib/Basic/Targets/X86.cpp

Show First 20 Lines • Show All 298 Lines • ▼ Show 20 Lines for (const auto &Feature : Features) {
} else if (Feature == "+enqcmd") { } else if (Feature == "+enqcmd") {
HasENQCMD = true; HasENQCMD = true;
} else if (Feature == "+amx-bf16") { } else if (Feature == "+amx-bf16") {
HasAMXBF16 = true; HasAMXBF16 = true;
} else if (Feature == "+amx-int8") { } else if (Feature == "+amx-int8") {
HasAMXINT8 = true; HasAMXINT8 = true;
} else if (Feature == "+amx-tile") { } else if (Feature == "+amx-tile") {
HasAMXTILE = true; HasAMXTILE = true;
} else if (Feature == "+avxvnni") {
HasAVXVNNI = true;
} else if (Feature == "+serialize") { } else if (Feature == "+serialize") {
HasSERIALIZE = true; HasSERIALIZE = true;
} else if (Feature == "+tsxldtrk") { } else if (Feature == "+tsxldtrk") {
HasTSXLDTRK = true; HasTSXLDTRK = true;
} }
X86SSEEnum Level = llvm::StringSwitch<X86SSEEnum>(Feature) X86SSEEnum Level = llvm::StringSwitch<X86SSEEnum>(Feature)
.Case("+avx512f", AVX512F) .Case("+avx512f", AVX512F)
▲ Show 20 Lines • Show All 390 Lines • ▼ Show 20 Linesvoid X86TargetInfo::getTargetDefines(const LangOptions &Opts,
if (HasENQCMD) if (HasENQCMD)
Builder.defineMacro("__ENQCMD__"); Builder.defineMacro("__ENQCMD__");
if (HasAMXTILE) if (HasAMXTILE)
Builder.defineMacro("__AMXTILE__"); Builder.defineMacro("__AMXTILE__");
if (HasAMXINT8) if (HasAMXINT8)
Builder.defineMacro("__AMXINT8__"); Builder.defineMacro("__AMXINT8__");
if (HasAMXBF16) if (HasAMXBF16)
Builder.defineMacro("__AMXBF16__"); Builder.defineMacro("__AMXBF16__");
if (HasAVXVNNI)
Builder.defineMacro("__AVXVNNI__");
if (HasSERIALIZE) if (HasSERIALIZE)
Builder.defineMacro("__SERIALIZE__"); Builder.defineMacro("__SERIALIZE__");
if (HasTSXLDTRK) if (HasTSXLDTRK)
Builder.defineMacro("__TSXLDTRK__"); Builder.defineMacro("__TSXLDTRK__");
// Each case falls through to the previous one here. // Each case falls through to the previous one here.
switch (SSELevel) { switch (SSELevel) {
case AVX512F: case AVX512F:
▲ Show 20 Lines • Show All 100 Lines • ▼ Show 20 Lines return llvm::StringSwitch<bool>(Name)
.Case("avx512dq", true) .Case("avx512dq", true)
.Case("avx512bitalg", true) .Case("avx512bitalg", true)
.Case("avx512bw", true) .Case("avx512bw", true)
.Case("avx512vl", true) .Case("avx512vl", true)
.Case("avx512vbmi", true) .Case("avx512vbmi", true)
.Case("avx512vbmi2", true) .Case("avx512vbmi2", true)
.Case("avx512ifma", true) .Case("avx512ifma", true)
.Case("avx512vp2intersect", true) .Case("avx512vp2intersect", true)
.Case("avxvnni", true)
.Case("bmi", true) .Case("bmi", true)
.Case("bmi2", true) .Case("bmi2", true)
.Case("cldemote", true) .Case("cldemote", true)
.Case("clflushopt", true) .Case("clflushopt", true)
.Case("clwb", true) .Case("clwb", true)
.Case("clzero", true) .Case("clzero", true)
.Case("cx16", true) .Case("cx16", true)
.Case("enqcmd", true) .Case("enqcmd", true)
▲ Show 20 Lines • Show All 54 Lines • ▼ Show 20 Lines
bool X86TargetInfo::hasFeature(StringRef Feature) const { bool X86TargetInfo::hasFeature(StringRef Feature) const {
return llvm::StringSwitch<bool>(Feature) return llvm::StringSwitch<bool>(Feature)
.Case("adx", HasADX) .Case("adx", HasADX)
.Case("aes", HasAES) .Case("aes", HasAES)
.Case("amx-bf16", HasAMXBF16) .Case("amx-bf16", HasAMXBF16)
.Case("amx-int8", HasAMXINT8) .Case("amx-int8", HasAMXINT8)
.Case("amx-tile", HasAMXTILE) .Case("amx-tile", HasAMXTILE)
.Case("avxvnni", HasAVXVNNI)
.Case("avx", SSELevel >= AVX) .Case("avx", SSELevel >= AVX)
.Case("avx2", SSELevel >= AVX2) .Case("avx2", SSELevel >= AVX2)
.Case("avx512f", SSELevel >= AVX512F) .Case("avx512f", SSELevel >= AVX512F)
.Case("avx512cd", HasAVX512CD) .Case("avx512cd", HasAVX512CD)
.Case("avx512vpopcntdq", HasAVX512VPOPCNTDQ) .Case("avx512vpopcntdq", HasAVX512VPOPCNTDQ)
.Case("avx512vnni", HasAVX512VNNI) .Case("avx512vnni", HasAVX512VNNI)
.Case("avx512bf16", HasAVX512BF16) .Case("avx512bf16", HasAVX512BF16)
.Case("avx512er", HasAVX512ER) .Case("avx512er", HasAVX512ER)
▲ Show 20 Lines • Show All 567 Lines • Show Last 20 Lines

clang/lib/CodeGen/CodeGenFunction.h

Show First 20 Lines • Show All 4,674 Lines • ▼ Show 20 Linesprivate:
llvm::Value *EmitX86CpuIs(StringRef CPUStr); llvm::Value *EmitX86CpuIs(StringRef CPUStr);
llvm::Value *EmitX86CpuSupports(const CallExpr *E); llvm::Value *EmitX86CpuSupports(const CallExpr *E);
llvm::Value *EmitX86CpuSupports(ArrayRef<StringRef> FeatureStrs); llvm::Value *EmitX86CpuSupports(ArrayRef<StringRef> FeatureStrs);
llvm::Value *EmitX86CpuSupports(uint64_t Mask); llvm::Value *EmitX86CpuSupports(uint64_t Mask);
llvm::Value *EmitX86CpuInit(); llvm::Value *EmitX86CpuInit();
llvm::Value *FormResolverCondition(const MultiVersionResolverOption &RO); llvm::Value *FormResolverCondition(const MultiVersionResolverOption &RO);
}; };
class TargetFeatures {
const llvm::StringMap<bool> &CallerFeatureMap;
bool getAndFeatures(StringRef &FeatureList) {
int InParentheses = 0;
bool HasFeatures = true;
size_t Start = 0;
for (size_t i = 0; i < FeatureList.size(); i++) {
switch (FeatureList[i]) {
default:
break;
case '(':
if (InParentheses == 0)
Start = i + 1;
InParentheses++;
break;
case ')':
InParentheses--;
assert(InParentheses >= 0 && "Parentheses are not in pair");
LLVM_FALLTHROUGH;
case '|':
case ',':
if (InParentheses == 0) {
if (HasFeatures && i != Start) {
StringRef F = FeatureList.slice(Start, i);
HasFeatures = FeatureList[i] == ')' ? hasRequiredFeatures(F)
: CallerFeatureMap.lookup(F);
}
Start = i + 1;
if (FeatureList[i] == '|') {
FeatureList = FeatureList.substr(Start);
return HasFeatures;
}
}
break;
}
}
assert(InParentheses == 0 && "Parentheses are not in pair");
if (HasFeatures && Start != FeatureList.size())
HasFeatures = CallerFeatureMap.lookup(FeatureList.substr(Start));
FeatureList = StringRef();
return HasFeatures;
}
public:
bool hasRequiredFeatures(StringRef FeatureList) {
bool HasFeatures = false;
while (!HasFeatures && !FeatureList.empty())
HasFeatures = getAndFeatures(FeatureList);
return HasFeatures;
}
TargetFeatures(const llvm::StringMap<bool> &CallerFeatureMap)
: CallerFeatureMap(CallerFeatureMap) {}
};
inline DominatingLLVMValue::saved_type inline DominatingLLVMValue::saved_type
DominatingLLVMValue::save(CodeGenFunction &CGF, llvm::Value *value) { DominatingLLVMValue::save(CodeGenFunction &CGF, llvm::Value *value) {
if (!needsSaving(value)) return saved_type(value, false); if (!needsSaving(value)) return saved_type(value, false);
// Otherwise, we need an alloca. // Otherwise, we need an alloca.
auto align = CharUnits::fromQuantity( auto align = CharUnits::fromQuantity(
CGF.CGM.getDataLayout().getPrefTypeAlignment(value->getType())); CGF.CGM.getDataLayout().getPrefTypeAlignment(value->getType()));
Address alloca = Address alloca =
Show All 25 Lines

clang/lib/CodeGen/CodeGenFunction.cpp

Show First 20 Lines • Show All 2,318 Lines • ▼ Show 20 Lines
void CGBuilderInserter::InsertHelper( void CGBuilderInserter::InsertHelper(
llvm::Instruction *I, const llvm::Twine &Name, llvm::BasicBlock *BB, llvm::Instruction *I, const llvm::Twine &Name, llvm::BasicBlock *BB,
llvm::BasicBlock::iterator InsertPt) const { llvm::BasicBlock::iterator InsertPt) const {
llvm::IRBuilderDefaultInserter::InsertHelper(I, Name, BB, InsertPt); llvm::IRBuilderDefaultInserter::InsertHelper(I, Name, BB, InsertPt);
if (CGF) if (CGF)
CGF->InsertHelper(I, Name, BB, InsertPt); CGF->InsertHelper(I, Name, BB, InsertPt);
} }
static bool hasRequiredFeatures(const SmallVectorImpl<StringRef> &ReqFeatures,
CodeGenModule &CGM, const FunctionDecl *FD,
std::string &FirstMissing) {
// If there aren't any required features listed then go ahead and return.
if (ReqFeatures.empty())
return false;
// Now build up the set of caller features and verify that all the required
// features are there.
llvm::StringMap<bool> CallerFeatureMap;
CGM.getContext().getFunctionFeatureMap(CallerFeatureMap, FD);
// If we have at least one of the features in the feature list return
// true, otherwise return false.
return std::all_of(
ReqFeatures.begin(), ReqFeatures.end(), [&](StringRef Feature) {
SmallVector<StringRef, 1> OrFeatures;
Feature.split(OrFeatures, '|');
return llvm::any_of(OrFeatures, [&](StringRef Feature) {
if (!CallerFeatureMap.lookup(Feature)) {
FirstMissing = Feature.str();
return false;
}
return true;
});
});
}
// Emits an error if we don't have a valid set of target features for the // Emits an error if we don't have a valid set of target features for the
// called function. // called function.
void CodeGenFunction::checkTargetFeatures(const CallExpr *E, void CodeGenFunction::checkTargetFeatures(const CallExpr *E,
const FunctionDecl *TargetDecl) { const FunctionDecl *TargetDecl) {
return checkTargetFeatures(E->getBeginLoc(), TargetDecl); return checkTargetFeatures(E->getBeginLoc(), TargetDecl);
} }
// Emits an error if we don't have a valid set of target features for the // Emits an error if we don't have a valid set of target features for the
Show All 10 Linesvoid CodeGenFunction::checkTargetFeatures(SourceLocation Loc,
if (!FD) if (!FD)
return; return;
// Grab the required features for the call. For a builtin this is listed in // Grab the required features for the call. For a builtin this is listed in
// the td file with the default cpu, for an always_inline function this is any // the td file with the default cpu, for an always_inline function this is any
// listed cpu and any listed features. // listed cpu and any listed features.
unsigned BuiltinID = TargetDecl->getBuiltinID(); unsigned BuiltinID = TargetDecl->getBuiltinID();
std::string MissingFeature; std::string MissingFeature;
llvm::StringMap<bool> CallerFeatureMap;
CGM.getContext().getFunctionFeatureMap(CallerFeatureMap, FD);
if (BuiltinID) { if (BuiltinID) {
SmallVector<StringRef, 1> ReqFeatures; StringRef FeatureList(
const char *FeatureList = CGM.getContext().BuiltinInfo.getRequiredFeatures(BuiltinID));
CGM.getContext().BuiltinInfo.getRequiredFeatures(BuiltinID);
// Return if the builtin doesn't have any required features. // Return if the builtin doesn't have any required features.
if (!FeatureList || StringRef(FeatureList) == "") if (FeatureList.empty())
return; return;
StringRef(FeatureList).split(ReqFeatures, ','); assert(FeatureList.find(' ') == StringRef::npos &&
if (!hasRequiredFeatures(ReqFeatures, CGM, FD, MissingFeature)) "Space in feature list");
TargetFeatures TF(CallerFeatureMap);
if (!TF.hasRequiredFeatures(FeatureList))
CGM.getDiags().Report(Loc, diag::err_builtin_needs_feature) CGM.getDiags().Report(Loc, diag::err_builtin_needs_feature)
<< TargetDecl->getDeclName() << TargetDecl->getDeclName() << FeatureList;
<< CGM.getContext().BuiltinInfo.getRequiredFeatures(BuiltinID);
} else if (!TargetDecl->isMultiVersion() && } else if (!TargetDecl->isMultiVersion() &&
TargetDecl->hasAttr<TargetAttr>()) { TargetDecl->hasAttr<TargetAttr>()) {
// Get the required features for the callee. // Get the required features for the callee.
const TargetAttr *TD = TargetDecl->getAttr<TargetAttr>(); const TargetAttr *TD = TargetDecl->getAttr<TargetAttr>();
ParsedTargetAttr ParsedAttr = ParsedTargetAttr ParsedAttr =
CGM.getContext().filterFunctionTargetAttrs(TD); CGM.getContext().filterFunctionTargetAttrs(TD);
SmallVector<StringRef, 1> ReqFeatures; SmallVector<StringRef, 1> ReqFeatures;
llvm::StringMap<bool> CalleeFeatureMap; llvm::StringMap<bool> CalleeFeatureMap;
CGM.getContext().getFunctionFeatureMap(CalleeFeatureMap, TargetDecl); CGM.getContext().getFunctionFeatureMap(CalleeFeatureMap, TargetDecl);
for (const auto &F : ParsedAttr.Features) { for (const auto &F : ParsedAttr.Features) {
if (F[0] == '+' && CalleeFeatureMap.lookup(F.substr(1))) if (F[0] == '+' && CalleeFeatureMap.lookup(F.substr(1)))
ReqFeatures.push_back(StringRef(F).substr(1)); ReqFeatures.push_back(StringRef(F).substr(1));
} }
for (const auto &F : CalleeFeatureMap) { for (const auto &F : CalleeFeatureMap) {
// Only positive features are "required". // Only positive features are "required".
if (F.getValue()) if (F.getValue())
ReqFeatures.push_back(F.getKey()); ReqFeatures.push_back(F.getKey());
} }
if (!hasRequiredFeatures(ReqFeatures, CGM, FD, MissingFeature)) if (!llvm::all_of(ReqFeatures, [&](StringRef Feature) {
if (!CallerFeatureMap.lookup(Feature)) {
MissingFeature = Feature.str();
return false;
}
return true;
}))
CGM.getDiags().Report(Loc, diag::err_function_needs_feature) CGM.getDiags().Report(Loc, diag::err_function_needs_feature)
<< FD->getDeclName() << TargetDecl->getDeclName() << MissingFeature; << FD->getDeclName() << TargetDecl->getDeclName() << MissingFeature;
} }
} }
void CodeGenFunction::EmitSanitizerStatReport(llvm::SanitizerStatKind SSK) { void CodeGenFunction::EmitSanitizerStatReport(llvm::SanitizerStatKind SSK) {
if (!CGM.getCodeGenOpts().SanitizeStats) if (!CGM.getCodeGenOpts().SanitizeStats)
return; return;
▲ Show 20 Lines • Show All 147 Lines • Show Last 20 Lines

clang/lib/Headers/CMakeLists.txt

Show All 29 Linesset(files
avx512vldqintrin.h avx512vldqintrin.h
avx512vlintrin.h avx512vlintrin.h
avx512vp2intersectintrin.h avx512vp2intersectintrin.h
avx512vlvp2intersectintrin.h avx512vlvp2intersectintrin.h
avx512vpopcntdqvlintrin.h avx512vpopcntdqvlintrin.h
avx512vnniintrin.h avx512vnniintrin.h
avx512vlvnniintrin.h avx512vlvnniintrin.h
avxintrin.h avxintrin.h
avxvnniintrin.h
commonvnniintrin.h
bmi2intrin.h bmi2intrin.h
bmiintrin.h bmiintrin.h
__clang_cuda_builtin_vars.h __clang_cuda_builtin_vars.h
__clang_cuda_math.h __clang_cuda_math.h
__clang_cuda_cmath.h __clang_cuda_cmath.h
__clang_cuda_complex_builtins.h __clang_cuda_complex_builtins.h
__clang_cuda_device_functions.h __clang_cuda_device_functions.h
__clang_cuda_intrinsics.h __clang_cuda_intrinsics.h
▲ Show 20 Lines • Show All 195 Lines • Show Last 20 Lines

clang/lib/Headers/avx512vlvnniintrin.h

/*===------------- avx512vlvnniintrin.h - VNNI intrinsics ------------------=== /*===------------- avx512vlvnniintrin.h - VNNI intrinsics ------------------===
* *
* *
* Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. * Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
* See https://llvm.org/LICENSE.txt for license information. * See https://llvm.org/LICENSE.txt for license information.
* SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception * SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
* *
*===-----------------------------------------------------------------------=== *===-----------------------------------------------------------------------===
*/ */
#ifndef __IMMINTRIN_H #ifndef __IMMINTRIN_H
#error "Never use <avx512vlvnniintrin.h> directly; include <immintrin.h> instead." #error "Never use <avx512vlvnniintrin.h> directly; include <immintrin.h> instead."
#endif #endif
#ifndef __AVX512VLVNNIINTRIN_H #ifndef __AVX512VLVNNIINTRIN_H
#define __AVX512VLVNNIINTRIN_H #define __AVX512VLVNNIINTRIN_H
#include "commonvnniintrin.h"
/* Define the default attributes for the functions in this file. */ /* Define the default attributes for the functions in this file. */
#define __DEFAULT_FN_ATTRS128 __attribute__((__always_inline__, __nodebug__, __target__("avx512vl,avx512vnni"), __min_vector_width__(128))) #define __DEFAULT_FN_ATTRS128 __attribute__((__always_inline__, __nodebug__, __target__("avx512vl,avx512vnni"), __min_vector_width__(128)))
#define __DEFAULT_FN_ATTRS256 __attribute__((__always_inline__, __nodebug__, __target__("avx512vl,avx512vnni"), __min_vector_width__(256))) #define __DEFAULT_FN_ATTRS256 __attribute__((__always_inline__, __nodebug__, __target__("avx512vl,avx512vnni"), __min_vector_width__(256)))
static __inline__ __m256i __DEFAULT_FN_ATTRS256 static __inline__ __m256i __DEFAULT_FN_ATTRS256
_mm256_dpbusd_epi32(__m256i __S, __m256i __A, __m256i __B)
{
return (__m256i)__builtin_ia32_vpdpbusd256((__v8si)__S, (__v8si)__A,
(__v8si)__B);
}
static __inline__ __m256i __DEFAULT_FN_ATTRS256
_mm256_mask_dpbusd_epi32(__m256i __S, __mmask8 __U, __m256i __A, __m256i __B) _mm256_mask_dpbusd_epi32(__m256i __S, __mmask8 __U, __m256i __A, __m256i __B)
{ {
return (__m256i)__builtin_ia32_selectd_256(__U, return (__m256i)__builtin_ia32_selectd_256(__U,
(__v8si)_mm256_dpbusd_epi32(__S, __A, __B), (__v8si)_mm256_dpbusd_epi32(__S, __A, __B),
(__v8si)__S); (__v8si)__S);
} }
static __inline__ __m256i __DEFAULT_FN_ATTRS256 static __inline__ __m256i __DEFAULT_FN_ATTRS256
_mm256_maskz_dpbusd_epi32(__mmask8 __U, __m256i __S, __m256i __A, __m256i __B) _mm256_maskz_dpbusd_epi32(__mmask8 __U, __m256i __S, __m256i __A, __m256i __B)
{ {
return (__m256i)__builtin_ia32_selectd_256(__U, return (__m256i)__builtin_ia32_selectd_256(__U,
(__v8si)_mm256_dpbusd_epi32(__S, __A, __B), (__v8si)_mm256_dpbusd_epi32(__S, __A, __B),
(__v8si)_mm256_setzero_si256()); (__v8si)_mm256_setzero_si256());
} }
static __inline__ __m256i __DEFAULT_FN_ATTRS256 static __inline__ __m256i __DEFAULT_FN_ATTRS256
_mm256_dpbusds_epi32(__m256i __S, __m256i __A, __m256i __B)
{
return (__m256i)__builtin_ia32_vpdpbusds256((__v8si)__S, (__v8si)__A,
(__v8si)__B);
}
static __inline__ __m256i __DEFAULT_FN_ATTRS256
_mm256_mask_dpbusds_epi32(__m256i __S, __mmask8 __U, __m256i __A, __m256i __B) _mm256_mask_dpbusds_epi32(__m256i __S, __mmask8 __U, __m256i __A, __m256i __B)
{ {
return (__m256i)__builtin_ia32_selectd_256(__U, return (__m256i)__builtin_ia32_selectd_256(__U,
(__v8si)_mm256_dpbusds_epi32(__S, __A, __B), (__v8si)_mm256_dpbusds_epi32(__S, __A, __B),
(__v8si)__S); (__v8si)__S);
} }
static __inline__ __m256i __DEFAULT_FN_ATTRS256 static __inline__ __m256i __DEFAULT_FN_ATTRS256
_mm256_maskz_dpbusds_epi32(__mmask8 __U, __m256i __S, __m256i __A, __m256i __B) _mm256_maskz_dpbusds_epi32(__mmask8 __U, __m256i __S, __m256i __A, __m256i __B)
{ {
return (__m256i)__builtin_ia32_selectd_256(__U, return (__m256i)__builtin_ia32_selectd_256(__U,
(__v8si)_mm256_dpbusds_epi32(__S, __A, __B), (__v8si)_mm256_dpbusds_epi32(__S, __A, __B),
(__v8si)_mm256_setzero_si256()); (__v8si)_mm256_setzero_si256());
} }
static __inline__ __m256i __DEFAULT_FN_ATTRS256 static __inline__ __m256i __DEFAULT_FN_ATTRS256
_mm256_dpwssd_epi32(__m256i __S, __m256i __A, __m256i __B)
{
return (__m256i)__builtin_ia32_vpdpwssd256((__v8si)__S, (__v8si)__A,
(__v8si)__B);
}
static __inline__ __m256i __DEFAULT_FN_ATTRS256
_mm256_mask_dpwssd_epi32(__m256i __S, __mmask8 __U, __m256i __A, __m256i __B) _mm256_mask_dpwssd_epi32(__m256i __S, __mmask8 __U, __m256i __A, __m256i __B)
{ {
return (__m256i)__builtin_ia32_selectd_256(__U, return (__m256i)__builtin_ia32_selectd_256(__U,
(__v8si)_mm256_dpwssd_epi32(__S, __A, __B), (__v8si)_mm256_dpwssd_epi32(__S, __A, __B),
(__v8si)__S); (__v8si)__S);
} }
static __inline__ __m256i __DEFAULT_FN_ATTRS256 static __inline__ __m256i __DEFAULT_FN_ATTRS256
_mm256_maskz_dpwssd_epi32(__mmask8 __U, __m256i __S, __m256i __A, __m256i __B) _mm256_maskz_dpwssd_epi32(__mmask8 __U, __m256i __S, __m256i __A, __m256i __B)
{ {
return (__m256i)__builtin_ia32_selectd_256(__U, return (__m256i)__builtin_ia32_selectd_256(__U,
(__v8si)_mm256_dpwssd_epi32(__S, __A, __B), (__v8si)_mm256_dpwssd_epi32(__S, __A, __B),
(__v8si)_mm256_setzero_si256()); (__v8si)_mm256_setzero_si256());
} }
static __inline__ __m256i __DEFAULT_FN_ATTRS256 static __inline__ __m256i __DEFAULT_FN_ATTRS256
_mm256_dpwssds_epi32(__m256i __S, __m256i __A, __m256i __B)
{
return (__m256i)__builtin_ia32_vpdpwssds256((__v8si)__S, (__v8si)__A,
(__v8si)__B);
}
static __inline__ __m256i __DEFAULT_FN_ATTRS256
_mm256_mask_dpwssds_epi32(__m256i __S, __mmask8 __U, __m256i __A, __m256i __B) _mm256_mask_dpwssds_epi32(__m256i __S, __mmask8 __U, __m256i __A, __m256i __B)
{ {
return (__m256i)__builtin_ia32_selectd_256(__U, return (__m256i)__builtin_ia32_selectd_256(__U,
(__v8si)_mm256_dpwssds_epi32(__S, __A, __B), (__v8si)_mm256_dpwssds_epi32(__S, __A, __B),
(__v8si)__S); (__v8si)__S);
} }
static __inline__ __m256i __DEFAULT_FN_ATTRS256 static __inline__ __m256i __DEFAULT_FN_ATTRS256
_mm256_maskz_dpwssds_epi32(__mmask8 __U, __m256i __S, __m256i __A, __m256i __B) _mm256_maskz_dpwssds_epi32(__mmask8 __U, __m256i __S, __m256i __A, __m256i __B)
{ {
return (__m256i)__builtin_ia32_selectd_256(__U, return (__m256i)__builtin_ia32_selectd_256(__U,
(__v8si)_mm256_dpwssds_epi32(__S, __A, __B), (__v8si)_mm256_dpwssds_epi32(__S, __A, __B),
(__v8si)_mm256_setzero_si256()); (__v8si)_mm256_setzero_si256());
} }
static __inline__ __m128i __DEFAULT_FN_ATTRS128 static __inline__ __m128i __DEFAULT_FN_ATTRS128
_mm_dpbusd_epi32(__m128i __S, __m128i __A, __m128i __B)
{
return (__m128i)__builtin_ia32_vpdpbusd128((__v4si)__S, (__v4si)__A,
(__v4si)__B);
}
static __inline__ __m128i __DEFAULT_FN_ATTRS128
_mm_mask_dpbusd_epi32(__m128i __S, __mmask8 __U, __m128i __A, __m128i __B) _mm_mask_dpbusd_epi32(__m128i __S, __mmask8 __U, __m128i __A, __m128i __B)
{ {
return (__m128i)__builtin_ia32_selectd_128(__U, return (__m128i)__builtin_ia32_selectd_128(__U,
(__v4si)_mm_dpbusd_epi32(__S, __A, __B), (__v4si)_mm_dpbusd_epi32(__S, __A, __B),
(__v4si)__S); (__v4si)__S);
} }
static __inline__ __m128i __DEFAULT_FN_ATTRS128 static __inline__ __m128i __DEFAULT_FN_ATTRS128
_mm_maskz_dpbusd_epi32(__mmask8 __U, __m128i __S, __m128i __A, __m128i __B) _mm_maskz_dpbusd_epi32(__mmask8 __U, __m128i __S, __m128i __A, __m128i __B)
{ {
return (__m128i)__builtin_ia32_selectd_128(__U, return (__m128i)__builtin_ia32_selectd_128(__U,
(__v4si)_mm_dpbusd_epi32(__S, __A, __B), (__v4si)_mm_dpbusd_epi32(__S, __A, __B),
(__v4si)_mm_setzero_si128()); (__v4si)_mm_setzero_si128());
} }
static __inline__ __m128i __DEFAULT_FN_ATTRS128 static __inline__ __m128i __DEFAULT_FN_ATTRS128
_mm_dpbusds_epi32(__m128i __S, __m128i __A, __m128i __B)
{
return (__m128i)__builtin_ia32_vpdpbusds128((__v4si)__S, (__v4si)__A,
(__v4si)__B);
}
static __inline__ __m128i __DEFAULT_FN_ATTRS128
_mm_mask_dpbusds_epi32(__m128i __S, __mmask8 __U, __m128i __A, __m128i __B) _mm_mask_dpbusds_epi32(__m128i __S, __mmask8 __U, __m128i __A, __m128i __B)
{ {
return (__m128i)__builtin_ia32_selectd_128(__U, return (__m128i)__builtin_ia32_selectd_128(__U,
(__v4si)_mm_dpbusds_epi32(__S, __A, __B), (__v4si)_mm_dpbusds_epi32(__S, __A, __B),
(__v4si)__S); (__v4si)__S);
} }
static __inline__ __m128i __DEFAULT_FN_ATTRS128 static __inline__ __m128i __DEFAULT_FN_ATTRS128
_mm_maskz_dpbusds_epi32(__mmask8 __U, __m128i __S, __m128i __A, __m128i __B) _mm_maskz_dpbusds_epi32(__mmask8 __U, __m128i __S, __m128i __A, __m128i __B)
{ {
return (__m128i)__builtin_ia32_selectd_128(__U, return (__m128i)__builtin_ia32_selectd_128(__U,
(__v4si)_mm_dpbusds_epi32(__S, __A, __B), (__v4si)_mm_dpbusds_epi32(__S, __A, __B),
(__v4si)_mm_setzero_si128()); (__v4si)_mm_setzero_si128());
} }
static __inline__ __m128i __DEFAULT_FN_ATTRS128 static __inline__ __m128i __DEFAULT_FN_ATTRS128
_mm_dpwssd_epi32(__m128i __S, __m128i __A, __m128i __B)
{
return (__m128i)__builtin_ia32_vpdpwssd128((__v4si)__S, (__v4si)__A,
(__v4si)__B);
}
static __inline__ __m128i __DEFAULT_FN_ATTRS128
_mm_mask_dpwssd_epi32(__m128i __S, __mmask8 __U, __m128i __A, __m128i __B) _mm_mask_dpwssd_epi32(__m128i __S, __mmask8 __U, __m128i __A, __m128i __B)
{ {
return (__m128i)__builtin_ia32_selectd_128(__U, return (__m128i)__builtin_ia32_selectd_128(__U,
(__v4si)_mm_dpwssd_epi32(__S, __A, __B), (__v4si)_mm_dpwssd_epi32(__S, __A, __B),
(__v4si)__S); (__v4si)__S);
} }
static __inline__ __m128i __DEFAULT_FN_ATTRS128 static __inline__ __m128i __DEFAULT_FN_ATTRS128
_mm_maskz_dpwssd_epi32(__mmask8 __U, __m128i __S, __m128i __A, __m128i __B) _mm_maskz_dpwssd_epi32(__mmask8 __U, __m128i __S, __m128i __A, __m128i __B)
{ {
return (__m128i)__builtin_ia32_selectd_128(__U, return (__m128i)__builtin_ia32_selectd_128(__U,
(__v4si)_mm_dpwssd_epi32(__S, __A, __B), (__v4si)_mm_dpwssd_epi32(__S, __A, __B),
(__v4si)_mm_setzero_si128()); (__v4si)_mm_setzero_si128());
} }
static __inline__ __m128i __DEFAULT_FN_ATTRS128 static __inline__ __m128i __DEFAULT_FN_ATTRS128
_mm_dpwssds_epi32(__m128i __S, __m128i __A, __m128i __B)
{
return (__m128i)__builtin_ia32_vpdpwssds128((__v4si)__S, (__v4si)__A,
(__v4si)__B);
}
static __inline__ __m128i __DEFAULT_FN_ATTRS128
_mm_mask_dpwssds_epi32(__m128i __S, __mmask8 __U, __m128i __A, __m128i __B) _mm_mask_dpwssds_epi32(__m128i __S, __mmask8 __U, __m128i __A, __m128i __B)
{ {
return (__m128i)__builtin_ia32_selectd_128(__U, return (__m128i)__builtin_ia32_selectd_128(__U,
(__v4si)_mm_dpwssds_epi32(__S, __A, __B), (__v4si)_mm_dpwssds_epi32(__S, __A, __B),
(__v4si)__S); (__v4si)__S);
} }
static __inline__ __m128i __DEFAULT_FN_ATTRS128 static __inline__ __m128i __DEFAULT_FN_ATTRS128
Show All 11 Lines

clang/lib/Headers/avxvnniintrin.h

  • This file was added.
/*===--------------- avxvnniintrin.h - VNNI intrinsics --------------------===
*
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*
*===-----------------------------------------------------------------------===
*/
#ifndef __IMMINTRIN_H
#error "Never use <avxvnniintrin.h> directly; include <immintrin.h> instead."
#endif
#ifndef __AVXVNNIINTRIN_H
#define __AVXVNNIINTRIN_H
#include "commonvnniintrin.h"
RKSimonUnsubmitted
Not Done
ReplyInline Actions

Is having a commonvnniintrin.h header the approach gcc/icc are taking?

RKSimon: Is having a commonvnniintrin.h header the approach gcc/icc are taking?
LiuChen3AuthorUnsubmitted
Done
ReplyInline Actions

GCC doesn't have commonvnniintrin.h headr, they only have avxvnniintrin.h. For ICC, I will check with this and give you answer later.

LiuChen3: GCC doesn't have commonvnniintrin.h headr, they only have avxvnniintrin.h. For ICC, I will…
LiuChen3AuthorUnsubmitted
Done
ReplyInline Actions

GCC and ICC doesn't break intrinsics into multiple headers.

LiuChen3: GCC and ICC doesn't break intrinsics into multiple headers.
RKSimonUnsubmitted
Not Done
ReplyInline Actions

Is there anyway that we can avoid this as well then? I get worried when header layouts start to diverge...

RKSimon: Is there anyway that we can avoid this as well then? I get worried when header layouts start to…
pengfeiUnsubmitted
Not Done
ReplyInline Actions

Hi Simon, why we need to keep the same layout with GCC and ICC?
We always create new header files for new instructions. We don't allow user include these headers directly. So I think it doesn't matter if GCC and ICC create the same headers or not.

pengfei: Hi Simon, why we need to keep the same layout with GCC and ICC? We always create new header…
craig.topperUnsubmitted
Not Done
ReplyInline Actions

As far as I know our header layout is largely the same as gcc for existing intrinsics. Can we just keep them the same?

craig.topper: As far as I know our header layout is largely the same as gcc for existing intrinsics. Can we…
pengfeiUnsubmitted
Not Done
ReplyInline Actions

I think we may still need add some information for intrinsics that can be used for AVXVNNI but not declared here. A proposal might be:

/* Below intrinsics defined in avx512vlvnniintrin.h can be used for */
/* AVXVNNI except used for compatibility with msvc.                 */
// #define _mm256_dpbusd_epi32(S, A, B)
// #define _mm256_dpbusds_epi32(S, A, B)
// #define _mm256_dpwssd_epi32(S, A, B)
...
pengfei: I think we may still need add some information for intrinsics that can be used for `AVXVNNI`…
pengfeiUnsubmitted
Not Done
ReplyInline Actions

I had a look at the Doxygen grammar. Maybe we can use like below:

/* Below intrinsics defined in avx512vlvnniintrin.h can be used for */
/* AVXVNNI except used for compatibility with msvc.                 */
/// \fn __m256i _mm256_dpbusd_epi32(__m256i __S, __m256i __A, __m256i __B)
/// \fn __m256i _mm256_dpbusds_epi32(__m256i __S, __m256i __A, __m256i __B)
/// \fn __m256i _mm256_dpwssd_epi32(__m256i __S, __m256i __A, __m256i __B)
...
pengfei: I had a look at the Doxygen grammar. Maybe we can use like below: ``` /* Below intrinsics…
pengfeiUnsubmitted
Not Done
ReplyInline Actions

Missing one line comment?

pengfei: Missing one line comment?
/* Intrinsics with _avx_ prefix are for compatibility with msvc. */
pengfeiUnsubmitted
Not Done
ReplyInline Actions

Maybe we can move commonvnniintrin.h to immintrin.h. Then we don't have a different layout.

#if !(defined(_MSC_VER) || defined(__SCE__)) || __has_feature(modules) ||      \
     defined(__AVX512VNNI__) || (defined(__AVX512VL__) && defined(__AVX512VNNI__))
#include <commonvnniintrin.h>
#endif
pengfei: Maybe we can move `commonvnniintrin.h` to `immintrin.h`. Then we don't have a different layout.
craig.topperUnsubmitted
Not Done
ReplyInline Actions

gcc appears to have just put the common intrinsics in avxvnniintrin.h. Why can't we do that?

craig.topper: gcc appears to have just put the common intrinsics in avxvnniintrin.h. Why can't we do that?
pengfeiUnsubmitted
Not Done
ReplyInline Actions

I asked GCC guys. They don't use macro like defined(__AVXVNNI__). If we put it in avxvnniintrin.h, we need handle macro AVX512VNNI in avxvnniintrin.h. It's bit confused.

@LiuChen3 I meant __AVXVNNI__ for the first __AVX512VNNI__ in last comment.

pengfei: I asked GCC guys. They don't use macro like `defined(__AVXVNNI__)`. If we put it in…
pengfeiUnsubmitted
Not Done
ReplyInline Actions

Or maybe we can put the common intrinsics in avx512vlvnniintrin.h and not to handle macro __AVXVNNI__ int it.
I think it's reasonable. For Linux, we always include all headers. For Windows, we can only allow user use the _avx_ intrinscis for AVXVNNI, which is compatible with MSVC.

pengfei: Or maybe we can put the common intrinsics in avx512vlvnniintrin.h and not to handle macro…
craig.topperUnsubmitted
Not Done
ReplyInline Actions

Minor clarification. It's not Linux vs Windows. Its -ms-compatibility that controls it. You can use clang on Windows without enabling MSVC compatibility. In which case MSC_VER won't be defined.

craig.topper: Minor clarification. It's not Linux vs Windows. Its -ms-compatibility that controls it. You can…
pengfeiUnsubmitted
Not Done
ReplyInline Actions

Right. Thank @craig.topper

pengfei: Right. Thank @craig.topper
/* Define the default attributes for the functions in this file. */
#define __DEFAULT_FN_ATTRS256 __attribute__((__always_inline__, __nodebug__, __target__("avxvnni"), __min_vector_width__(256)))
#define __DEFAULT_FN_ATTRS128 __attribute__((__always_inline__, __nodebug__, __target__("avxvnni"), __min_vector_width__(128)))
static __inline__ __m256i __DEFAULT_FN_ATTRS256
_mm256_dpbusd_avx_epi32(__m256i __S, __m256i __A, __m256i __B)
RKSimonUnsubmitted
Done
ReplyInline Actions

Please can you add doxygen descriptions to each intrinsic.

RKSimon: Please can you add doxygen descriptions to each intrinsic.
{
return (__m256i)__builtin_ia32_vpdpbusd256((__v8si)__S, (__v8si)__A, (__v8si)__B);
}
static __inline__ __m256i __DEFAULT_FN_ATTRS256
_mm256_dpbusds_avx_epi32(__m256i __S, __m256i __A, __m256i __B)
{
return (__m256i)__builtin_ia32_vpdpbusds256((__v8si)__S, (__v8si)__A, (__v8si)__B);
}
static __inline__ __m256i __DEFAULT_FN_ATTRS256
_mm256_dpwssd_avx_epi32(__m256i __S, __m256i __A, __m256i __B)
{
return (__m256i)__builtin_ia32_vpdpwssd256((__v8si)__S, (__v8si)__A, (__v8si)__B);
}
static __inline__ __m256i __DEFAULT_FN_ATTRS256
_mm256_dpwssds_avx_epi32(__m256i __S, __m256i __A, __m256i __B)
{
return (__m256i)__builtin_ia32_vpdpwssds256((__v8si)__S, (__v8si)__A, (__v8si)__B);
}
static __inline__ __m128i __DEFAULT_FN_ATTRS128
_mm_dpbusd_avx_epi32(__m128i __S, __m128i __A, __m128i __B)
{
return (__m128i)__builtin_ia32_vpdpbusd128((__v4si)__S, (__v4si)__A, (__v4si)__B);
}
static __inline__ __m128i __DEFAULT_FN_ATTRS128
_mm_dpbusds_avx_epi32(__m128i __S, __m128i __A, __m128i __B)
{
return (__m128i)__builtin_ia32_vpdpbusds128((__v4si)__S, (__v4si)__A, (__v4si)__B);
}
static __inline__ __m128i __DEFAULT_FN_ATTRS128
_mm_dpwssd_avx_epi32(__m128i __S, __m128i __A, __m128i __B)
{
return (__m128i)__builtin_ia32_vpdpwssd128((__v4si)__S, (__v4si)__A, (__v4si)__B);
}
static __inline__ __m128i __DEFAULT_FN_ATTRS128
_mm_dpwssds_avx_epi32(__m128i __S, __m128i __A, __m128i __B)
{
return (__m128i)__builtin_ia32_vpdpwssds128((__v4si)__S, (__v4si)__A, (__v4si)__B);
}
#undef __DEFAULT_FN_ATTRS128
#undef __DEFAULT_FN_ATTRS256
#endif // __AVXVNNIINTRIN_H

clang/lib/Headers/commonvnniintrin.h

  • This file was added.
/*===--------------- commonvnniintrin.h - VNNI intrinsics ------------------===
*
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
* THE SOFTWARE.
*
*===-----------------------------------------------------------------------===
*/
#ifndef __IMMINTRIN_H
#error "Never use <commonvnniintrin.h> directly; include <immintrin.h> instead."
#endif
#ifndef __COMMONVNNIINTRIN_H
#define __COMMONVNNIINTRIN_H
/// Multiply groups of 4 adjacent pairs of unsigned 8-bit integers in \a __A with
/// corresponding signed 8-bit integers in \a __B, producing 4 intermediate signed
/// 16-bit results. Sum these 4 results with the corresponding 32-bit integer
/// in \a __S, and store the packed 32-bit results in DST.
///
/// This intrinsic corresponds to the <c> VPDPBUSD </c> instructions.
///
/// \operation
/// FOR j := 0 to 7
/// tmp1.word := Signed(ZeroExtend16(__A.byte[4*j]) * SignExtend16(__B.byte[4*j]))
/// tmp2.word := Signed(ZeroExtend16(__A.byte[4*j+1]) * SignExtend16(__B.byte[4*j+1]))
/// tmp3.word := Signed(ZeroExtend16(__A.byte[4*j+2]) * SignExtend16(__B.byte[4*j+2]))
/// tmp4.word := Signed(ZeroExtend16(__A.byte[4*j+3]) * SignExtend16(__B.byte[4*j+3]))
/// DST.dword[j] := __S.dword[j] + tmp1 + tmp2 + tmp3 + tmp4
/// ENDFOR
/// DST[MAX:256] := 0
/// \endoperation
#define _mm256_dpbusd_epi32(__S, __A, __B) \
(__m256i)__builtin_ia32_vpdpbusd256((__v8si)__S, (__v8si)__A, (__v8si)__B)
craig.topperUnsubmitted
Not Done
ReplyInline Actions

Arguments to macros shouldn't start with __

craig.topper: Arguments to macros shouldn't start with __
craig.topperUnsubmitted
Not Done
ReplyInline Actions

Because this is a macro this needs to have parentheses around S, A, and B.

craig.topper: Because this is a macro this needs to have parentheses around S, A, and B.
/// Multiply groups of 4 adjacent pairs of unsigned 8-bit integers in \a __A with
/// corresponding signed 8-bit integers in \a __B, producing 4 intermediate signed
/// 16-bit results. Sum these 4 results with the corresponding 32-bit integer
/// in \a __S using signed saturation, and store the packed 32-bit results in DST.
///
/// This intrinsic corresponds to the <c> VPDPBUSDS </c> instructions.
///
/// \operation
/// FOR j := 0 to 7
/// tmp1.word := Signed(ZeroExtend16(__A.byte[4*j]) * SignExtend16(__B.byte[4*j]))
/// tmp2.word := Signed(ZeroExtend16(__A.byte[4*j+1]) * SignExtend16(__B.byte[4*j+1]))
/// tmp3.word := Signed(ZeroExtend16(__A.byte[4*j+2]) * SignExtend16(__B.byte[4*j+2]))
/// tmp4.word := Signed(ZeroExtend16(__A.byte[4*j+3]) * SignExtend16(__B.byte[4*j+3]))
/// DST.dword[j] := Saturate32(__S.dword[j] + tmp1 + tmp2 + tmp3 + tmp4)
/// ENDFOR
/// DST[MAX:256] := 0
/// \endoperation
#define _mm256_dpbusds_epi32(__S, __A, __B) \
(__m256i)__builtin_ia32_vpdpbusds256((__v8si)__S, (__v8si)__A, (__v8si)__B)
/// Multiply groups of 2 adjacent pairs of signed 16-bit integers in \a __A with
/// corresponding 16-bit integers in \a __B, producing 2 intermediate signed 32-bit
/// results. Sum these 2 results with the corresponding 32-bit integer in \a __S,
/// and store the packed 32-bit results in DST.
///
/// This intrinsic corresponds to the <c> VPDPWSSD </c> instructions.
///
/// \operation
/// FOR j := 0 to 7
/// tmp1.dword := SignExtend32(__A.word[2*j]) * SignExtend32(__B.word[2*j])
/// tmp2.dword := SignExtend32(__A.word[2*j+1]) * SignExtend32(__B.word[2*j+1])
/// DST.dword[j] := __S.dword[j] + tmp1 + tmp2
/// ENDFOR
/// DST[MAX:256] := 0
/// \endoperation
#define _mm256_dpwssd_epi32(__S, __A, __B) \
(__m256i)__builtin_ia32_vpdpwssd256((__v8si)__S, (__v8si)__A, (__v8si)__B)
/// Multiply groups of 2 adjacent pairs of signed 16-bit integers in \a __A with
/// corresponding 16-bit integers in \a __B, producing 2 intermediate signed 32-bit
/// results. Sum these 2 results with the corresponding 32-bit integer in \a __S
/// using signed saturation, and store the packed 32-bit results in DST.
///
/// This intrinsic corresponds to the <c> VPDPWSSDS </c> instructions.
///
/// \operation
/// FOR j := 0 to 7
/// tmp1.dword := SignExtend32(__A.word[2*j]) * SignExtend32(__B.word[2*j])
/// tmp2.dword := SignExtend32(__A.word[2*j+1]) * SignExtend32(__B.word[2*j+1])
/// DST.dword[j] := Saturate32(__S.dword[j] + tmp1 + tmp2)
/// ENDFOR
/// DST[MAX:256] := 0
/// \endoperation
#define _mm256_dpwssds_epi32(__S, __A, __B) \
(__m256i)__builtin_ia32_vpdpwssds256((__v8si)__S, (__v8si)__A, (__v8si)__B)
/// Multiply groups of 4 adjacent pairs of unsigned 8-bit integers in \a __A with
/// corresponding signed 8-bit integers in \a __B, producing 4 intermediate signed
/// 16-bit results. Sum these 4 results with the corresponding 32-bit integer
/// in \a __S, and store the packed 32-bit results in DST.
///
/// This intrinsic corresponds to the <c> VPDPBUSD </c> instructions.
///
/// \operation
/// FOR j := 0 to 3
/// tmp1.word := Signed(ZeroExtend16(__A.byte[4*j]) * SignExtend16(__B.byte[4*j]))
/// tmp2.word := Signed(ZeroExtend16(__A.byte[4*j+1]) * SignExtend16(__B.byte[4*j+1]))
/// tmp3.word := Signed(ZeroExtend16(__A.byte[4*j+2]) * SignExtend16(__B.byte[4*j+2]))
/// tmp4.word := Signed(ZeroExtend16(__A.byte[4*j+3]) * SignExtend16(__B.byte[4*j+3]))
/// DST.dword[j] := __S.dword[j] + tmp1 + tmp2 + tmp3 + tmp4
/// ENDFOR
/// DST[MAX:128] := 0
/// \endoperation
#define _mm_dpbusd_epi32(__S, __A, __B) \
(__m128i)__builtin_ia32_vpdpbusd128((__v4si)__S, (__v4si)__A, (__v4si)__B)
/// Multiply groups of 4 adjacent pairs of unsigned 8-bit integers in \a __A with
/// corresponding signed 8-bit integers in \a __B, producing 4 intermediate signed
/// 16-bit results. Sum these 4 results with the corresponding 32-bit integer
/// in \a __S using signed saturation, and store the packed 32-bit results in DST.
///
/// This intrinsic corresponds to the <c> VPDPBUSDS </c> instructions.
///
/// \operation
/// FOR j := 0 to 3
/// tmp1.word := Signed(ZeroExtend16(__A.byte[4*j]) * SignExtend16(__B.byte[4*j]))
/// tmp2.word := Signed(ZeroExtend16(__A.byte[4*j+1]) * SignExtend16(__B.byte[4*j+1]))
/// tmp3.word := Signed(ZeroExtend16(__A.byte[4*j+2]) * SignExtend16(__B.byte[4*j+2]))
/// tmp4.word := Signed(ZeroExtend16(__A.byte[4*j+3]) * SignExtend16(__B.byte[4*j+3]))
/// DST.dword[j] := Saturate32(__S.dword[j] + tmp1 + tmp2 + tmp3 + tmp4)
/// ENDFOR
/// DST[MAX:128] := 0
/// \endoperation
#define _mm_dpbusds_epi32(__S, __A, __B) \
(__m128i)__builtin_ia32_vpdpbusds128((__v4si)__S, (__v4si)__A, (__v4si)__B)
/// Multiply groups of 2 adjacent pairs of signed 16-bit integers in \a __A with
/// corresponding 16-bit integers in \a __B, producing 2 intermediate signed 32-bit
/// results. Sum these 2 results with the corresponding 32-bit integer in \a __S,
/// and store the packed 32-bit results in DST.
///
/// This intrinsic corresponds to the <c> VPDPWSSD </c> instructions.
///
/// \operation
/// FOR j := 0 to 3
/// tmp1.dword := SignExtend32(__A.word[2*j]) * SignExtend32(__B.word[2*j])
/// tmp2.dword := SignExtend32(__A.word[2*j+1]) * SignExtend32(__B.word[2*j+1])
/// DST.dword[j] := __S.dword[j] + tmp1 + tmp2
/// ENDFOR
/// DST[MAX:128] := 0
/// \endoperation
#define _mm_dpwssd_epi32(__S, __A, __B) \
(__m128i)__builtin_ia32_vpdpwssd128((__v4si)__S, (__v4si)__A, (__v4si)__B)
/// Multiply groups of 2 adjacent pairs of signed 16-bit integers in \a __A with
/// corresponding 16-bit integers in \a __B, producing 2 intermediate signed 32-bit
/// results. Sum these 2 results with the corresponding 32-bit integer in \a __S
/// using signed saturation, and store the packed 32-bit results in DST.
///
/// This intrinsic corresponds to the <c> VPDPWSSDS </c> instructions.
///
/// \operation
/// FOR j := 0 to 3
/// tmp1.dword := SignExtend32(__A.word[2*j]) * SignExtend32(__B.word[2*j])
/// tmp2.dword := SignExtend32(__A.word[2*j+1]) * SignExtend32(__B.word[2*j+1])
/// DST.dword[j] := Saturate32(__S.dword[j] + tmp1 + tmp2)
/// ENDFOR
/// DST[MAX:128] := 0
/// \endoperation
#define _mm_dpwssds_epi32(__S, __A, __B) \
(__m128i)__builtin_ia32_vpdpwssds128((__v4si)__S, (__v4si)__A, (__v4si)__B)
#endif // __COMMONVNNIINTRIN_H

clang/lib/Headers/cpuid.h

Show First 20 Lines • Show All 189 Lines • ▼ Show 20 Lines
#define bit_TSXLDTRK 0x00010000 #define bit_TSXLDTRK 0x00010000
#define bit_PCONFIG 0x00040000 #define bit_PCONFIG 0x00040000
#define bit_IBT 0x00100000 #define bit_IBT 0x00100000
#define bit_AMXBF16 0x00400000 #define bit_AMXBF16 0x00400000
#define bit_AMXTILE 0x01000000 #define bit_AMXTILE 0x01000000
#define bit_AMXINT8 0x02000000 #define bit_AMXINT8 0x02000000
/* Features in %eax for leaf 7 sub-leaf 1 */ /* Features in %eax for leaf 7 sub-leaf 1 */
#define bit_AVXVNNI 0x00000008
RKSimonUnsubmitted
Not Done
ReplyInline Actions

@LiuChen3 https://github.com/llvm/llvm-project/issues/53392 reports this should be 0x00000010?

RKSimon: @liuchen3 https://github.com/llvm/llvm-project/issues/53392 reports this should be 0x00000010?
craig.topperUnsubmitted
Not Done
ReplyInline Actions

Maybe we should start writing these as 1 << 4. This is not the first shifted by one issue we've had in this file.

craig.topper: Maybe we should start writing these as 1 << 4. This is not the first shifted by one issue we've…
LiuChen3AuthorUnsubmitted
Done
ReplyInline Actions

It my fault. I have a fix here: https://reviews.llvm.org/D118103 .

LiuChen3: It my fault. I have a fix here: https://reviews.llvm.org/D118103 .
#define bit_AVX512BF16 0x00000020 #define bit_AVX512BF16 0x00000020
/* Features in %eax for leaf 13 sub-leaf 1 */ /* Features in %eax for leaf 13 sub-leaf 1 */
#define bit_XSAVEOPT 0x00000001 #define bit_XSAVEOPT 0x00000001
#define bit_XSAVEC 0x00000002 #define bit_XSAVEC 0x00000002
#define bit_XSAVES 0x00000008 #define bit_XSAVES 0x00000008
/* Features in %eax for leaf 0x14 sub-leaf 0 */ /* Features in %eax for leaf 0x14 sub-leaf 0 */
▲ Show 20 Lines • Show All 106 Lines • Show Last 20 Lines

clang/lib/Headers/immintrin.h

Show First 20 Lines • Show All 133 Lines • ▼ Show 20 Lines
#endif #endif
#if !(defined(_MSC_VER) || defined(__SCE__)) || __has_feature(modules) || \ #if !(defined(_MSC_VER) || defined(__SCE__)) || __has_feature(modules) || \
defined(__AVX512VNNI__) defined(__AVX512VNNI__)
#include <avx512vnniintrin.h> #include <avx512vnniintrin.h>
#endif #endif
#if !(defined(_MSC_VER) || defined(__SCE__)) || __has_feature(modules) || \ #if !(defined(_MSC_VER) || defined(__SCE__)) || __has_feature(modules) || \
defined(__AVXVNNI__)
#include <avxvnniintrin.h>
#endif
#if !(defined(_MSC_VER) || defined(__SCE__)) || __has_feature(modules) || \
(defined(__AVX512VL__) && defined(__AVX512VNNI__)) (defined(__AVX512VL__) && defined(__AVX512VNNI__))
pengfeiUnsubmitted
Not Done
ReplyInline Actions

Change back the spaces.

pengfei: Change back the spaces.
#include <avx512vlvnniintrin.h> #include <avx512vlvnniintrin.h>
#endif #endif
#if !(defined(_MSC_VER) || defined(__SCE__)) || __has_feature(modules) || \ #if !(defined(_MSC_VER) || defined(__SCE__)) || __has_feature(modules) || \
defined(__AVX512DQ__) defined(__AVX512DQ__)
#include <avx512dqintrin.h> #include <avx512dqintrin.h>
#endif #endif
▲ Show 20 Lines • Show All 443 Lines • Show Last 20 Lines

clang/test/CodeGen/attr-target-x86.c

Show First 20 Lines • Show All 44 Lines • ▼ Show 20 Lines
// CHECK: qax{{.*}} #5 // CHECK: qax{{.*}} #5
// CHECK: qq{{.*}} #6 // CHECK: qq{{.*}} #6
// CHECK: lake{{.*}} #7 // CHECK: lake{{.*}} #7
// CHECK: use_before_def{{.*}} #7 // CHECK: use_before_def{{.*}} #7
// CHECK: walrus{{.*}} #8 // CHECK: walrus{{.*}} #8
// CHECK: #0 = {{.*}}"target-cpu"="i686" "target-features"="+cx8,+x87" "tune-cpu"="i686" // CHECK: #0 = {{.*}}"target-cpu"="i686" "target-features"="+cx8,+x87" "tune-cpu"="i686"
// CHECK: #1 = {{.*}}"target-cpu"="ivybridge" "target-features"="+avx,+cx16,+cx8,+f16c,+fsgsbase,+fxsr,+mmx,+pclmul,+popcnt,+rdrnd,+sahf,+sse,+sse2,+sse3,+sse4.1,+sse4.2,+ssse3,+x87,+xsave,+xsaveopt" // CHECK: #1 = {{.*}}"target-cpu"="ivybridge" "target-features"="+avx,+cx16,+cx8,+f16c,+fsgsbase,+fxsr,+mmx,+pclmul,+popcnt,+rdrnd,+sahf,+sse,+sse2,+sse3,+sse4.1,+sse4.2,+ssse3,+x87,+xsave,+xsaveopt"
// CHECK-NOT: tune-cpu // CHECK-NOT: tune-cpu
// CHECK: #2 = {{.*}}"target-cpu"="i686" "target-features"="+cx8,+x87,-aes,-avx,-avx2,-avx512bf16,-avx512bitalg,-avx512bw,-avx512cd,-avx512dq,-avx512er,-avx512f,-avx512ifma,-avx512pf,-avx512vbmi,-avx512vbmi2,-avx512vl,-avx512vnni,-avx512vp2intersect,-avx512vpopcntdq,-f16c,-fma,-fma4,-gfni,-kl,-pclmul,-sha,-sse2,-sse3,-sse4.1,-sse4.2,-sse4a,-ssse3,-vaes,-vpclmulqdq,-widekl,-xop" "tune-cpu"="i686" // CHECK: #2 = {{.*}}"target-cpu"="i686" "target-features"="+cx8,+x87,-aes,-avx,-avx2,-avx512bf16,-avx512bitalg,-avx512bw,-avx512cd,-avx512dq,-avx512er,-avx512f,-avx512ifma,-avx512pf,-avx512vbmi,-avx512vbmi2,-avx512vl,-avx512vnni,-avx512vp2intersect,-avx512vpopcntdq,-avxvnni,-f16c,-fma,-fma4,-gfni,-kl,-pclmul,-sha,-sse2,-sse3,-sse4.1,-sse4.2,-sse4a,-ssse3,-vaes,-vpclmulqdq,-widekl,-xop" "tune-cpu"="i686"
// CHECK: #3 = {{.*}}"target-cpu"="i686" "target-features"="+cx8,+mmx,+popcnt,+sse,+sse2,+sse3,+sse4.1,+sse4.2,+ssse3,+x87" "tune-cpu"="i686" // CHECK: #3 = {{.*}}"target-cpu"="i686" "target-features"="+cx8,+mmx,+popcnt,+sse,+sse2,+sse3,+sse4.1,+sse4.2,+ssse3,+x87" "tune-cpu"="i686"
// CHECK: #4 = {{.*}}"target-cpu"="i686" "target-features"="+cx8,+x87,-avx,-avx2,-avx512bf16,-avx512bitalg,-avx512bw,-avx512cd,-avx512dq,-avx512er,-avx512f,-avx512ifma,-avx512pf,-avx512vbmi,-avx512vbmi2,-avx512vl,-avx512vnni,-avx512vp2intersect,-avx512vpopcntdq,-f16c,-fma,-fma4,-sse4.1,-sse4.2,-vaes,-vpclmulqdq,-xop" "tune-cpu"="i686" // CHECK: #4 = {{.*}}"target-cpu"="i686" "target-features"="+cx8,+x87,-avx,-avx2,-avx512bf16,-avx512bitalg,-avx512bw,-avx512cd,-avx512dq,-avx512er,-avx512f,-avx512ifma,-avx512pf,-avx512vbmi,-avx512vbmi2,-avx512vl,-avx512vnni,-avx512vp2intersect,-avx512vpopcntdq,-avxvnni,-f16c,-fma,-fma4,-sse4.1,-sse4.2,-vaes,-vpclmulqdq,-xop" "tune-cpu"="i686"
// CHECK: #5 = {{.*}}"target-cpu"="ivybridge" "target-features"="+avx,+cx16,+cx8,+f16c,+fsgsbase,+fxsr,+mmx,+pclmul,+popcnt,+rdrnd,+sahf,+sse,+sse2,+sse3,+sse4.1,+sse4.2,+ssse3,+x87,+xsave,+xsaveopt,-aes,-vaes" // CHECK: #5 = {{.*}}"target-cpu"="ivybridge" "target-features"="+avx,+cx16,+cx8,+f16c,+fsgsbase,+fxsr,+mmx,+pclmul,+popcnt,+rdrnd,+sahf,+sse,+sse2,+sse3,+sse4.1,+sse4.2,+ssse3,+x87,+xsave,+xsaveopt,-aes,-vaes"
// CHECK-NOT: tune-cpu // CHECK-NOT: tune-cpu
// CHECK: #6 = {{.*}}"target-cpu"="i686" "target-features"="+cx8,+x87,-3dnow,-3dnowa,-mmx" // CHECK: #6 = {{.*}}"target-cpu"="i686" "target-features"="+cx8,+x87,-3dnow,-3dnowa,-mmx"
// CHECK: #7 = {{.*}}"target-cpu"="lakemont" "target-features"="+cx8,+mmx" // CHECK: #7 = {{.*}}"target-cpu"="lakemont" "target-features"="+cx8,+mmx"
// CHECK-NOT: tune-cpu // CHECK-NOT: tune-cpu
// CHECK: #8 = {{.*}}"target-cpu"="i686" "target-features"="+cx8,+x87" "tune-cpu"="sandybridge" // CHECK: #8 = {{.*}}"target-cpu"="i686" "target-features"="+cx8,+x87" "tune-cpu"="sandybridge"

clang/test/CodeGen/avxvnni-builtins.c

  • This file was added.
// RUN: %clang_cc1 -ffreestanding %s -triple=x86_64-apple-darwin -target-feature +avxvnni -emit-llvm -o - -Wall -Werror | FileCheck %s
RKSimonUnsubmitted
Not Done
ReplyInline Actions

Please can you move this test file under the X86 subdirectory?

RKSimon: Please can you move this test file under the X86 subdirectory?
#include <immintrin.h>
__m256i test_mm256_dpbusd_epi32(__m256i __S, __m256i __A, __m256i __B) {
// CHECK-LABEL: @test_mm256_dpbusd_epi32
// CHECK: @llvm.x86.avx512.vpdpbusd.256
return _mm256_dpbusd_epi32(__S, __A, __B);
}
__m256i test_mm256_dpbusds_epi32(__m256i __S, __m256i __A, __m256i __B) {
// CHECK-LABEL: @test_mm256_dpbusds_epi32
// CHECK: @llvm.x86.avx512.vpdpbusds.256
return _mm256_dpbusds_epi32(__S, __A, __B);
}
__m256i test_mm256_dpwssd_epi32(__m256i __S, __m256i __A, __m256i __B) {
// CHECK-LABEL: @test_mm256_dpwssd_epi32
// CHECK: @llvm.x86.avx512.vpdpwssd.256
return _mm256_dpwssd_epi32(__S, __A, __B);
}
__m256i test_mm256_dpwssds_epi32(__m256i __S, __m256i __A, __m256i __B) {
// CHECK-LABEL: @test_mm256_dpwssds_epi32
// CHECK: @llvm.x86.avx512.vpdpwssds.256
return _mm256_dpwssds_epi32(__S, __A, __B);
}
__m128i test_mm_dpbusd_epi32(__m128i __S, __m128i __A, __m128i __B) {
// CHECK-LABEL: @test_mm_dpbusd_epi32
// CHECK: @llvm.x86.avx512.vpdpbusd.128
return _mm_dpbusd_epi32(__S, __A, __B);
}
__m128i test_mm_dpbusds_epi32(__m128i __S, __m128i __A, __m128i __B) {
// CHECK-LABEL: @test_mm_dpbusds_epi32
// CHECK: @llvm.x86.avx512.vpdpbusds.128
return _mm_dpbusds_epi32(__S, __A, __B);
}
__m128i test_mm_dpwssd_epi32(__m128i __S, __m128i __A, __m128i __B) {
// CHECK-LABEL: @test_mm_dpwssd_epi32
// CHECK: @llvm.x86.avx512.vpdpwssd.128
return _mm_dpwssd_epi32(__S, __A, __B);
}
__m128i test_mm_dpwssds_epi32(__m128i __S, __m128i __A, __m128i __B) {
// CHECK-LABEL: @test_mm_dpwssds_epi32
// CHECK: @llvm.x86.avx512.vpdpwssds.128
return _mm_dpwssds_epi32(__S, __A, __B);
}
__m256i test_mm256_dpbusd_avx_epi32(__m256i __S, __m256i __A, __m256i __B) {
// CHECK-LABEL: @test_mm256_dpbusd_avx_epi32
// CHECK: @llvm.x86.avx512.vpdpbusd.256
return _mm256_dpbusd_avx_epi32(__S, __A, __B);
}
__m256i test_mm256_dpbusds_avx_epi32(__m256i __S, __m256i __A, __m256i __B) {
// CHECK-LABEL: @test_mm256_dpbusds_avx_epi32
// CHECK: @llvm.x86.avx512.vpdpbusds.256
return _mm256_dpbusds_avx_epi32(__S, __A, __B);
}
__m256i test_mm256_dpwssd_avx_epi32(__m256i __S, __m256i __A, __m256i __B) {
// CHECK-LABEL: @test_mm256_dpwssd_avx_epi32
// CHECK: @llvm.x86.avx512.vpdpwssd.256
return _mm256_dpwssd_avx_epi32(__S, __A, __B);
}
__m256i test_mm256_dpwssds_avx_epi32(__m256i __S, __m256i __A, __m256i __B) {
// CHECK-LABEL: @test_mm256_dpwssds_avx_epi32
// CHECK: @llvm.x86.avx512.vpdpwssds.256
return _mm256_dpwssds_avx_epi32(__S, __A, __B);
}
__m128i test_mm_dpbusd_avx_epi32(__m128i __S, __m128i __A, __m128i __B) {
// CHECK-LABEL: @test_mm_dpbusd_avx_epi32
// CHECK: @llvm.x86.avx512.vpdpbusd.128
return _mm_dpbusd_avx_epi32(__S, __A, __B);
}
__m128i test_mm_dpbusds_avx_epi32(__m128i __S, __m128i __A, __m128i __B) {
// CHECK-LABEL: @test_mm_dpbusds_avx_epi32
// CHECK: @llvm.x86.avx512.vpdpbusds.128
return _mm_dpbusds_avx_epi32(__S, __A, __B);
}
__m128i test_mm_dpwssd_avx_epi32(__m128i __S, __m128i __A, __m128i __B) {
// CHECK-LABEL: @test_mm_dpwssd_avx_epi32
// CHECK: @llvm.x86.avx512.vpdpwssd.128
return _mm_dpwssd_avx_epi32(__S, __A, __B);
}
__m128i test_mm_dpwssds_avx_epi32(__m128i __S, __m128i __A, __m128i __B) {
// CHECK-LABEL: @test_mm_dpwssds_avx_epi32
// CHECK: @llvm.x86.avx512.vpdpwssds.128
return _mm_dpwssds_avx_epi32(__S, __A, __B);
}

clang/test/CodeGen/builtins-nvptx-mma.cu

Show All 29 Lines
// CHECK-LABEL: test_wmma_buitins // CHECK-LABEL: test_wmma_buitins
__device__ void test_wmma_buitins(int *src, int *dst, __device__ void test_wmma_buitins(int *src, int *dst,
float *fsrc, float *fdst, int ldm) { float *fsrc, float *fdst, int ldm) {
#if (PTX >= 60) && (SM >= 70) #if (PTX >= 60) && (SM >= 70)
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.a.col.stride.f16 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.a.col.stride.f16
// expected-error-re@+1 {{'__hmma_m16n16k16_ld_a' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_ld_a' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_ld_a(dst, src, ldm, 1); __hmma_m16n16k16_ld_a(dst, src, ldm, 1);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.a.row.stride.f16 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.a.row.stride.f16
// expected-error-re@+1 {{'__hmma_m16n16k16_ld_a' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_ld_a' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_ld_a(dst, src, ldm, 0); __hmma_m16n16k16_ld_a(dst, src, ldm, 0);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.b.col.stride.f16 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.b.col.stride.f16
// expected-error-re@+1 {{'__hmma_m16n16k16_ld_b' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_ld_b' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_ld_b(dst, src, ldm, 1); __hmma_m16n16k16_ld_b(dst, src, ldm, 1);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.b.row.stride.f16 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.b.row.stride.f16
// expected-error-re@+1 {{'__hmma_m16n16k16_ld_b' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_ld_b' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_ld_b(dst, src, ldm, 0); __hmma_m16n16k16_ld_b(dst, src, ldm, 0);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.c.col.stride.f16 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.c.col.stride.f16
// expected-error-re@+1 {{'__hmma_m16n16k16_ld_c_f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_ld_c_f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_ld_c_f16(dst, src, ldm, 1); __hmma_m16n16k16_ld_c_f16(dst, src, ldm, 1);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.c.row.stride.f16 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.c.row.stride.f16
// expected-error-re@+1 {{'__hmma_m16n16k16_ld_c_f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_ld_c_f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_ld_c_f16(dst, src, ldm, 0); __hmma_m16n16k16_ld_c_f16(dst, src, ldm, 0);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.c.col.stride.f32 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.c.col.stride.f32
// expected-error-re@+1 {{'__hmma_m16n16k16_ld_c_f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_ld_c_f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_ld_c_f32(fdst, fsrc, ldm, 1); __hmma_m16n16k16_ld_c_f32(fdst, fsrc, ldm, 1);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.c.row.stride.f32 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.c.row.stride.f32
// expected-error-re@+1 {{'__hmma_m16n16k16_ld_c_f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_ld_c_f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_ld_c_f32(fdst, fsrc, ldm, 0); __hmma_m16n16k16_ld_c_f32(fdst, fsrc, ldm, 0);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.store.d.col.stride.f16 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.store.d.col.stride.f16
// expected-error-re@+1 {{'__hmma_m16n16k16_st_c_f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_st_c_f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_st_c_f16(dst, src, ldm, 1); __hmma_m16n16k16_st_c_f16(dst, src, ldm, 1);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.store.d.row.stride.f16 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.store.d.row.stride.f16
// expected-error-re@+1 {{'__hmma_m16n16k16_st_c_f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_st_c_f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_st_c_f16(dst, src, ldm, 0); __hmma_m16n16k16_st_c_f16(dst, src, ldm, 0);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.store.d.col.stride.f32 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.store.d.col.stride.f32
// expected-error-re@+1 {{'__hmma_m16n16k16_st_c_f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_st_c_f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_st_c_f32(fdst, fsrc, ldm, 1); __hmma_m16n16k16_st_c_f32(fdst, fsrc, ldm, 1);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.store.d.row.stride.f32 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.store.d.row.stride.f32
// expected-error-re@+1 {{'__hmma_m16n16k16_st_c_f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_st_c_f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_st_c_f32(fdst, fsrc, ldm, 0); __hmma_m16n16k16_st_c_f32(fdst, fsrc, ldm, 0);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f16.f16 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f16.f16
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f16(dst, src, src, src, 3, 0); __hmma_m16n16k16_mma_f16f16(dst, src, src, src, 3, 0);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f16.f16.satfinite // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f16.f16.satfinite
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f16(dst, src, src, src, 3, 1); __hmma_m16n16k16_mma_f16f16(dst, src, src, src, 3, 1);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f16.f16 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f16.f16
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f16(dst, src, src, src, 2, 0); __hmma_m16n16k16_mma_f16f16(dst, src, src, src, 2, 0);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f16.f16.satfinite // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f16.f16.satfinite
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f16(dst, src, src, src, 2, 1); __hmma_m16n16k16_mma_f16f16(dst, src, src, src, 2, 1);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f16.f16 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f16.f16
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f16(dst, src, src, src, 1, 0); __hmma_m16n16k16_mma_f16f16(dst, src, src, src, 1, 0);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f16.f16.satfinite // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f16.f16.satfinite
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f16(dst, src, src, src, 1, 1); __hmma_m16n16k16_mma_f16f16(dst, src, src, src, 1, 1);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f16.f16 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f16.f16
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f16(dst, src, src, src, 0, 0); __hmma_m16n16k16_mma_f16f16(dst, src, src, src, 0, 0);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f16.f16.satfinite // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f16.f16.satfinite
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f16(dst, src, src, src, 0, 1); __hmma_m16n16k16_mma_f16f16(dst, src, src, src, 0, 1);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f32.f16 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f32.f16
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 3, 0); __hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 3, 0);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f32.f16.satfinite // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f32.f16.satfinite
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 3, 1); __hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 3, 1);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f32.f16 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f32.f16
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 2, 0); __hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 2, 0);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f32.f16.satfinite // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f32.f16.satfinite
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 2, 1); __hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 2, 1);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f32.f16 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f32.f16
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 1, 0); __hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 1, 0);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f32.f16.satfinite // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f32.f16.satfinite
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 1, 1); __hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 1, 1);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f32.f16 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f32.f16
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 0, 0); __hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 0, 0);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f32.f16.satfinite // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f32.f16.satfinite
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 0, 1); __hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 0, 1);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f16.f32 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f16.f32
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 3, 0); __hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 3, 0);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f16.f32.satfinite // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f16.f32.satfinite
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 3, 1); __hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 3, 1);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f16.f32 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f16.f32
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 2, 0); __hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 2, 0);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f16.f32.satfinite // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f16.f32.satfinite
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 2, 1); __hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 2, 1);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f16.f32 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f16.f32
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 1, 0); __hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 1, 0);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f16.f32.satfinite // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f16.f32.satfinite
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 1, 1); __hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 1, 1);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f16.f32 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f16.f32
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 0, 0); __hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 0, 0);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f16.f32.satfinite // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f16.f32.satfinite
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 0, 1); __hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 0, 1);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f32.f32 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f32.f32
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 3, 0); __hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 3, 0);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f32.f32.satfinite // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f32.f32.satfinite
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 3, 1); __hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 3, 1);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f32.f32 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f32.f32
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 2, 0); __hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 2, 0);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f32.f32.satfinite // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f32.f32.satfinite
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 2, 1); __hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 2, 1);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f32.f32 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f32.f32
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 1, 0); __hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 1, 0);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f32.f32.satfinite // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f32.f32.satfinite
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 1, 1); __hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 1, 1);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f32.f32 // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f32.f32
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 0, 0); __hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 0, 0);
// CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f32.f32.satfinite // CHECK_PTX60_SM70: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f32.f32.satfinite
// expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // expected-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 0, 1); __hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 0, 1);
#endif // (PTX >= 60) && (SM >= 70) #endif // (PTX >= 60) && (SM >= 70)
#if (PTX >= 61) && (SM >= 70) #if (PTX >= 61) && (SM >= 70)
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.a.col.stride.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.a.col.stride.f16
// expected-error-re@+1 {{'__hmma_m32n8k16_ld_a' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_ld_a' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_ld_a(dst, src, ldm, 1); __hmma_m32n8k16_ld_a(dst, src, ldm, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.a.row.stride.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.a.row.stride.f16
// expected-error-re@+1 {{'__hmma_m32n8k16_ld_a' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_ld_a' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_ld_a(dst, src, ldm, 0); __hmma_m32n8k16_ld_a(dst, src, ldm, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.b.col.stride.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.b.col.stride.f16
// expected-error-re@+1 {{'__hmma_m32n8k16_ld_b' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_ld_b' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_ld_b(dst, src, ldm, 1); __hmma_m32n8k16_ld_b(dst, src, ldm, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.b.row.stride.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.b.row.stride.f16
// expected-error-re@+1 {{'__hmma_m32n8k16_ld_b' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_ld_b' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_ld_b(dst, src, ldm, 0); __hmma_m32n8k16_ld_b(dst, src, ldm, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.c.col.stride.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.c.col.stride.f16
// expected-error-re@+1 {{'__hmma_m32n8k16_ld_c_f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_ld_c_f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_ld_c_f16(dst, src, ldm, 1); __hmma_m32n8k16_ld_c_f16(dst, src, ldm, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.c.row.stride.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.c.row.stride.f16
// expected-error-re@+1 {{'__hmma_m32n8k16_ld_c_f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_ld_c_f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_ld_c_f16(dst, src, ldm, 0); __hmma_m32n8k16_ld_c_f16(dst, src, ldm, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.c.col.stride.f32 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.c.col.stride.f32
// expected-error-re@+1 {{'__hmma_m32n8k16_ld_c_f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_ld_c_f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_ld_c_f32(fdst, fsrc, ldm, 1); __hmma_m32n8k16_ld_c_f32(fdst, fsrc, ldm, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.c.row.stride.f32 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.c.row.stride.f32
// expected-error-re@+1 {{'__hmma_m32n8k16_ld_c_f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_ld_c_f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_ld_c_f32(fdst, fsrc, ldm, 0); __hmma_m32n8k16_ld_c_f32(fdst, fsrc, ldm, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.store.d.col.stride.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.store.d.col.stride.f16
// expected-error-re@+1 {{'__hmma_m32n8k16_st_c_f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_st_c_f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_st_c_f16(dst, src, ldm, 1); __hmma_m32n8k16_st_c_f16(dst, src, ldm, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.store.d.row.stride.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.store.d.row.stride.f16
// expected-error-re@+1 {{'__hmma_m32n8k16_st_c_f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_st_c_f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_st_c_f16(dst, src, ldm, 0); __hmma_m32n8k16_st_c_f16(dst, src, ldm, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.store.d.col.stride.f32 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.store.d.col.stride.f32
// expected-error-re@+1 {{'__hmma_m32n8k16_st_c_f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_st_c_f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_st_c_f32(fdst, fsrc, ldm, 1); __hmma_m32n8k16_st_c_f32(fdst, fsrc, ldm, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.store.d.row.stride.f32 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.store.d.row.stride.f32
// expected-error-re@+1 {{'__hmma_m32n8k16_st_c_f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_st_c_f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_st_c_f32(fdst, fsrc, ldm, 0); __hmma_m32n8k16_st_c_f32(fdst, fsrc, ldm, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.a.col.stride.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.a.col.stride.f16
// expected-error-re@+1 {{'__hmma_m8n32k16_ld_a' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_ld_a' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_ld_a(dst, src, ldm, 1); __hmma_m8n32k16_ld_a(dst, src, ldm, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.a.row.stride.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.a.row.stride.f16
// expected-error-re@+1 {{'__hmma_m8n32k16_ld_a' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_ld_a' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_ld_a(dst, src, ldm, 0); __hmma_m8n32k16_ld_a(dst, src, ldm, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.b.col.stride.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.b.col.stride.f16
// expected-error-re@+1 {{'__hmma_m8n32k16_ld_b' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_ld_b' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_ld_b(dst, src, ldm, 1); __hmma_m8n32k16_ld_b(dst, src, ldm, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.b.row.stride.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.b.row.stride.f16
// expected-error-re@+1 {{'__hmma_m8n32k16_ld_b' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_ld_b' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_ld_b(dst, src, ldm, 0); __hmma_m8n32k16_ld_b(dst, src, ldm, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.c.col.stride.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.c.col.stride.f16
// expected-error-re@+1 {{'__hmma_m8n32k16_ld_c_f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_ld_c_f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_ld_c_f16(dst, src, ldm, 1); __hmma_m8n32k16_ld_c_f16(dst, src, ldm, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.c.row.stride.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.c.row.stride.f16
// expected-error-re@+1 {{'__hmma_m8n32k16_ld_c_f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_ld_c_f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_ld_c_f16(dst, src, ldm, 0); __hmma_m8n32k16_ld_c_f16(dst, src, ldm, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.c.col.stride.f32 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.c.col.stride.f32
// expected-error-re@+1 {{'__hmma_m8n32k16_ld_c_f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_ld_c_f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_ld_c_f32(fdst, fsrc, ldm, 1); __hmma_m8n32k16_ld_c_f32(fdst, fsrc, ldm, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.c.row.stride.f32 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.c.row.stride.f32
// expected-error-re@+1 {{'__hmma_m8n32k16_ld_c_f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_ld_c_f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_ld_c_f32(fdst, fsrc, ldm, 0); __hmma_m8n32k16_ld_c_f32(fdst, fsrc, ldm, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.store.d.col.stride.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.store.d.col.stride.f16
// expected-error-re@+1 {{'__hmma_m8n32k16_st_c_f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_st_c_f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_st_c_f16(dst, src, ldm, 1); __hmma_m8n32k16_st_c_f16(dst, src, ldm, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.store.d.row.stride.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.store.d.row.stride.f16
// expected-error-re@+1 {{'__hmma_m8n32k16_st_c_f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_st_c_f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_st_c_f16(dst, src, ldm, 0); __hmma_m8n32k16_st_c_f16(dst, src, ldm, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.store.d.col.stride.f32 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.store.d.col.stride.f32
// expected-error-re@+1 {{'__hmma_m8n32k16_st_c_f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_st_c_f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_st_c_f32(fdst, fsrc, ldm, 1); __hmma_m8n32k16_st_c_f32(fdst, fsrc, ldm, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.store.d.row.stride.f32 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.store.d.row.stride.f32
// expected-error-re@+1 {{'__hmma_m8n32k16_st_c_f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_st_c_f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_st_c_f32(fdst, fsrc, ldm, 0); __hmma_m8n32k16_st_c_f32(fdst, fsrc, ldm, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f16.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f16.f16
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f16(dst, src, src, src, 3, 0); __hmma_m32n8k16_mma_f16f16(dst, src, src, src, 3, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f16.f16.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f16.f16.satfinite
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f16(dst, src, src, src, 3, 1); __hmma_m32n8k16_mma_f16f16(dst, src, src, src, 3, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f16.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f16.f16
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f16(dst, src, src, src, 2, 0); __hmma_m32n8k16_mma_f16f16(dst, src, src, src, 2, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f16.f16.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f16.f16.satfinite
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f16(dst, src, src, src, 2, 1); __hmma_m32n8k16_mma_f16f16(dst, src, src, src, 2, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f16.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f16.f16
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f16(dst, src, src, src, 1, 0); __hmma_m32n8k16_mma_f16f16(dst, src, src, src, 1, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f16.f16.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f16.f16.satfinite
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f16(dst, src, src, src, 1, 1); __hmma_m32n8k16_mma_f16f16(dst, src, src, src, 1, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f16.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f16.f16
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f16(dst, src, src, src, 0, 0); __hmma_m32n8k16_mma_f16f16(dst, src, src, src, 0, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f16.f16.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f16.f16.satfinite
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f16(dst, src, src, src, 0, 1); __hmma_m32n8k16_mma_f16f16(dst, src, src, src, 0, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f32.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f32.f16
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 3, 0); __hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 3, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f32.f16.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f32.f16.satfinite
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 3, 1); __hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 3, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f32.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f32.f16
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 2, 0); __hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 2, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f32.f16.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f32.f16.satfinite
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 2, 1); __hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 2, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f32.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f32.f16
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 1, 0); __hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 1, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f32.f16.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f32.f16.satfinite
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 1, 1); __hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 1, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f32.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f32.f16
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 0, 0); __hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 0, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f32.f16.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f32.f16.satfinite
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 0, 1); __hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 0, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f16.f32 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f16.f32
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 3, 0); __hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 3, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f16.f32.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f16.f32.satfinite
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 3, 1); __hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 3, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f16.f32 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f16.f32
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 2, 0); __hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 2, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f16.f32.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f16.f32.satfinite
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 2, 1); __hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 2, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f16.f32 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f16.f32
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 1, 0); __hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 1, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f16.f32.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f16.f32.satfinite
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 1, 1); __hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 1, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f16.f32 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f16.f32
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 0, 0); __hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 0, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f16.f32.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f16.f32.satfinite
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 0, 1); __hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 0, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f32.f32 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f32.f32
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 3, 0); __hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 3, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f32.f32.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f32.f32.satfinite
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 3, 1); __hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 3, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f32.f32 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f32.f32
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 2, 0); __hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 2, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f32.f32.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f32.f32.satfinite
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 2, 1); __hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 2, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f32.f32 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f32.f32
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 1, 0); __hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 1, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f32.f32.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f32.f32.satfinite
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 1, 1); __hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 1, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f32.f32 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f32.f32
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 0, 0); __hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 0, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f32.f32.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f32.f32.satfinite
// expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 0, 1); __hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 0, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f16.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f16.f16
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f16(dst, src, src, src, 3, 0); __hmma_m8n32k16_mma_f16f16(dst, src, src, src, 3, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f16.f16.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f16.f16.satfinite
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f16(dst, src, src, src, 3, 1); __hmma_m8n32k16_mma_f16f16(dst, src, src, src, 3, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f16.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f16.f16
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f16(dst, src, src, src, 2, 0); __hmma_m8n32k16_mma_f16f16(dst, src, src, src, 2, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f16.f16.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f16.f16.satfinite
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f16(dst, src, src, src, 2, 1); __hmma_m8n32k16_mma_f16f16(dst, src, src, src, 2, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f16.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f16.f16
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f16(dst, src, src, src, 1, 0); __hmma_m8n32k16_mma_f16f16(dst, src, src, src, 1, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f16.f16.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f16.f16.satfinite
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f16(dst, src, src, src, 1, 1); __hmma_m8n32k16_mma_f16f16(dst, src, src, src, 1, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f16.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f16.f16
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f16(dst, src, src, src, 0, 0); __hmma_m8n32k16_mma_f16f16(dst, src, src, src, 0, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f16.f16.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f16.f16.satfinite
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f16(dst, src, src, src, 0, 1); __hmma_m8n32k16_mma_f16f16(dst, src, src, src, 0, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f32.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f32.f16
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 3, 0); __hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 3, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f32.f16.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f32.f16.satfinite
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 3, 1); __hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 3, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f32.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f32.f16
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 2, 0); __hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 2, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f32.f16.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f32.f16.satfinite
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 2, 1); __hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 2, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f32.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f32.f16
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 1, 0); __hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 1, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f32.f16.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f32.f16.satfinite
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 1, 1); __hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 1, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f32.f16 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f32.f16
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 0, 0); __hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 0, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f32.f16.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f32.f16.satfinite
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 0, 1); __hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 0, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f16.f32 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f16.f32
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 3, 0); __hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 3, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f16.f32.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f16.f32.satfinite
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 3, 1); __hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 3, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f16.f32 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f16.f32
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 2, 0); __hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 2, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f16.f32.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f16.f32.satfinite
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 2, 1); __hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 2, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f16.f32 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f16.f32
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 1, 0); __hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 1, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f16.f32.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f16.f32.satfinite
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 1, 1); __hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 1, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f16.f32 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f16.f32
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 0, 0); __hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 0, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f16.f32.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f16.f32.satfinite
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 0, 1); __hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 0, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f32.f32 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f32.f32
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 3, 0); __hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 3, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f32.f32.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f32.f32.satfinite
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 3, 1); __hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 3, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f32.f32 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f32.f32
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 2, 0); __hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 2, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f32.f32.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f32.f32.satfinite
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 2, 1); __hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 2, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f32.f32 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f32.f32
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 1, 0); __hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 1, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f32.f32.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f32.f32.satfinite
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 1, 1); __hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 1, 1);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f32.f32 // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f32.f32
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 0, 0); __hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 0, 0);
// CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f32.f32.satfinite // CHECK_PTX61_SM70: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f32.f32.satfinite
// expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // expected-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 0, 1); __hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 0, 1);
#endif // (PTX >= 61) && (SM >= 70) #endif // (PTX >= 61) && (SM >= 70)
#if (PTX >= 63) && (SM >= 72) #if (PTX >= 63) && (SM >= 72)
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.a.col.stride.s8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.a.col.stride.s8
// expected-error-re@+1 {{'__imma_m16n16k16_ld_a_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_ld_a_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_ld_a_s8(dst, src, ldm, 1); __imma_m16n16k16_ld_a_s8(dst, src, ldm, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.a.row.stride.s8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.a.row.stride.s8
// expected-error-re@+1 {{'__imma_m16n16k16_ld_a_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_ld_a_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_ld_a_s8(dst, src, ldm, 0); __imma_m16n16k16_ld_a_s8(dst, src, ldm, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.a.col.stride.u8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.a.col.stride.u8
// expected-error-re@+1 {{'__imma_m16n16k16_ld_a_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_ld_a_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_ld_a_u8(dst, src, ldm, 1); __imma_m16n16k16_ld_a_u8(dst, src, ldm, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.a.row.stride.u8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.a.row.stride.u8
// expected-error-re@+1 {{'__imma_m16n16k16_ld_a_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_ld_a_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_ld_a_u8(dst, src, ldm, 0); __imma_m16n16k16_ld_a_u8(dst, src, ldm, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.b.col.stride.s8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.b.col.stride.s8
// expected-error-re@+1 {{'__imma_m16n16k16_ld_b_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_ld_b_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_ld_b_s8(dst, src, ldm, 1); __imma_m16n16k16_ld_b_s8(dst, src, ldm, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.b.row.stride.s8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.b.row.stride.s8
// expected-error-re@+1 {{'__imma_m16n16k16_ld_b_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_ld_b_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_ld_b_s8(dst, src, ldm, 0); __imma_m16n16k16_ld_b_s8(dst, src, ldm, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.b.col.stride.u8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.b.col.stride.u8
// expected-error-re@+1 {{'__imma_m16n16k16_ld_b_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_ld_b_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_ld_b_u8(dst, src, ldm, 1); __imma_m16n16k16_ld_b_u8(dst, src, ldm, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.b.row.stride.u8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.b.row.stride.u8
// expected-error-re@+1 {{'__imma_m16n16k16_ld_b_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_ld_b_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_ld_b_u8(dst, src, ldm, 0); __imma_m16n16k16_ld_b_u8(dst, src, ldm, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.c.col.stride.s32 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.c.col.stride.s32
// expected-error-re@+1 {{'__imma_m16n16k16_ld_c' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_ld_c' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_ld_c(dst, src, ldm, 1); __imma_m16n16k16_ld_c(dst, src, ldm, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.c.row.stride.s32 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.c.row.stride.s32
// expected-error-re@+1 {{'__imma_m16n16k16_ld_c' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_ld_c' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_ld_c(dst, src, ldm, 0); __imma_m16n16k16_ld_c(dst, src, ldm, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.store.d.col.stride.s32 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.store.d.col.stride.s32
// expected-error-re@+1 {{'__imma_m16n16k16_st_c_i32' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_st_c_i32' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_st_c_i32(dst, src, ldm, 1); __imma_m16n16k16_st_c_i32(dst, src, ldm, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.store.d.row.stride.s32 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.store.d.row.stride.s32
// expected-error-re@+1 {{'__imma_m16n16k16_st_c_i32' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_st_c_i32' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_st_c_i32(dst, src, ldm, 0); __imma_m16n16k16_st_c_i32(dst, src, ldm, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.a.col.stride.s8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.a.col.stride.s8
// expected-error-re@+1 {{'__imma_m32n8k16_ld_a_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_ld_a_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_ld_a_s8(dst, src, ldm, 1); __imma_m32n8k16_ld_a_s8(dst, src, ldm, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.a.row.stride.s8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.a.row.stride.s8
// expected-error-re@+1 {{'__imma_m32n8k16_ld_a_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_ld_a_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_ld_a_s8(dst, src, ldm, 0); __imma_m32n8k16_ld_a_s8(dst, src, ldm, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.a.col.stride.u8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.a.col.stride.u8
// expected-error-re@+1 {{'__imma_m32n8k16_ld_a_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_ld_a_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_ld_a_u8(dst, src, ldm, 1); __imma_m32n8k16_ld_a_u8(dst, src, ldm, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.a.row.stride.u8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.a.row.stride.u8
// expected-error-re@+1 {{'__imma_m32n8k16_ld_a_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_ld_a_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_ld_a_u8(dst, src, ldm, 0); __imma_m32n8k16_ld_a_u8(dst, src, ldm, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.b.col.stride.s8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.b.col.stride.s8
// expected-error-re@+1 {{'__imma_m32n8k16_ld_b_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_ld_b_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_ld_b_s8(dst, src, ldm, 1); __imma_m32n8k16_ld_b_s8(dst, src, ldm, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.b.row.stride.s8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.b.row.stride.s8
// expected-error-re@+1 {{'__imma_m32n8k16_ld_b_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_ld_b_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_ld_b_s8(dst, src, ldm, 0); __imma_m32n8k16_ld_b_s8(dst, src, ldm, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.b.col.stride.u8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.b.col.stride.u8
// expected-error-re@+1 {{'__imma_m32n8k16_ld_b_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_ld_b_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_ld_b_u8(dst, src, ldm, 1); __imma_m32n8k16_ld_b_u8(dst, src, ldm, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.b.row.stride.u8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.b.row.stride.u8
// expected-error-re@+1 {{'__imma_m32n8k16_ld_b_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_ld_b_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_ld_b_u8(dst, src, ldm, 0); __imma_m32n8k16_ld_b_u8(dst, src, ldm, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.c.col.stride.s32 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.c.col.stride.s32
// expected-error-re@+1 {{'__imma_m32n8k16_ld_c' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_ld_c' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_ld_c(dst, src, ldm, 1); __imma_m32n8k16_ld_c(dst, src, ldm, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.c.row.stride.s32 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.c.row.stride.s32
// expected-error-re@+1 {{'__imma_m32n8k16_ld_c' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_ld_c' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_ld_c(dst, src, ldm, 0); __imma_m32n8k16_ld_c(dst, src, ldm, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.store.d.col.stride.s32 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.store.d.col.stride.s32
// expected-error-re@+1 {{'__imma_m32n8k16_st_c_i32' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_st_c_i32' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_st_c_i32(dst, src, ldm, 1); __imma_m32n8k16_st_c_i32(dst, src, ldm, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.store.d.row.stride.s32 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.store.d.row.stride.s32
// expected-error-re@+1 {{'__imma_m32n8k16_st_c_i32' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_st_c_i32' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_st_c_i32(dst, src, ldm, 0); __imma_m32n8k16_st_c_i32(dst, src, ldm, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.a.col.stride.s8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.a.col.stride.s8
// expected-error-re@+1 {{'__imma_m8n32k16_ld_a_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_ld_a_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_ld_a_s8(dst, src, ldm, 1); __imma_m8n32k16_ld_a_s8(dst, src, ldm, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.a.row.stride.s8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.a.row.stride.s8
// expected-error-re@+1 {{'__imma_m8n32k16_ld_a_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_ld_a_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_ld_a_s8(dst, src, ldm, 0); __imma_m8n32k16_ld_a_s8(dst, src, ldm, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.a.col.stride.u8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.a.col.stride.u8
// expected-error-re@+1 {{'__imma_m8n32k16_ld_a_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_ld_a_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_ld_a_u8(dst, src, ldm, 1); __imma_m8n32k16_ld_a_u8(dst, src, ldm, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.a.row.stride.u8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.a.row.stride.u8
// expected-error-re@+1 {{'__imma_m8n32k16_ld_a_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_ld_a_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_ld_a_u8(dst, src, ldm, 0); __imma_m8n32k16_ld_a_u8(dst, src, ldm, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.b.col.stride.s8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.b.col.stride.s8
// expected-error-re@+1 {{'__imma_m8n32k16_ld_b_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_ld_b_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_ld_b_s8(dst, src, ldm, 1); __imma_m8n32k16_ld_b_s8(dst, src, ldm, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.b.row.stride.s8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.b.row.stride.s8
// expected-error-re@+1 {{'__imma_m8n32k16_ld_b_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_ld_b_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_ld_b_s8(dst, src, ldm, 0); __imma_m8n32k16_ld_b_s8(dst, src, ldm, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.b.col.stride.u8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.b.col.stride.u8
// expected-error-re@+1 {{'__imma_m8n32k16_ld_b_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_ld_b_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_ld_b_u8(dst, src, ldm, 1); __imma_m8n32k16_ld_b_u8(dst, src, ldm, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.b.row.stride.u8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.b.row.stride.u8
// expected-error-re@+1 {{'__imma_m8n32k16_ld_b_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_ld_b_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_ld_b_u8(dst, src, ldm, 0); __imma_m8n32k16_ld_b_u8(dst, src, ldm, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.c.col.stride.s32 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.c.col.stride.s32
// expected-error-re@+1 {{'__imma_m8n32k16_ld_c' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_ld_c' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_ld_c(dst, src, ldm, 1); __imma_m8n32k16_ld_c(dst, src, ldm, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.c.row.stride.s32 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.c.row.stride.s32
// expected-error-re@+1 {{'__imma_m8n32k16_ld_c' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_ld_c' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_ld_c(dst, src, ldm, 0); __imma_m8n32k16_ld_c(dst, src, ldm, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.store.d.col.stride.s32 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.store.d.col.stride.s32
// expected-error-re@+1 {{'__imma_m8n32k16_st_c_i32' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_st_c_i32' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_st_c_i32(dst, src, ldm, 1); __imma_m8n32k16_st_c_i32(dst, src, ldm, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.store.d.row.stride.s32 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.store.d.row.stride.s32
// expected-error-re@+1 {{'__imma_m8n32k16_st_c_i32' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_st_c_i32' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_st_c_i32(dst, src, ldm, 0); __imma_m8n32k16_st_c_i32(dst, src, ldm, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.s8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.s8
// expected-error-re@+1 {{'__imma_m16n16k16_mma_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_mma_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_mma_s8(dst, src, src, src, 3, 0); __imma_m16n16k16_mma_s8(dst, src, src, src, 3, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.s8.satfinite // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.s8.satfinite
// expected-error-re@+1 {{'__imma_m16n16k16_mma_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_mma_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_mma_s8(dst, src, src, src, 3, 1); __imma_m16n16k16_mma_s8(dst, src, src, src, 3, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.s8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.s8
// expected-error-re@+1 {{'__imma_m16n16k16_mma_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_mma_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_mma_s8(dst, src, src, src, 2, 0); __imma_m16n16k16_mma_s8(dst, src, src, src, 2, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.s8.satfinite // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.s8.satfinite
// expected-error-re@+1 {{'__imma_m16n16k16_mma_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_mma_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_mma_s8(dst, src, src, src, 2, 1); __imma_m16n16k16_mma_s8(dst, src, src, src, 2, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.s8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.s8
// expected-error-re@+1 {{'__imma_m16n16k16_mma_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_mma_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_mma_s8(dst, src, src, src, 1, 0); __imma_m16n16k16_mma_s8(dst, src, src, src, 1, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.s8.satfinite // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.s8.satfinite
// expected-error-re@+1 {{'__imma_m16n16k16_mma_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_mma_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_mma_s8(dst, src, src, src, 1, 1); __imma_m16n16k16_mma_s8(dst, src, src, src, 1, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.s8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.s8
// expected-error-re@+1 {{'__imma_m16n16k16_mma_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_mma_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_mma_s8(dst, src, src, src, 0, 0); __imma_m16n16k16_mma_s8(dst, src, src, src, 0, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.s8.satfinite // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.s8.satfinite
// expected-error-re@+1 {{'__imma_m16n16k16_mma_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_mma_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_mma_s8(dst, src, src, src, 0, 1); __imma_m16n16k16_mma_s8(dst, src, src, src, 0, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.u8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.u8
// expected-error-re@+1 {{'__imma_m16n16k16_mma_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_mma_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_mma_u8(dst, src, src, src, 3, 0); __imma_m16n16k16_mma_u8(dst, src, src, src, 3, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.u8.satfinite // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.u8.satfinite
// expected-error-re@+1 {{'__imma_m16n16k16_mma_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_mma_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_mma_u8(dst, src, src, src, 3, 1); __imma_m16n16k16_mma_u8(dst, src, src, src, 3, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.u8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.u8
// expected-error-re@+1 {{'__imma_m16n16k16_mma_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_mma_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_mma_u8(dst, src, src, src, 2, 0); __imma_m16n16k16_mma_u8(dst, src, src, src, 2, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.u8.satfinite // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.u8.satfinite
// expected-error-re@+1 {{'__imma_m16n16k16_mma_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_mma_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_mma_u8(dst, src, src, src, 2, 1); __imma_m16n16k16_mma_u8(dst, src, src, src, 2, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.u8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.u8
// expected-error-re@+1 {{'__imma_m16n16k16_mma_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_mma_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_mma_u8(dst, src, src, src, 1, 0); __imma_m16n16k16_mma_u8(dst, src, src, src, 1, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.u8.satfinite // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.u8.satfinite
// expected-error-re@+1 {{'__imma_m16n16k16_mma_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_mma_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_mma_u8(dst, src, src, src, 1, 1); __imma_m16n16k16_mma_u8(dst, src, src, src, 1, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.u8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.u8
// expected-error-re@+1 {{'__imma_m16n16k16_mma_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_mma_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_mma_u8(dst, src, src, src, 0, 0); __imma_m16n16k16_mma_u8(dst, src, src, src, 0, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.u8.satfinite // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.u8.satfinite
// expected-error-re@+1 {{'__imma_m16n16k16_mma_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m16n16k16_mma_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m16n16k16_mma_u8(dst, src, src, src, 0, 1); __imma_m16n16k16_mma_u8(dst, src, src, src, 0, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.s8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.s8
// expected-error-re@+1 {{'__imma_m32n8k16_mma_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_mma_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_mma_s8(dst, src, src, src, 3, 0); __imma_m32n8k16_mma_s8(dst, src, src, src, 3, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.s8.satfinite // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.s8.satfinite
// expected-error-re@+1 {{'__imma_m32n8k16_mma_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_mma_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_mma_s8(dst, src, src, src, 3, 1); __imma_m32n8k16_mma_s8(dst, src, src, src, 3, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.s8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.s8
// expected-error-re@+1 {{'__imma_m32n8k16_mma_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_mma_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_mma_s8(dst, src, src, src, 2, 0); __imma_m32n8k16_mma_s8(dst, src, src, src, 2, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.s8.satfinite // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.s8.satfinite
// expected-error-re@+1 {{'__imma_m32n8k16_mma_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_mma_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_mma_s8(dst, src, src, src, 2, 1); __imma_m32n8k16_mma_s8(dst, src, src, src, 2, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.s8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.s8
// expected-error-re@+1 {{'__imma_m32n8k16_mma_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_mma_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_mma_s8(dst, src, src, src, 1, 0); __imma_m32n8k16_mma_s8(dst, src, src, src, 1, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.s8.satfinite // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.s8.satfinite
// expected-error-re@+1 {{'__imma_m32n8k16_mma_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_mma_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_mma_s8(dst, src, src, src, 1, 1); __imma_m32n8k16_mma_s8(dst, src, src, src, 1, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.s8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.s8
// expected-error-re@+1 {{'__imma_m32n8k16_mma_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_mma_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_mma_s8(dst, src, src, src, 0, 0); __imma_m32n8k16_mma_s8(dst, src, src, src, 0, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.s8.satfinite // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.s8.satfinite
// expected-error-re@+1 {{'__imma_m32n8k16_mma_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_mma_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_mma_s8(dst, src, src, src, 0, 1); __imma_m32n8k16_mma_s8(dst, src, src, src, 0, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.u8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.u8
// expected-error-re@+1 {{'__imma_m32n8k16_mma_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_mma_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_mma_u8(dst, src, src, src, 3, 0); __imma_m32n8k16_mma_u8(dst, src, src, src, 3, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.u8.satfinite // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.u8.satfinite
// expected-error-re@+1 {{'__imma_m32n8k16_mma_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_mma_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_mma_u8(dst, src, src, src, 3, 1); __imma_m32n8k16_mma_u8(dst, src, src, src, 3, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.u8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.u8
// expected-error-re@+1 {{'__imma_m32n8k16_mma_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_mma_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_mma_u8(dst, src, src, src, 2, 0); __imma_m32n8k16_mma_u8(dst, src, src, src, 2, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.u8.satfinite // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.u8.satfinite
// expected-error-re@+1 {{'__imma_m32n8k16_mma_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_mma_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_mma_u8(dst, src, src, src, 2, 1); __imma_m32n8k16_mma_u8(dst, src, src, src, 2, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.u8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.u8
// expected-error-re@+1 {{'__imma_m32n8k16_mma_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_mma_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_mma_u8(dst, src, src, src, 1, 0); __imma_m32n8k16_mma_u8(dst, src, src, src, 1, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.u8.satfinite // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.u8.satfinite
// expected-error-re@+1 {{'__imma_m32n8k16_mma_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_mma_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_mma_u8(dst, src, src, src, 1, 1); __imma_m32n8k16_mma_u8(dst, src, src, src, 1, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.u8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.u8
// expected-error-re@+1 {{'__imma_m32n8k16_mma_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_mma_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_mma_u8(dst, src, src, src, 0, 0); __imma_m32n8k16_mma_u8(dst, src, src, src, 0, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.u8.satfinite // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.u8.satfinite
// expected-error-re@+1 {{'__imma_m32n8k16_mma_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m32n8k16_mma_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m32n8k16_mma_u8(dst, src, src, src, 0, 1); __imma_m32n8k16_mma_u8(dst, src, src, src, 0, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.s8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.s8
// expected-error-re@+1 {{'__imma_m8n32k16_mma_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_mma_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_mma_s8(dst, src, src, src, 3, 0); __imma_m8n32k16_mma_s8(dst, src, src, src, 3, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.s8.satfinite // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.s8.satfinite
// expected-error-re@+1 {{'__imma_m8n32k16_mma_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_mma_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_mma_s8(dst, src, src, src, 3, 1); __imma_m8n32k16_mma_s8(dst, src, src, src, 3, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.s8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.s8
// expected-error-re@+1 {{'__imma_m8n32k16_mma_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_mma_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_mma_s8(dst, src, src, src, 2, 0); __imma_m8n32k16_mma_s8(dst, src, src, src, 2, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.s8.satfinite // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.s8.satfinite
// expected-error-re@+1 {{'__imma_m8n32k16_mma_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_mma_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_mma_s8(dst, src, src, src, 2, 1); __imma_m8n32k16_mma_s8(dst, src, src, src, 2, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.s8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.s8
// expected-error-re@+1 {{'__imma_m8n32k16_mma_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_mma_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_mma_s8(dst, src, src, src, 1, 0); __imma_m8n32k16_mma_s8(dst, src, src, src, 1, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.s8.satfinite // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.s8.satfinite
// expected-error-re@+1 {{'__imma_m8n32k16_mma_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_mma_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_mma_s8(dst, src, src, src, 1, 1); __imma_m8n32k16_mma_s8(dst, src, src, src, 1, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.s8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.s8
// expected-error-re@+1 {{'__imma_m8n32k16_mma_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_mma_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_mma_s8(dst, src, src, src, 0, 0); __imma_m8n32k16_mma_s8(dst, src, src, src, 0, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.s8.satfinite // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.s8.satfinite
// expected-error-re@+1 {{'__imma_m8n32k16_mma_s8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_mma_s8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_mma_s8(dst, src, src, src, 0, 1); __imma_m8n32k16_mma_s8(dst, src, src, src, 0, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.u8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.u8
// expected-error-re@+1 {{'__imma_m8n32k16_mma_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_mma_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_mma_u8(dst, src, src, src, 3, 0); __imma_m8n32k16_mma_u8(dst, src, src, src, 3, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.u8.satfinite // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.u8.satfinite
// expected-error-re@+1 {{'__imma_m8n32k16_mma_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_mma_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_mma_u8(dst, src, src, src, 3, 1); __imma_m8n32k16_mma_u8(dst, src, src, src, 3, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.u8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.u8
// expected-error-re@+1 {{'__imma_m8n32k16_mma_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_mma_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_mma_u8(dst, src, src, src, 2, 0); __imma_m8n32k16_mma_u8(dst, src, src, src, 2, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.u8.satfinite // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.u8.satfinite
// expected-error-re@+1 {{'__imma_m8n32k16_mma_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_mma_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_mma_u8(dst, src, src, src, 2, 1); __imma_m8n32k16_mma_u8(dst, src, src, src, 2, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.u8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.u8
// expected-error-re@+1 {{'__imma_m8n32k16_mma_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_mma_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_mma_u8(dst, src, src, src, 1, 0); __imma_m8n32k16_mma_u8(dst, src, src, src, 1, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.u8.satfinite // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.u8.satfinite
// expected-error-re@+1 {{'__imma_m8n32k16_mma_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_mma_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_mma_u8(dst, src, src, src, 1, 1); __imma_m8n32k16_mma_u8(dst, src, src, src, 1, 1);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.u8 // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.u8
// expected-error-re@+1 {{'__imma_m8n32k16_mma_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_mma_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_mma_u8(dst, src, src, src, 0, 0); __imma_m8n32k16_mma_u8(dst, src, src, src, 0, 0);
// CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.u8.satfinite // CHECK_PTX63_SM72: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.u8.satfinite
// expected-error-re@+1 {{'__imma_m8n32k16_mma_u8' needs target feature sm_72{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n32k16_mma_u8' needs target feature (sm_72{{.*}},(ptx63{{.*}}}}
__imma_m8n32k16_mma_u8(dst, src, src, src, 0, 1); __imma_m8n32k16_mma_u8(dst, src, src, src, 0, 1);
#endif // (PTX >= 63) && (SM >= 72) #endif // (PTX >= 63) && (SM >= 72)
#if (PTX >= 63) && (SM >= 75) #if (PTX >= 63) && (SM >= 75)
// CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k128.load.a.row.stride.b1 // CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k128.load.a.row.stride.b1
// expected-error-re@+1 {{'__bmma_m8n8k128_ld_a_b1' needs target feature sm_75{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__bmma_m8n8k128_ld_a_b1' needs target feature (sm_75{{.*}},(ptx63{{.*}}}}
__bmma_m8n8k128_ld_a_b1(dst, src, ldm, 0); __bmma_m8n8k128_ld_a_b1(dst, src, ldm, 0);
// CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k128.load.b.col.stride.b1 // CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k128.load.b.col.stride.b1
// expected-error-re@+1 {{'__bmma_m8n8k128_ld_b_b1' needs target feature sm_75{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__bmma_m8n8k128_ld_b_b1' needs target feature (sm_75{{.*}},(ptx63{{.*}}}}
__bmma_m8n8k128_ld_b_b1(dst, src, ldm, 1); __bmma_m8n8k128_ld_b_b1(dst, src, ldm, 1);
// CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k128.load.c.col.stride.s32 // CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k128.load.c.col.stride.s32
// expected-error-re@+1 {{'__bmma_m8n8k128_ld_c' needs target feature sm_75{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__bmma_m8n8k128_ld_c' needs target feature (sm_75{{.*}},(ptx63{{.*}}}}
__bmma_m8n8k128_ld_c(dst, src, ldm, 1); __bmma_m8n8k128_ld_c(dst, src, ldm, 1);
// CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k128.load.c.row.stride.s32 // CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k128.load.c.row.stride.s32
// expected-error-re@+1 {{'__bmma_m8n8k128_ld_c' needs target feature sm_75{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__bmma_m8n8k128_ld_c' needs target feature (sm_75{{.*}},(ptx63{{.*}}}}
__bmma_m8n8k128_ld_c(dst, src, ldm, 0); __bmma_m8n8k128_ld_c(dst, src, ldm, 0);
// CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k128.store.d.col.stride.s32 // CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k128.store.d.col.stride.s32
// expected-error-re@+1 {{'__bmma_m8n8k128_st_c_i32' needs target feature sm_75{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__bmma_m8n8k128_st_c_i32' needs target feature (sm_75{{.*}},(ptx63{{.*}}}}
__bmma_m8n8k128_st_c_i32(dst, src, ldm, 1); __bmma_m8n8k128_st_c_i32(dst, src, ldm, 1);
// CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k128.store.d.row.stride.s32 // CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k128.store.d.row.stride.s32
// expected-error-re@+1 {{'__bmma_m8n8k128_st_c_i32' needs target feature sm_75{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__bmma_m8n8k128_st_c_i32' needs target feature (sm_75{{.*}},(ptx63{{.*}}}}
__bmma_m8n8k128_st_c_i32(dst, src, ldm, 0); __bmma_m8n8k128_st_c_i32(dst, src, ldm, 0);
// CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k32.load.a.row.stride.s4 // CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k32.load.a.row.stride.s4
// expected-error-re@+1 {{'__imma_m8n8k32_ld_a_s4' needs target feature sm_75{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n8k32_ld_a_s4' needs target feature (sm_75{{.*}},(ptx63{{.*}}}}
__imma_m8n8k32_ld_a_s4(dst, src, ldm, 0); __imma_m8n8k32_ld_a_s4(dst, src, ldm, 0);
// CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k32.load.a.row.stride.u4 // CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k32.load.a.row.stride.u4
// expected-error-re@+1 {{'__imma_m8n8k32_ld_a_u4' needs target feature sm_75{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n8k32_ld_a_u4' needs target feature (sm_75{{.*}},(ptx63{{.*}}}}
__imma_m8n8k32_ld_a_u4(dst, src, ldm, 0); __imma_m8n8k32_ld_a_u4(dst, src, ldm, 0);
// CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k32.load.b.col.stride.s4 // CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k32.load.b.col.stride.s4
// expected-error-re@+1 {{'__imma_m8n8k32_ld_b_s4' needs target feature sm_75{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n8k32_ld_b_s4' needs target feature (sm_75{{.*}},(ptx63{{.*}}}}
__imma_m8n8k32_ld_b_s4(dst, src, ldm, 1); __imma_m8n8k32_ld_b_s4(dst, src, ldm, 1);
// CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k32.load.b.col.stride.u4 // CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k32.load.b.col.stride.u4
// expected-error-re@+1 {{'__imma_m8n8k32_ld_b_u4' needs target feature sm_75{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n8k32_ld_b_u4' needs target feature (sm_75{{.*}},(ptx63{{.*}}}}
__imma_m8n8k32_ld_b_u4(dst, src, ldm, 1); __imma_m8n8k32_ld_b_u4(dst, src, ldm, 1);
// CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k32.load.c.col.stride.s32 // CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k32.load.c.col.stride.s32
// expected-error-re@+1 {{'__imma_m8n8k32_ld_c' needs target feature sm_75{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n8k32_ld_c' needs target feature (sm_75{{.*}},(ptx63{{.*}}}}
__imma_m8n8k32_ld_c(dst, src, ldm, 1); __imma_m8n8k32_ld_c(dst, src, ldm, 1);
// CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k32.load.c.row.stride.s32 // CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k32.load.c.row.stride.s32
// expected-error-re@+1 {{'__imma_m8n8k32_ld_c' needs target feature sm_75{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n8k32_ld_c' needs target feature (sm_75{{.*}},(ptx63{{.*}}}}
__imma_m8n8k32_ld_c(dst, src, ldm, 0); __imma_m8n8k32_ld_c(dst, src, ldm, 0);
// CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k32.store.d.col.stride.s32 // CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k32.store.d.col.stride.s32
// expected-error-re@+1 {{'__imma_m8n8k32_st_c_i32' needs target feature sm_75{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n8k32_st_c_i32' needs target feature (sm_75{{.*}},(ptx63{{.*}}}}
__imma_m8n8k32_st_c_i32(dst, src, ldm, 1); __imma_m8n8k32_st_c_i32(dst, src, ldm, 1);
// CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k32.store.d.row.stride.s32 // CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k32.store.d.row.stride.s32
// expected-error-re@+1 {{'__imma_m8n8k32_st_c_i32' needs target feature sm_75{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n8k32_st_c_i32' needs target feature (sm_75{{.*}},(ptx63{{.*}}}}
__imma_m8n8k32_st_c_i32(dst, src, ldm, 0); __imma_m8n8k32_st_c_i32(dst, src, ldm, 0);
// CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k128.mma.row.col.b1 // CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k128.mma.row.col.b1
// expected-error-re@+1 {{'__bmma_m8n8k128_mma_xor_popc_b1' needs target feature sm_75{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__bmma_m8n8k128_mma_xor_popc_b1' needs target feature (sm_75{{.*}},(ptx63{{.*}}}}
__bmma_m8n8k128_mma_xor_popc_b1(dst, src, src, src, 1); __bmma_m8n8k128_mma_xor_popc_b1(dst, src, src, src, 1);
// CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k32.mma.row.col.s4 // CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k32.mma.row.col.s4
// expected-error-re@+1 {{'__imma_m8n8k32_mma_s4' needs target feature sm_75{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n8k32_mma_s4' needs target feature (sm_75{{.*}},(ptx63{{.*}}}}
__imma_m8n8k32_mma_s4(dst, src, src, src, 1, 0); __imma_m8n8k32_mma_s4(dst, src, src, src, 1, 0);
// CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k32.mma.row.col.s4.satfinite // CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k32.mma.row.col.s4.satfinite
// expected-error-re@+1 {{'__imma_m8n8k32_mma_s4' needs target feature sm_75{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n8k32_mma_s4' needs target feature (sm_75{{.*}},(ptx63{{.*}}}}
__imma_m8n8k32_mma_s4(dst, src, src, src, 1, 1); __imma_m8n8k32_mma_s4(dst, src, src, src, 1, 1);
// CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k32.mma.row.col.u4 // CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k32.mma.row.col.u4
// expected-error-re@+1 {{'__imma_m8n8k32_mma_u4' needs target feature sm_75{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n8k32_mma_u4' needs target feature (sm_75{{.*}},(ptx63{{.*}}}}
__imma_m8n8k32_mma_u4(dst, src, src, src, 1, 0); __imma_m8n8k32_mma_u4(dst, src, src, src, 1, 0);
// CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k32.mma.row.col.u4.satfinite // CHECK_PTX63_SM75: call {{.*}} @llvm.nvvm.wmma.m8n8k32.mma.row.col.u4.satfinite
// expected-error-re@+1 {{'__imma_m8n8k32_mma_u4' needs target feature sm_75{{.*}},ptx63{{.*}}}} // expected-error-re@+1 {{'__imma_m8n8k32_mma_u4' needs target feature (sm_75{{.*}},(ptx63{{.*}}}}
__imma_m8n8k32_mma_u4(dst, src, src, src, 1, 1); __imma_m8n8k32_mma_u4(dst, src, src, src, 1, 1);
#endif // (PTX >= 63) && (SM >= 75) #endif // (PTX >= 63) && (SM >= 75)
} }

clang/test/CodeGen/builtins-nvptx-sm_70.cu

Show All 24 Lines
// that encounters an error, so -verify will not be able to find errors in // that encounters an error, so -verify will not be able to find errors in
// subsequent functions. // subsequent functions.
// CHECK-LABEL: nvvm_wmma_m16n16k16 // CHECK-LABEL: nvvm_wmma_m16n16k16
__device__ void nvvm_wmma_m16n16k16(int *src, int *dst, __device__ void nvvm_wmma_m16n16k16(int *src, int *dst,
float *fsrc, float *fdst, float *fsrc, float *fdst,
int ldm) { int ldm) {
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.a.row.stride.f16 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.a.row.stride.f16
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_ld_a' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_ld_a' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_ld_a(dst, src, ldm, 0); __hmma_m16n16k16_ld_a(dst, src, ldm, 0);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.a.col.stride.f16 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.a.col.stride.f16
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_ld_a' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_ld_a' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_ld_a(dst, src+1, ldm, 1); __hmma_m16n16k16_ld_a(dst, src+1, ldm, 1);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.b.row.stride.f16 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.b.row.stride.f16
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_ld_b' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_ld_b' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_ld_b(dst, src, ldm, 0); __hmma_m16n16k16_ld_b(dst, src, ldm, 0);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.b.col.stride.f16 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.b.col.stride.f16
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_ld_b' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_ld_b' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_ld_b(dst, src+2, ldm, 1); __hmma_m16n16k16_ld_b(dst, src+2, ldm, 1);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.c.row.stride.f16 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.c.row.stride.f16
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_ld_c_f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_ld_c_f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_ld_c_f16(dst, src, ldm, 0); __hmma_m16n16k16_ld_c_f16(dst, src, ldm, 0);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.c.col.stride.f16 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.c.col.stride.f16
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_ld_c_f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_ld_c_f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_ld_c_f16(dst, src, ldm, 1); __hmma_m16n16k16_ld_c_f16(dst, src, ldm, 1);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.c.row.stride.f32 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.c.row.stride.f32
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_ld_c_f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_ld_c_f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_ld_c_f32(fdst, fsrc, ldm, 0); __hmma_m16n16k16_ld_c_f32(fdst, fsrc, ldm, 0);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.c.col.stride.f32 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.load.c.col.stride.f32
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_ld_c_f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_ld_c_f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_ld_c_f32(fdst, fsrc, ldm, 1); __hmma_m16n16k16_ld_c_f32(fdst, fsrc, ldm, 1);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.store.d.row.stride.f16 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.store.d.row.stride.f16
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_st_c_f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_st_c_f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_st_c_f16(dst, src, ldm, 0); __hmma_m16n16k16_st_c_f16(dst, src, ldm, 0);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.store.d.col.stride.f16 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.store.d.col.stride.f16
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_st_c_f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_st_c_f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_st_c_f16(dst, src, ldm, 1); __hmma_m16n16k16_st_c_f16(dst, src, ldm, 1);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.store.d.row.stride.f32 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.store.d.row.stride.f32
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_st_c_f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_st_c_f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_st_c_f32(fdst, fsrc, ldm, 0); __hmma_m16n16k16_st_c_f32(fdst, fsrc, ldm, 0);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.store.d.col.stride.f32 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.store.d.col.stride.f32
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_st_c_f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_st_c_f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_st_c_f32(fdst, fsrc, ldm, 1); __hmma_m16n16k16_st_c_f32(fdst, fsrc, ldm, 1);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f16.f16 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f16.f16
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f16(dst, src, src, src, 0, 0); __hmma_m16n16k16_mma_f16f16(dst, src, src, src, 0, 0);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f16.f16.satfinite // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f16.f16.satfinite
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f16(dst, src, src, src, 0, 1); __hmma_m16n16k16_mma_f16f16(dst, src, src, src, 0, 1);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f16.f16 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f16.f16
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f16(dst, src, src, src, 1, 0); __hmma_m16n16k16_mma_f16f16(dst, src, src, src, 1, 0);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f16.f16.satfinite // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f16.f16.satfinite
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f16(dst, src, src, src, 1, 1); __hmma_m16n16k16_mma_f16f16(dst, src, src, src, 1, 1);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f16.f16 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f16.f16
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f16(dst, src, src, src, 2, 0); __hmma_m16n16k16_mma_f16f16(dst, src, src, src, 2, 0);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f16.f16.satfinite // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f16.f16.satfinite
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f16(dst, src, src, src, 2, 1); __hmma_m16n16k16_mma_f16f16(dst, src, src, src, 2, 1);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f16.f16 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f16.f16
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f16(dst, src, src, src, 3, 0); __hmma_m16n16k16_mma_f16f16(dst, src, src, src, 3, 0);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f16.f16.satfinite // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f16.f16.satfinite
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f16(dst, src, src, src, 3, 1); __hmma_m16n16k16_mma_f16f16(dst, src, src, src, 3, 1);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f16.f32 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f16.f32
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 0, 0); __hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 0, 0);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f16.f32.satfinite // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f16.f32.satfinite
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 0, 1); __hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 0, 1);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f16.f32 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f16.f32
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 1, 0); __hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 1, 0);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f16.f32.satfinite // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f16.f32.satfinite
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 1, 1); __hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 1, 1);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f16.f32 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f16.f32
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 2, 0); __hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 2, 0);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f16.f32.satfinite // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f16.f32.satfinite
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 2, 1); __hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 2, 1);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f16.f32 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f16.f32
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 3, 0); __hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 3, 0);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f16.f32.satfinite // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f16.f32.satfinite
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 3, 1); __hmma_m16n16k16_mma_f16f32(dst, src, src, fsrc, 3, 1);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f32.f16 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f32.f16
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 0, 0); __hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 0, 0);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f32.f16.satfinite // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f32.f16.satfinite
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 0, 1); __hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 0, 1);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f32.f16 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f32.f16
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 1, 0); __hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 1, 0);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f32.f16.satfinite // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f32.f16.satfinite
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 1, 1); __hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 1, 1);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f32.f16 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f32.f16
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 2, 0); __hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 2, 0);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f32.f16.satfinite // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f32.f16.satfinite
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 2, 1); __hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 2, 1);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f32.f16 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f32.f16
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 3, 0); __hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 3, 0);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f32.f16.satfinite // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f32.f16.satfinite
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 3, 1); __hmma_m16n16k16_mma_f32f16(fdst, src, src, src, 3, 1);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f32.f32 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f32.f32
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 0, 0); __hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 0, 0);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f32.f32.satfinite // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.row.f32.f32.satfinite
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 0, 1); __hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 0, 1);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f32.f32 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f32.f32
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 1, 0); __hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 1, 0);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f32.f32.satfinite // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.row.col.f32.f32.satfinite
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 1, 1); __hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 1, 1);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f32.f32 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f32.f32
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 2, 0); __hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 2, 0);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f32.f32.satfinite // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.row.f32.f32.satfinite
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 2, 1); __hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 2, 1);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f32.f32 // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f32.f32
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 3, 0); __hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 3, 0);
// CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f32.f32.satfinite // CHECK_M16: call {{.*}} @llvm.nvvm.wmma.m16n16k16.mma.col.col.f32.f32.satfinite
// pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature sm_70{{.*}},ptx60{{.*}}}} // pre-sm_70-error-re@+1 {{'__hmma_m16n16k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx60{{.*}}}}
__hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 3, 1); __hmma_m16n16k16_mma_f32f32(fdst, src, src, fsrc, 3, 1);
} }
#ifdef PTX61 #ifdef PTX61
// CHECK-LABEL: nvvm_wmma_m32n8k16 // CHECK-LABEL: nvvm_wmma_m32n8k16
__device__ void nvvm_wmma_m32n8k16(int *src, int *dst, __device__ void nvvm_wmma_m32n8k16(int *src, int *dst,
float *fsrc, float *fdst, float *fsrc, float *fdst,
int ldm) { int ldm) {
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.a.row.stride.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.a.row.stride.f16
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_ld_a' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_ld_a' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_ld_a(dst, src, ldm, 0); __hmma_m32n8k16_ld_a(dst, src, ldm, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.a.col.stride.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.a.col.stride.f16
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_ld_a' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_ld_a' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_ld_a(dst, src+1, ldm, 1); __hmma_m32n8k16_ld_a(dst, src+1, ldm, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.b.row.stride.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.b.row.stride.f16
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_ld_b' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_ld_b' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_ld_b(dst, src, ldm, 0); __hmma_m32n8k16_ld_b(dst, src, ldm, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.b.col.stride.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.b.col.stride.f16
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_ld_b' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_ld_b' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_ld_b(dst, src+2, ldm, 1); __hmma_m32n8k16_ld_b(dst, src+2, ldm, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.c.row.stride.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.c.row.stride.f16
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_ld_c_f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_ld_c_f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_ld_c_f16(dst, src, ldm, 0); __hmma_m32n8k16_ld_c_f16(dst, src, ldm, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.c.col.stride.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.c.col.stride.f16
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_ld_c_f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_ld_c_f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_ld_c_f16(dst, src, ldm, 1); __hmma_m32n8k16_ld_c_f16(dst, src, ldm, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.c.row.stride.f32 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.c.row.stride.f32
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_ld_c_f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_ld_c_f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_ld_c_f32(fdst, fsrc, ldm, 0); __hmma_m32n8k16_ld_c_f32(fdst, fsrc, ldm, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.c.col.stride.f32 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.load.c.col.stride.f32
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_ld_c_f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_ld_c_f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_ld_c_f32(fdst, fsrc, ldm, 1); __hmma_m32n8k16_ld_c_f32(fdst, fsrc, ldm, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.store.d.row.stride.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.store.d.row.stride.f16
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_st_c_f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_st_c_f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_st_c_f16(dst, src, ldm, 0); __hmma_m32n8k16_st_c_f16(dst, src, ldm, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.store.d.col.stride.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.store.d.col.stride.f16
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_st_c_f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_st_c_f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_st_c_f16(dst, src, ldm, 1); __hmma_m32n8k16_st_c_f16(dst, src, ldm, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.store.d.row.stride.f32 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.store.d.row.stride.f32
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_st_c_f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_st_c_f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_st_c_f32(fdst, fsrc, ldm, 0); __hmma_m32n8k16_st_c_f32(fdst, fsrc, ldm, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.store.d.col.stride.f32 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.store.d.col.stride.f32
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_st_c_f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_st_c_f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_st_c_f32(fdst, fsrc, ldm, 1); __hmma_m32n8k16_st_c_f32(fdst, fsrc, ldm, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f16.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f16.f16
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f16(dst, src, src, src, 0, 0); __hmma_m32n8k16_mma_f16f16(dst, src, src, src, 0, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f16.f16.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f16.f16.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f16(dst, src, src, src, 0, 1); __hmma_m32n8k16_mma_f16f16(dst, src, src, src, 0, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f16.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f16.f16
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f16(dst, src, src, src, 1, 0); __hmma_m32n8k16_mma_f16f16(dst, src, src, src, 1, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f16.f16.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f16.f16.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f16(dst, src, src, src, 1, 1); __hmma_m32n8k16_mma_f16f16(dst, src, src, src, 1, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f16.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f16.f16
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f16(dst, src, src, src, 2, 0); __hmma_m32n8k16_mma_f16f16(dst, src, src, src, 2, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f16.f16.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f16.f16.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f16(dst, src, src, src, 2, 1); __hmma_m32n8k16_mma_f16f16(dst, src, src, src, 2, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f16.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f16.f16
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f16(dst, src, src, src, 3, 0); __hmma_m32n8k16_mma_f16f16(dst, src, src, src, 3, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f16.f16.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f16.f16.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f16(dst, src, src, src, 3, 1); __hmma_m32n8k16_mma_f16f16(dst, src, src, src, 3, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f16.f32 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f16.f32
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 0, 0); __hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 0, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f16.f32.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f16.f32.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 0, 1); __hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 0, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f16.f32 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f16.f32
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 1, 0); __hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 1, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f16.f32.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f16.f32.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 1, 1); __hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 1, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f16.f32 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f16.f32
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 2, 0); __hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 2, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f16.f32.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f16.f32.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 2, 1); __hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 2, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f16.f32 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f16.f32
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 3, 0); __hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 3, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f16.f32.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f16.f32.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 3, 1); __hmma_m32n8k16_mma_f16f32(dst, src, src, fsrc, 3, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f32.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f32.f16
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 0, 0); __hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 0, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f32.f16.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f32.f16.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 0, 1); __hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 0, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f32.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f32.f16
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 1, 0); __hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 1, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f32.f16.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f32.f16.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 1, 1); __hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 1, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f32.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f32.f16
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 2, 0); __hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 2, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f32.f16.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f32.f16.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 2, 1); __hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 2, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f32.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f32.f16
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 3, 0); __hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 3, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f32.f16.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f32.f16.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 3, 1); __hmma_m32n8k16_mma_f32f16(fdst, src, src, src, 3, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f32.f32 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f32.f32
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 0, 0); __hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 0, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f32.f32.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.row.f32.f32.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 0, 1); __hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 0, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f32.f32 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f32.f32
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 1, 0); __hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 1, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f32.f32.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.row.col.f32.f32.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 1, 1); __hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 1, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f32.f32 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f32.f32
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 2, 0); __hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 2, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f32.f32.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.row.f32.f32.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 2, 1); __hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 2, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f32.f32 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f32.f32
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 3, 0); __hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 3, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f32.f32.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m32n8k16.mma.col.col.f32.f32.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m32n8k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 3, 1); __hmma_m32n8k16_mma_f32f32(fdst, src, src, fsrc, 3, 1);
// m8n32k16 variants. // m8n32k16 variants.
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.a.row.stride.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.a.row.stride.f16
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_ld_a' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_ld_a' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_ld_a(dst, src, ldm, 0); __hmma_m8n32k16_ld_a(dst, src, ldm, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.a.col.stride.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.a.col.stride.f16
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_ld_a' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_ld_a' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_ld_a(dst, src+1, ldm, 1); __hmma_m8n32k16_ld_a(dst, src+1, ldm, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.b.row.stride.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.b.row.stride.f16
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_ld_b' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_ld_b' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_ld_b(dst, src, ldm, 0); __hmma_m8n32k16_ld_b(dst, src, ldm, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.b.col.stride.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.b.col.stride.f16
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_ld_b' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_ld_b' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_ld_b(dst, src+2, ldm, 1); __hmma_m8n32k16_ld_b(dst, src+2, ldm, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.c.row.stride.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.c.row.stride.f16
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_ld_c_f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_ld_c_f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_ld_c_f16(dst, src, ldm, 0); __hmma_m8n32k16_ld_c_f16(dst, src, ldm, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.c.col.stride.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.c.col.stride.f16
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_ld_c_f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_ld_c_f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_ld_c_f16(dst, src, ldm, 1); __hmma_m8n32k16_ld_c_f16(dst, src, ldm, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.c.row.stride.f32 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.c.row.stride.f32
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_ld_c_f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_ld_c_f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_ld_c_f32(fdst, fsrc, ldm, 0); __hmma_m8n32k16_ld_c_f32(fdst, fsrc, ldm, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.c.col.stride.f32 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.load.c.col.stride.f32
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_ld_c_f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_ld_c_f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_ld_c_f32(fdst, fsrc, ldm, 1); __hmma_m8n32k16_ld_c_f32(fdst, fsrc, ldm, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.store.d.row.stride.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.store.d.row.stride.f16
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_st_c_f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_st_c_f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_st_c_f16(dst, src, ldm, 0); __hmma_m8n32k16_st_c_f16(dst, src, ldm, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.store.d.col.stride.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.store.d.col.stride.f16
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_st_c_f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_st_c_f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_st_c_f16(dst, src, ldm, 1); __hmma_m8n32k16_st_c_f16(dst, src, ldm, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.store.d.row.stride.f32 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.store.d.row.stride.f32
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_st_c_f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_st_c_f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_st_c_f32(fdst, fsrc, ldm, 0); __hmma_m8n32k16_st_c_f32(fdst, fsrc, ldm, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.store.d.col.stride.f32 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.store.d.col.stride.f32
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_st_c_f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_st_c_f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_st_c_f32(fdst, fsrc, ldm, 1); __hmma_m8n32k16_st_c_f32(fdst, fsrc, ldm, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f16.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f16.f16
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f16(dst, src, src, src, 0, 0); __hmma_m8n32k16_mma_f16f16(dst, src, src, src, 0, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f16.f16.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f16.f16.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f16(dst, src, src, src, 0, 1); __hmma_m8n32k16_mma_f16f16(dst, src, src, src, 0, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f16.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f16.f16
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f16(dst, src, src, src, 1, 0); __hmma_m8n32k16_mma_f16f16(dst, src, src, src, 1, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f16.f16.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f16.f16.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f16(dst, src, src, src, 1, 1); __hmma_m8n32k16_mma_f16f16(dst, src, src, src, 1, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f16.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f16.f16
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f16(dst, src, src, src, 2, 0); __hmma_m8n32k16_mma_f16f16(dst, src, src, src, 2, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f16.f16.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f16.f16.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f16(dst, src, src, src, 2, 1); __hmma_m8n32k16_mma_f16f16(dst, src, src, src, 2, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f16.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f16.f16
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f16(dst, src, src, src, 3, 0); __hmma_m8n32k16_mma_f16f16(dst, src, src, src, 3, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f16.f16.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f16.f16.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f16(dst, src, src, src, 3, 1); __hmma_m8n32k16_mma_f16f16(dst, src, src, src, 3, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f16.f32 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f16.f32
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 0, 0); __hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 0, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f16.f32.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f16.f32.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 0, 1); __hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 0, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f16.f32 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f16.f32
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 1, 0); __hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 1, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f16.f32.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f16.f32.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 1, 1); __hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 1, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f16.f32 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f16.f32
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 2, 0); __hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 2, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f16.f32.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f16.f32.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 2, 1); __hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 2, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f16.f32 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f16.f32
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 3, 0); __hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 3, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f16.f32.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f16.f32.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f16f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 3, 1); __hmma_m8n32k16_mma_f16f32(dst, src, src, fsrc, 3, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f32.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f32.f16
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 0, 0); __hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 0, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f32.f16.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f32.f16.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 0, 1); __hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 0, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f32.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f32.f16
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 1, 0); __hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 1, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f32.f16.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f32.f16.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 1, 1); __hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 1, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f32.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f32.f16
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 2, 0); __hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 2, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f32.f16.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f32.f16.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 2, 1); __hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 2, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f32.f16 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f32.f16
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 3, 0); __hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 3, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f32.f16.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f32.f16.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f16' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 3, 1); __hmma_m8n32k16_mma_f32f16(fdst, src, src, src, 3, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f32.f32 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f32.f32
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 0, 0); __hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 0, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f32.f32.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.row.f32.f32.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 0, 1); __hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 0, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f32.f32 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f32.f32
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 1, 0); __hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 1, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f32.f32.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.row.col.f32.f32.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 1, 1); __hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 1, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f32.f32 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f32.f32
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 2, 0); __hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 2, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f32.f32.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.row.f32.f32.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 2, 1); __hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 2, 1);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f32.f32 // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f32.f32
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 3, 0); __hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 3, 0);
// CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f32.f32.satfinite // CHECK_M32_M8: call {{.*}} @llvm.nvvm.wmma.m8n32k16.mma.col.col.f32.f32.satfinite
// pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature sm_70{{.*}},ptx61{{.*}}}} // pre-ptx61-error-re@+1 {{'__hmma_m8n32k16_mma_f32f32' needs target feature (sm_70{{.*}},(ptx61{{.*}}}}
__hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 3, 1); __hmma_m8n32k16_mma_f32f32(fdst, src, src, fsrc, 3, 1);
} }
#endif #endif

clang/test/Driver/x86-target-features.c

Show First 20 Lines • Show All 272 Lines • ▼ Show 20 Lines
// RUN: %clang -target i386-unknown-linux-gnu -march=i386 -mno-amx-bf16 %s -### -o %t.o 2>&1 | FileCheck -check-prefix=NO-AMX-BF16 %s // RUN: %clang -target i386-unknown-linux-gnu -march=i386 -mno-amx-bf16 %s -### -o %t.o 2>&1 | FileCheck -check-prefix=NO-AMX-BF16 %s
// AMX-BF16: "-target-feature" "+amx-bf16" // AMX-BF16: "-target-feature" "+amx-bf16"
// NO-AMX-BF16: "-target-feature" "-amx-bf16" // NO-AMX-BF16: "-target-feature" "-amx-bf16"
// RUN: %clang -target i386-unknown-linux-gnu -march=i386 -mamx-int8 %s -### -o %t.o 2>&1 | FileCheck --check-prefix=AMX-INT8 %s // RUN: %clang -target i386-unknown-linux-gnu -march=i386 -mamx-int8 %s -### -o %t.o 2>&1 | FileCheck --check-prefix=AMX-INT8 %s
// RUN: %clang -target i386-unknown-linux-gnu -march=i386 -mno-amx-int8 %s -### -o %t.o 2>&1 | FileCheck --check-prefix=NO-AMX-INT8 %s // RUN: %clang -target i386-unknown-linux-gnu -march=i386 -mno-amx-int8 %s -### -o %t.o 2>&1 | FileCheck --check-prefix=NO-AMX-INT8 %s
// AMX-INT8: "-target-feature" "+amx-int8" // AMX-INT8: "-target-feature" "+amx-int8"
// NO-AMX-INT8: "-target-feature" "-amx-int8" // NO-AMX-INT8: "-target-feature" "-amx-int8"
// RUN: %clang -target i386-unknown-linux-gnu -march=i386 -mavxvnni %s -### -o %t.o 2>&1 | FileCheck --check-prefix=AVX-VNNI %s
// RUN: %clang -target i386-unknown-linux-gnu -march=i386 -mno-avxvnni %s -### -o %t.o 2>&1 | FileCheck --check-prefix=NO-AVX-VNNI %s
// AVX-VNNI: "-target-feature" "+avxvnni"
// NO-AVX-VNNI: "-target-feature" "-avxvnni"

clang/test/Preprocessor/x86_target_features.c

Show First 20 Lines • Show All 522 Lines • ▼ Show 20 Lines
// RUN: %clang -target i386-unknown-unknown -march=atom -mtsxldtrk -x c -E -dM -o - %s | FileCheck -match-full-lines --check-prefix=TSXLDTRK %s // RUN: %clang -target i386-unknown-unknown -march=atom -mtsxldtrk -x c -E -dM -o - %s | FileCheck -match-full-lines --check-prefix=TSXLDTRK %s
// TSXLDTRK: #define __TSXLDTRK__ 1 // TSXLDTRK: #define __TSXLDTRK__ 1
// RUN: %clang -target i386-unknown-unknown -march=atom -mno-tsxldtrk -x c -E -dM -o - %s | FileCheck -match-full-lines --check-prefix=NOTSXLDTRK %s // RUN: %clang -target i386-unknown-unknown -march=atom -mno-tsxldtrk -x c -E -dM -o - %s | FileCheck -match-full-lines --check-prefix=NOTSXLDTRK %s
// NOTSXLDTRK-NOT: #define __TSXLDTRK__ 1 // NOTSXLDTRK-NOT: #define __TSXLDTRK__ 1
// RUN: %clang -target i386-unknown-unknown -march=atom -mavxvnni -x c -E -dM -o - %s | FileCheck -match-full-lines --check-prefix=AVXVNNI %s
// AVXVNNI: #define __AVX2__ 1
// AVXVNNI: #define __AVXVNNI__ 1
// RUN: %clang -target i386-unknown-unknown -march=atom -mno-avxvnni -x c -E -dM -o - %s | FileCheck -match-full-lines --check-prefix=NOAVXVNNI %s
// NOAVXVNNI-NOT: #define __AVXVNNI__ 1
// RUN: %clang -target i386-unknown-unknown -march=atom -mavxvnni -mno-avx2 -x c -E -dM -o - %s | FileCheck -match-full-lines --check-prefix=AVXVNNINOAVX2 %s
// AVXVNNINOAVX2-NOT: #define __AVX2__ 1
// AVXVNNINOAVX2-NOT: #define __AVXVNNI__ 1

clang/unittests/CodeGen/CMakeLists.txt

set(LLVM_LINK_COMPONENTS set(LLVM_LINK_COMPONENTS
Core Core
Support Support
) )
add_clang_unittest(ClangCodeGenTests add_clang_unittest(ClangCodeGenTests
BufferSourceTest.cpp BufferSourceTest.cpp
CodeGenExternalTest.cpp CodeGenExternalTest.cpp
IncrementalProcessingTest.cpp IncrementalProcessingTest.cpp
TBAAMetadataTest.cpp TBAAMetadataTest.cpp
CheckTargetFeaturesTest.cpp
) )
clang_target_link_libraries(ClangCodeGenTests clang_target_link_libraries(ClangCodeGenTests
PRIVATE PRIVATE
clangAST clangAST
clangBasic clangBasic
clangCodeGen clangCodeGen
clangFrontend clangFrontend
clangLex clangLex
clangParse clangParse
clangSerialization clangSerialization
) )

clang/unittests/CodeGen/CheckTargetFeaturesTest.cpp

  • This file was added.
#include "../lib/CodeGen/CodeGenFunction.h"
#include "gtest/gtest.h"
using namespace llvm;
TEST(CheckTargetFeaturesTest, checkBuiltinFeatures) {
auto doCheck = [](StringRef BuiltinFeatures, StringRef FuncFeatures) {
SmallVector<StringRef, 1> Features;
FuncFeatures.split(Features, ',');
StringMap<bool> SM;
for (StringRef F : Features)
SM.insert(std::make_pair(F, true));
clang::CodeGen::TargetFeatures TF(SM);
return TF.hasRequiredFeatures(BuiltinFeatures);
};
ASSERT_TRUE(doCheck("A|B,C|D", "A"));
ASSERT_FALSE(doCheck("(A|B),(C|D)", "A"));
ASSERT_TRUE(doCheck("(A|B),(C|D)", "A,C"));
ASSERT_FALSE(doCheck("(A,B|C),D", "A,C"));
ASSERT_FALSE(doCheck("(A,B|C),D", "A,D"));
ASSERT_TRUE(doCheck("(A,B|C),D", "C,D"));
ASSERT_TRUE(doCheck("(A,B|C),D", "A,B,D"));
ASSERT_FALSE(doCheck("(A,(B|C)),D", "C,D"));
ASSERT_TRUE(doCheck("(A,(B|C)),D", "A,C,D"));
}

llvm/include/llvm/Support/X86TargetParser.def

Show First 20 Lines • Show All 181 Lines • ▼ Show 20 Lines
X86_FEATURE (VZEROUPPER, "vzeroupper") X86_FEATURE (VZEROUPPER, "vzeroupper")
X86_FEATURE (WAITPKG, "waitpkg") X86_FEATURE (WAITPKG, "waitpkg")
X86_FEATURE (WBNOINVD, "wbnoinvd") X86_FEATURE (WBNOINVD, "wbnoinvd")
X86_FEATURE (X87, "x87") X86_FEATURE (X87, "x87")
X86_FEATURE (XSAVE, "xsave") X86_FEATURE (XSAVE, "xsave")
X86_FEATURE (XSAVEC, "xsavec") X86_FEATURE (XSAVEC, "xsavec")
X86_FEATURE (XSAVEOPT, "xsaveopt") X86_FEATURE (XSAVEOPT, "xsaveopt")
X86_FEATURE (XSAVES, "xsaves") X86_FEATURE (XSAVES, "xsaves")
X86_FEATURE (AVXVNNI, "avxvnni")
// These features aren't really CPU features, but the frontend can set them. // These features aren't really CPU features, but the frontend can set them.
X86_FEATURE (RETPOLINE_EXTERNAL_THUNK, "retpoline-external-thunk") X86_FEATURE (RETPOLINE_EXTERNAL_THUNK, "retpoline-external-thunk")
X86_FEATURE (RETPOLINE_INDIRECT_BRANCHES, "retpoline-indirect-branches") X86_FEATURE (RETPOLINE_INDIRECT_BRANCHES, "retpoline-indirect-branches")
X86_FEATURE (RETPOLINE_INDIRECT_CALLS, "retpoline-indirect-calls") X86_FEATURE (RETPOLINE_INDIRECT_CALLS, "retpoline-indirect-calls")
X86_FEATURE (LVI_CFI, "lvi-cfi") X86_FEATURE (LVI_CFI, "lvi-cfi")
X86_FEATURE (LVI_LOAD_HARDENING, "lvi-load-hardening") X86_FEATURE (LVI_LOAD_HARDENING, "lvi-load-hardening")
#undef X86_FEATURE_COMPAT #undef X86_FEATURE_COMPAT
#undef X86_FEATURE #undef X86_FEATURE

llvm/lib/Support/Host.cpp

Show First 20 Lines • Show All 1,489 Lines • ▼ Show 20 Lines#endif
// detecting features using the "-march=native" flag. // detecting features using the "-march=native" flag.
// For more info, see X86 ISA docs. // For more info, see X86 ISA docs.
Features["pconfig"] = HasLeaf7 && ((EDX >> 18) & 1); Features["pconfig"] = HasLeaf7 && ((EDX >> 18) & 1);
Features["amx-bf16"] = HasLeaf7 && ((EDX >> 22) & 1) && HasAMXSave; Features["amx-bf16"] = HasLeaf7 && ((EDX >> 22) & 1) && HasAMXSave;
Features["amx-tile"] = HasLeaf7 && ((EDX >> 24) & 1) && HasAMXSave; Features["amx-tile"] = HasLeaf7 && ((EDX >> 24) & 1) && HasAMXSave;
Features["amx-int8"] = HasLeaf7 && ((EDX >> 25) & 1) && HasAMXSave; Features["amx-int8"] = HasLeaf7 && ((EDX >> 25) & 1) && HasAMXSave;
bool HasLeaf7Subleaf1 = bool HasLeaf7Subleaf1 =
MaxLevel >= 7 && !getX86CpuIDAndInfoEx(0x7, 0x1, &EAX, &EBX, &ECX, &EDX); MaxLevel >= 7 && !getX86CpuIDAndInfoEx(0x7, 0x1, &EAX, &EBX, &ECX, &EDX);
Features["avxvnni"] = HasLeaf7Subleaf1 && ((EAX >> 4) & 1) && HasAVXSave;
Features["avx512bf16"] = HasLeaf7Subleaf1 && ((EAX >> 5) & 1) && HasAVX512Save; Features["avx512bf16"] = HasLeaf7Subleaf1 && ((EAX >> 5) & 1) && HasAVX512Save;
bool HasLeafD = MaxLevel >= 0xd && bool HasLeafD = MaxLevel >= 0xd &&
!getX86CpuIDAndInfoEx(0xd, 0x1, &EAX, &EBX, &ECX, &EDX); !getX86CpuIDAndInfoEx(0xd, 0x1, &EAX, &EBX, &ECX, &EDX);
// Only enable XSAVE if OS has enabled support for saving YMM state. // Only enable XSAVE if OS has enabled support for saving YMM state.
Features["xsaveopt"] = HasLeafD && ((EAX >> 0) & 1) && HasAVXSave; Features["xsaveopt"] = HasLeafD && ((EAX >> 0) & 1) && HasAVXSave;
Features["xsavec"] = HasLeafD && ((EAX >> 1) & 1) && HasAVXSave; Features["xsavec"] = HasLeafD && ((EAX >> 1) & 1) && HasAVXSave;
▲ Show 20 Lines • Show All 105 Lines • Show Last 20 Lines

llvm/lib/Support/X86TargetParser.cpp

Show First 20 Lines • Show All 191 Lines • ▼ Show 20 Linesstatic constexpr FeatureBitset FeaturesICLClient =
FeatureRDPID | FeatureVAES | FeatureVPCLMULQDQ; FeatureRDPID | FeatureVAES | FeatureVPCLMULQDQ;
static constexpr FeatureBitset FeaturesICLServer = static constexpr FeatureBitset FeaturesICLServer =
FeaturesICLClient | FeaturePCONFIG | FeatureWBNOINVD; FeaturesICLClient | FeaturePCONFIG | FeatureWBNOINVD;
static constexpr FeatureBitset FeaturesTigerlake = static constexpr FeatureBitset FeaturesTigerlake =
FeaturesICLClient | FeatureAVX512VP2INTERSECT | FeatureMOVDIR64B | FeaturesICLClient | FeatureAVX512VP2INTERSECT | FeatureMOVDIR64B |
FeatureMOVDIRI | FeatureSHSTK | FeatureKL | FeatureWIDEKL; FeatureMOVDIRI | FeatureSHSTK | FeatureKL | FeatureWIDEKL;
static constexpr FeatureBitset FeaturesSapphireRapids = static constexpr FeatureBitset FeaturesSapphireRapids =
FeaturesICLServer | FeatureAMX_TILE | FeatureAMX_INT8 | FeatureAMX_BF16 | FeaturesICLServer | FeatureAMX_TILE | FeatureAMX_INT8 | FeatureAMX_BF16 |
FeatureAVX512BF16 | FeatureAVX512VP2INTERSECT | FeatureCLDEMOTE | FeatureENQCMD | FeatureAVX512BF16 | FeatureAVX512VP2INTERSECT | FeatureCLDEMOTE | FeatureENQCMD |
FeatureMOVDIR64B | FeatureMOVDIRI | FeaturePTWRITE | FeatureSERIALIZE | FeatureMOVDIR64B | FeatureMOVDIRI | FeaturePTWRITE | FeatureSERIALIZE |
FeatureSHSTK | FeatureTSXLDTRK | FeatureWAITPKG; FeatureSHSTK | FeatureTSXLDTRK | FeatureWAITPKG | FeatureAVXVNNI;
craig.topperUnsubmitted
Not Done
ReplyInline Actions

Add here too.

craig.topper: Add here too.
// Intel Atom processors. // Intel Atom processors.
// Bonnell has feature parity with Core2 and adds MOVBE. // Bonnell has feature parity with Core2 and adds MOVBE.
static constexpr FeatureBitset FeaturesBonnell = FeaturesCore2 | FeatureMOVBE; static constexpr FeatureBitset FeaturesBonnell = FeaturesCore2 | FeatureMOVBE;
// Silvermont has parity with Westmere and Bonnell plus PRFCHW and RDRND. // Silvermont has parity with Westmere and Bonnell plus PRFCHW and RDRND.
static constexpr FeatureBitset FeaturesSilvermont = static constexpr FeatureBitset FeaturesSilvermont =
FeaturesBonnell | FeaturesWestmere | FeaturePRFCHW | FeatureRDRND; FeaturesBonnell | FeaturesWestmere | FeaturePRFCHW | FeatureRDRND;
static constexpr FeatureBitset FeaturesGoldmont = static constexpr FeatureBitset FeaturesGoldmont =
▲ Show 20 Lines • Show All 326 Lines • ▼ Show 20 Lines
static constexpr FeatureBitset ImpliedFeaturesAMX_TILE = {}; static constexpr FeatureBitset ImpliedFeaturesAMX_TILE = {};
static constexpr FeatureBitset ImpliedFeaturesAMX_BF16 = FeatureAMX_TILE; static constexpr FeatureBitset ImpliedFeaturesAMX_BF16 = FeatureAMX_TILE;
static constexpr FeatureBitset ImpliedFeaturesAMX_INT8 = FeatureAMX_TILE; static constexpr FeatureBitset ImpliedFeaturesAMX_INT8 = FeatureAMX_TILE;
// Key Locker Features // Key Locker Features
static constexpr FeatureBitset ImpliedFeaturesKL = FeatureSSE2; static constexpr FeatureBitset ImpliedFeaturesKL = FeatureSSE2;
static constexpr FeatureBitset ImpliedFeaturesWIDEKL = FeatureKL; static constexpr FeatureBitset ImpliedFeaturesWIDEKL = FeatureKL;
// AVXVNNI Features
static constexpr FeatureBitset ImpliedFeaturesAVXVNNI = FeatureAVX2;
static constexpr FeatureInfo FeatureInfos[X86::CPU_FEATURE_MAX] = { static constexpr FeatureInfo FeatureInfos[X86::CPU_FEATURE_MAX] = {
#define X86_FEATURE(ENUM, STR) {{STR}, ImpliedFeatures##ENUM}, #define X86_FEATURE(ENUM, STR) {{STR}, ImpliedFeatures##ENUM},
#include "llvm/Support/X86TargetParser.def" #include "llvm/Support/X86TargetParser.def"
}; };
void llvm::X86::getFeaturesForCPU(StringRef CPU, void llvm::X86::getFeaturesForCPU(StringRef CPU,
SmallVectorImpl<StringRef> &EnabledFeatures) { SmallVectorImpl<StringRef> &EnabledFeatures) {
auto I = llvm::find_if(Processors, auto I = llvm::find_if(Processors,
▲ Show 20 Lines • Show All 69 Lines • Show Last 20 Lines

llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp

Show First 20 Lines • Show All 3,727 Lines • ▼ Show 20 Lines if (ForcedVEXEncoding == VEXEncoding_EVEX &&
(MCID.TSFlags & X86II::EncodingMask) != X86II::EVEX) (MCID.TSFlags & X86II::EncodingMask) != X86II::EVEX)
return Match_Unsupported; return Match_Unsupported;
if ((ForcedVEXEncoding == VEXEncoding_VEX || if ((ForcedVEXEncoding == VEXEncoding_VEX ||
ForcedVEXEncoding == VEXEncoding_VEX3) && ForcedVEXEncoding == VEXEncoding_VEX3) &&
(MCID.TSFlags & X86II::EncodingMask) != X86II::VEX) (MCID.TSFlags & X86II::EncodingMask) != X86II::VEX)
return Match_Unsupported; return Match_Unsupported;
// These instructions are only available with {vex}, {vex2} or {vex3} prefix
if (MCID.TSFlags & X86II::ExplicitVEXPrefix &&
(ForcedVEXEncoding != VEXEncoding_VEX &&
ForcedVEXEncoding != VEXEncoding_VEX3))
pengfeiUnsubmitted
Not Done
ReplyInline Actions

Missing check for VEX2?

pengfei: Missing check for VEX2?
return Match_Unsupported;
// These instructions match ambiguously with their VEX encoded counterparts // These instructions match ambiguously with their VEX encoded counterparts
// and appear first in the matching table. Reject them unless we're forcing // and appear first in the matching table. Reject them unless we're forcing
// EVEX encoding. // EVEX encoding.
// FIXME: We really need a way to break the ambiguity. // FIXME: We really need a way to break the ambiguity.
switch (Opc) { switch (Opc) {
case X86::VCVTSD2SIZrm_Int: case X86::VCVTSD2SIZrm_Int:
case X86::VCVTSD2SI64Zrm_Int: case X86::VCVTSD2SI64Zrm_Int:
case X86::VCVTSS2SIZrm_Int: case X86::VCVTSS2SIZrm_Int:
▲ Show 20 Lines • Show All 863 Lines • Show Last 20 Lines

llvm/lib/Target/X86/MCTargetDesc/X86BaseInfo.h

Show First 20 Lines • Show All 943 Lines • ▼ Show 20 Lines enum : uint64_t {
CD8_Scale_Mask = 127ULL << CD8_Scale_Shift, CD8_Scale_Mask = 127ULL << CD8_Scale_Shift,
/// Explicitly specified rounding control /// Explicitly specified rounding control
EVEX_RCShift = CD8_Scale_Shift + 7, EVEX_RCShift = CD8_Scale_Shift + 7,
EVEX_RC = 1ULL << EVEX_RCShift, EVEX_RC = 1ULL << EVEX_RCShift,
// NOTRACK prefix // NOTRACK prefix
NoTrackShift = EVEX_RCShift + 1, NoTrackShift = EVEX_RCShift + 1,
NOTRACK = 1ULL << NoTrackShift NOTRACK = 1ULL << NoTrackShift,
// Force VEX encoding
ExplicitVEXShift = NoTrackShift + 1,
ExplicitVEXPrefix = 1ULL << ExplicitVEXShift
}; };
/// \returns true if the instruction with given opcode is a prefix. /// \returns true if the instruction with given opcode is a prefix.
inline bool isPrefix(uint64_t TSFlags) { inline bool isPrefix(uint64_t TSFlags) {
return (TSFlags & X86II::FormMask) == PrefixByte; return (TSFlags & X86II::FormMask) == PrefixByte;
} }
/// \returns true if the instruction with given opcode is a pseudo. /// \returns true if the instruction with given opcode is a pseudo.
▲ Show 20 Lines • Show All 259 Lines • Show Last 20 Lines

llvm/lib/Target/X86/MCTargetDesc/X86InstPrinterCommon.cpp

Show First 20 Lines • Show All 339 Lines • ▼ Show 20 Lines if ((TSFlags & X86II::LOCK) || (Flags & X86::IP_HAS_LOCK))
O << "\tlock\t"; O << "\tlock\t";
if ((TSFlags & X86II::NOTRACK) || (Flags & X86::IP_HAS_NOTRACK)) if ((TSFlags & X86II::NOTRACK) || (Flags & X86::IP_HAS_NOTRACK))
O << "\tnotrack\t"; O << "\tnotrack\t";
if (Flags & X86::IP_HAS_REPEAT_NE) if (Flags & X86::IP_HAS_REPEAT_NE)
O << "\trepne\t"; O << "\trepne\t";
else if (Flags & X86::IP_HAS_REPEAT) else if (Flags & X86::IP_HAS_REPEAT)
O << "\trep\t"; O << "\trep\t";
if (TSFlags & X86II::ExplicitVEXPrefix)
// These all require a pseudo prefix
O << "\t{vex}";
} }
craig.topperUnsubmitted
Done
ReplyInline Actions

Put parentheses around the & parts. Similar to lock and notrack above. I don't want to think about operator precedence.

craig.topper: Put parentheses around the & parts. Similar to lock and notrack above. I don't want to think…
void X86InstPrinterCommon::printVKPair(const MCInst *MI, unsigned OpNo, void X86InstPrinterCommon::printVKPair(const MCInst *MI, unsigned OpNo,
raw_ostream &OS) { raw_ostream &OS) {
// In assembly listings, a pair is represented by one of its members, any // In assembly listings, a pair is represented by one of its members, any
// of the two. Here, we pick k0, k2, k4, k6, but we could as well // of the two. Here, we pick k0, k2, k4, k6, but we could as well
// print K2_K3 as "k3". It would probably make a lot more sense, if // print K2_K3 as "k3". It would probably make a lot more sense, if
// the assembly would look something like: // the assembly would look something like:
// "vp2intersect %zmm5, %zmm7, {%k2, %k3}" // "vp2intersect %zmm5, %zmm7, {%k2, %k3}"
Show All 17 Lines

llvm/lib/Target/X86/X86.td

Show First 20 Lines • Show All 165 Lines • ▼ Show 20 Lines
def FeatureIFMA : SubtargetFeature<"avx512ifma", "HasIFMA", "true", def FeatureIFMA : SubtargetFeature<"avx512ifma", "HasIFMA", "true",
"Enable AVX-512 Integer Fused Multiple-Add", "Enable AVX-512 Integer Fused Multiple-Add",
[FeatureAVX512]>; [FeatureAVX512]>;
def FeaturePKU : SubtargetFeature<"pku", "HasPKU", "true", def FeaturePKU : SubtargetFeature<"pku", "HasPKU", "true",
"Enable protection keys">; "Enable protection keys">;
def FeatureVNNI : SubtargetFeature<"avx512vnni", "HasVNNI", "true", def FeatureVNNI : SubtargetFeature<"avx512vnni", "HasVNNI", "true",
"Enable AVX-512 Vector Neural Network Instructions", "Enable AVX-512 Vector Neural Network Instructions",
[FeatureAVX512]>; [FeatureAVX512]>;
def FeatureAVXVNNI : SubtargetFeature<"avxvnni", "HasAVXVNNI", "true",
"Support AVX_VNNI encoding",
[FeatureAVX2]>;
def FeatureBF16 : SubtargetFeature<"avx512bf16", "HasBF16", "true", def FeatureBF16 : SubtargetFeature<"avx512bf16", "HasBF16", "true",
"Support bfloat16 floating point", "Support bfloat16 floating point",
[FeatureBWI]>; [FeatureBWI]>;
def FeatureBITALG : SubtargetFeature<"avx512bitalg", "HasBITALG", "true", def FeatureBITALG : SubtargetFeature<"avx512bitalg", "HasBITALG", "true",
"Enable AVX-512 Bit Algorithms", "Enable AVX-512 Bit Algorithms",
[FeatureBWI]>; [FeatureBWI]>;
def FeatureVP2INTERSECT : SubtargetFeature<"avx512vp2intersect", def FeatureVP2INTERSECT : SubtargetFeature<"avx512vp2intersect",
"HasVP2INTERSECT", "true", "HasVP2INTERSECT", "true",
▲ Show 20 Lines • Show All 569 Lines • ▼ Show 20 Linesdef ProcessorFeatures {
list<SubtargetFeature> SPRAdditionalFeatures = [FeatureAMXTILE, list<SubtargetFeature> SPRAdditionalFeatures = [FeatureAMXTILE,
FeatureAMXINT8, FeatureAMXINT8,
FeatureAMXBF16, FeatureAMXBF16,
FeatureBF16, FeatureBF16,
FeatureSERIALIZE, FeatureSERIALIZE,
FeatureCLDEMOTE, FeatureCLDEMOTE,
FeatureWAITPKG, FeatureWAITPKG,
FeaturePTWRITE, FeaturePTWRITE,
FeatureAVXVNNI,
craig.topperUnsubmitted
Not Done
ReplyInline Actions

Please add this to Alderlake which was added to X86.td by @bkramer yesterday

craig.topper: Please add this to Alderlake which was added to X86.td by @bkramer yesterday
FeatureTSXLDTRK, FeatureTSXLDTRK,
FeatureENQCMD, FeatureENQCMD,
FeatureSHSTK, FeatureSHSTK,
FeatureVP2INTERSECT, FeatureVP2INTERSECT,
FeatureMOVDIRI, FeatureMOVDIRI,
FeatureMOVDIR64B]; FeatureMOVDIR64B];
list<SubtargetFeature> SPRTuning = ICXTuning; list<SubtargetFeature> SPRTuning = ICXTuning;
list<SubtargetFeature> SPRFeatures = list<SubtargetFeature> SPRFeatures =
▲ Show 20 Lines • Show All 669 Lines • Show Last 20 Lines

llvm/lib/Target/X86/X86EvexToVex.cpp

Show First 20 Lines • Show All 79 Lines • ▼ Show 20 Linespublic:
MachineFunctionProperties getRequiredProperties() const override { MachineFunctionProperties getRequiredProperties() const override {
return MachineFunctionProperties().set( return MachineFunctionProperties().set(
MachineFunctionProperties::Property::NoVRegs); MachineFunctionProperties::Property::NoVRegs);
} }
private: private:
/// Machine instruction info used throughout the class. /// Machine instruction info used throughout the class.
const X86InstrInfo *TII = nullptr; const X86InstrInfo *TII = nullptr;
const X86Subtarget *ST = nullptr;
}; };
} // end anonymous namespace } // end anonymous namespace
char EvexToVexInstPass::ID = 0; char EvexToVexInstPass::ID = 0;
bool EvexToVexInstPass::runOnMachineFunction(MachineFunction &MF) { bool EvexToVexInstPass::runOnMachineFunction(MachineFunction &MF) {
TII = MF.getSubtarget<X86Subtarget>().getInstrInfo(); TII = MF.getSubtarget<X86Subtarget>().getInstrInfo();
const X86Subtarget &ST = MF.getSubtarget<X86Subtarget>(); ST = &MF.getSubtarget<X86Subtarget>();
if (!ST.hasAVX512()) if (!ST->hasAVX512())
return false; return false;
bool Changed = false; bool Changed = false;
/// Go over all basic blocks in function and replace /// Go over all basic blocks in function and replace
/// EVEX encoded instrs by VEX encoding when possible. /// EVEX encoded instrs by VEX encoding when possible.
for (MachineBasicBlock &MBB : MF) { for (MachineBasicBlock &MBB : MF) {
Show All 32 Lines for (const MachineOperand &MO : MI.explicit_operands()) {
if (isHiRegIdx(Reg)) if (isHiRegIdx(Reg))
return true; return true;
} }
return false; return false;
} }
// Do any custom cleanup needed to finalize the conversion. // Do any custom cleanup needed to finalize the conversion.
static bool performCustomAdjustments(MachineInstr &MI, unsigned NewOpc) { static bool performCustomAdjustments(MachineInstr &MI, unsigned NewOpc,
const X86Subtarget *ST) {
(void)NewOpc; (void)NewOpc;
unsigned Opc = MI.getOpcode(); unsigned Opc = MI.getOpcode();
switch (Opc) { switch (Opc) {
case X86::VPDPBUSDSZ256m:
case X86::VPDPBUSDSZ256r:
case X86::VPDPBUSDSZ128m:
case X86::VPDPBUSDSZ128r:
case X86::VPDPBUSDZ256m:
case X86::VPDPBUSDZ256r:
case X86::VPDPBUSDZ128m:
case X86::VPDPBUSDZ128r:
case X86::VPDPWSSDSZ256m:
case X86::VPDPWSSDSZ256r:
case X86::VPDPWSSDSZ128m:
case X86::VPDPWSSDSZ128r:
case X86::VPDPWSSDZ256m:
case X86::VPDPWSSDZ256r:
case X86::VPDPWSSDZ128m:
case X86::VPDPWSSDZ128r:
// These can only VEX convert if AVXVNNI is enabled.
return ST->hasAVXVNNI();
case X86::VALIGNDZ128rri: case X86::VALIGNDZ128rri:
case X86::VALIGNDZ128rmi: case X86::VALIGNDZ128rmi:
case X86::VALIGNQZ128rri: case X86::VALIGNQZ128rri:
case X86::VALIGNQZ128rmi: { case X86::VALIGNQZ128rmi: {
assert((NewOpc == X86::VPALIGNRrri || NewOpc == X86::VPALIGNRrmi) && assert((NewOpc == X86::VPALIGNRrri || NewOpc == X86::VPALIGNRrmi) &&
"Unexpected new opcode!"); "Unexpected new opcode!");
unsigned Scale = (Opc == X86::VALIGNQZ128rri || unsigned Scale = (Opc == X86::VALIGNQZ128rri ||
Opc == X86::VALIGNQZ128rmi) ? 8 : 4; Opc == X86::VALIGNQZ128rmi) ? 8 : 4;
▲ Show 20 Lines • Show All 95 Lines • ▼ Show 20 Lines#endif
if (I == Table.end() || I->EvexOpcode != MI.getOpcode()) if (I == Table.end() || I->EvexOpcode != MI.getOpcode())
return false; return false;
unsigned NewOpc = I->VexOpcode; unsigned NewOpc = I->VexOpcode;
if (usesExtendedRegister(MI)) if (usesExtendedRegister(MI))
return false; return false;
if (!performCustomAdjustments(MI, NewOpc)) if (!performCustomAdjustments(MI, NewOpc, ST))
return false; return false;
MI.setDesc(TII->get(NewOpc)); MI.setDesc(TII->get(NewOpc));
MI.setAsmPrinterFlag(X86::AC_EVEX_2_VEX); MI.setAsmPrinterFlag(X86::AC_EVEX_2_VEX);
return true; return true;
} }
INITIALIZE_PASS(EvexToVexInstPass, EVEX2VEX_NAME, EVEX2VEX_DESC, false, false) INITIALIZE_PASS(EvexToVexInstPass, EVEX2VEX_NAME, EVEX2VEX_DESC, false, false)
FunctionPass *llvm::createX86EvexToVexInsts() { FunctionPass *llvm::createX86EvexToVexInsts() {
return new EvexToVexInstPass(); return new EvexToVexInstPass();
} }

llvm/lib/Target/X86/X86InstrFoldTables.cpp

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 3,736 Lines • ▼ Show 20 Linesstatic const X86MemoryFoldTableEntry MemoryFoldTable3[] = {
{ X86::VPCMPWZ256rrik, X86::VPCMPWZ256rmik, 0 }, { X86::VPCMPWZ256rrik, X86::VPCMPWZ256rmik, 0 },
{ X86::VPCMPWZrrik, X86::VPCMPWZrmik, 0 }, { X86::VPCMPWZrrik, X86::VPCMPWZrmik, 0 },
{ X86::VPCONFLICTDZ128rrk, X86::VPCONFLICTDZ128rmk, 0 }, { X86::VPCONFLICTDZ128rrk, X86::VPCONFLICTDZ128rmk, 0 },
{ X86::VPCONFLICTDZ256rrk, X86::VPCONFLICTDZ256rmk, 0 }, { X86::VPCONFLICTDZ256rrk, X86::VPCONFLICTDZ256rmk, 0 },
{ X86::VPCONFLICTDZrrk, X86::VPCONFLICTDZrmk, 0 }, { X86::VPCONFLICTDZrrk, X86::VPCONFLICTDZrmk, 0 },
{ X86::VPCONFLICTQZ128rrk, X86::VPCONFLICTQZ128rmk, 0 }, { X86::VPCONFLICTQZ128rrk, X86::VPCONFLICTQZ128rmk, 0 },
{ X86::VPCONFLICTQZ256rrk, X86::VPCONFLICTQZ256rmk, 0 }, { X86::VPCONFLICTQZ256rrk, X86::VPCONFLICTQZ256rmk, 0 },
{ X86::VPCONFLICTQZrrk, X86::VPCONFLICTQZrmk, 0 }, { X86::VPCONFLICTQZrrk, X86::VPCONFLICTQZrmk, 0 },
{ X86::VPDPBUSDSYrr, X86::VPDPBUSDSYrm, 0 },
{ X86::VPDPBUSDSZ128r, X86::VPDPBUSDSZ128m, 0 }, { X86::VPDPBUSDSZ128r, X86::VPDPBUSDSZ128m, 0 },
{ X86::VPDPBUSDSZ256r, X86::VPDPBUSDSZ256m, 0 }, { X86::VPDPBUSDSZ256r, X86::VPDPBUSDSZ256m, 0 },
{ X86::VPDPBUSDSZr, X86::VPDPBUSDSZm, 0 }, { X86::VPDPBUSDSZr, X86::VPDPBUSDSZm, 0 },
{ X86::VPDPBUSDSrr, X86::VPDPBUSDSrm, 0 },
{ X86::VPDPBUSDYrr, X86::VPDPBUSDYrm, 0 },
{ X86::VPDPBUSDZ128r, X86::VPDPBUSDZ128m, 0 }, { X86::VPDPBUSDZ128r, X86::VPDPBUSDZ128m, 0 },
{ X86::VPDPBUSDZ256r, X86::VPDPBUSDZ256m, 0 }, { X86::VPDPBUSDZ256r, X86::VPDPBUSDZ256m, 0 },
{ X86::VPDPBUSDZr, X86::VPDPBUSDZm, 0 }, { X86::VPDPBUSDZr, X86::VPDPBUSDZm, 0 },
{ X86::VPDPBUSDrr, X86::VPDPBUSDrm, 0 },
{ X86::VPDPWSSDSYrr, X86::VPDPWSSDSYrm, 0 },
{ X86::VPDPWSSDSZ128r, X86::VPDPWSSDSZ128m, 0 }, { X86::VPDPWSSDSZ128r, X86::VPDPWSSDSZ128m, 0 },
{ X86::VPDPWSSDSZ256r, X86::VPDPWSSDSZ256m, 0 }, { X86::VPDPWSSDSZ256r, X86::VPDPWSSDSZ256m, 0 },
{ X86::VPDPWSSDSZr, X86::VPDPWSSDSZm, 0 }, { X86::VPDPWSSDSZr, X86::VPDPWSSDSZm, 0 },
{ X86::VPDPWSSDSrr, X86::VPDPWSSDSrm, 0 },
{ X86::VPDPWSSDYrr, X86::VPDPWSSDYrm, 0 },
{ X86::VPDPWSSDZ128r, X86::VPDPWSSDZ128m, 0 }, { X86::VPDPWSSDZ128r, X86::VPDPWSSDZ128m, 0 },
{ X86::VPDPWSSDZ256r, X86::VPDPWSSDZ256m, 0 }, { X86::VPDPWSSDZ256r, X86::VPDPWSSDZ256m, 0 },
{ X86::VPDPWSSDZr, X86::VPDPWSSDZm, 0 }, { X86::VPDPWSSDZr, X86::VPDPWSSDZm, 0 },
{ X86::VPDPWSSDrr, X86::VPDPWSSDrm, 0 },
{ X86::VPERMBZ128rrkz, X86::VPERMBZ128rmkz, 0 }, { X86::VPERMBZ128rrkz, X86::VPERMBZ128rmkz, 0 },
{ X86::VPERMBZ256rrkz, X86::VPERMBZ256rmkz, 0 }, { X86::VPERMBZ256rrkz, X86::VPERMBZ256rmkz, 0 },
{ X86::VPERMBZrrkz, X86::VPERMBZrmkz, 0 }, { X86::VPERMBZrrkz, X86::VPERMBZrmkz, 0 },
{ X86::VPERMDZ256rrkz, X86::VPERMDZ256rmkz, 0 }, { X86::VPERMDZ256rrkz, X86::VPERMDZ256rmkz, 0 },
{ X86::VPERMDZrrkz, X86::VPERMDZrmkz, 0 }, { X86::VPERMDZrrkz, X86::VPERMDZrmkz, 0 },
{ X86::VPERMI2B128rr, X86::VPERMI2B128rm, 0 }, { X86::VPERMI2B128rr, X86::VPERMI2B128rm, 0 },
{ X86::VPERMI2B256rr, X86::VPERMI2B256rm, 0 }, { X86::VPERMI2B256rr, X86::VPERMI2B256rm, 0 },
{ X86::VPERMI2Brr, X86::VPERMI2Brm, 0 }, { X86::VPERMI2Brr, X86::VPERMI2Brm, 0 },
▲ Show 20 Lines • Show All 1,919 Lines • Show Last 20 Lines

llvm/lib/Target/X86/X86InstrFormats.td

Show First 20 Lines • Show All 257 Lines • ▼ Show 20 Lines
} }
// Mark the instruction as "illegal to memory fold/unfold" // Mark the instruction as "illegal to memory fold/unfold"
class NotMemoryFoldable { bit isMemoryFoldable = 0; } class NotMemoryFoldable { bit isMemoryFoldable = 0; }
// Prevent EVEX->VEX conversion from considering this instruction. // Prevent EVEX->VEX conversion from considering this instruction.
class NotEVEX2VEXConvertible { bit notEVEX2VEXConvertible = 1; } class NotEVEX2VEXConvertible { bit notEVEX2VEXConvertible = 1; }
// Force the instruction to use VEX encoding.
class ExplicitVEXPrefix { bit ExplicitVEXPrefix = 1; }
class X86Inst<bits<8> opcod, Format f, ImmType i, dag outs, dag ins, class X86Inst<bits<8> opcod, Format f, ImmType i, dag outs, dag ins,
string AsmStr, Domain d = GenericDomain> string AsmStr, Domain d = GenericDomain>
: Instruction { : Instruction {
let Namespace = "X86"; let Namespace = "X86";
bits<8> Opcode = opcod; bits<8> Opcode = opcod;
Format Form = f; Format Form = f;
bits<7> FormBits = Form.Value; bits<7> FormBits = Form.Value;
▲ Show 20 Lines • Show All 68 Lines • ▼ Show 20 Linesclass X86Inst<bits<8> opcod, Format f, ImmType i, dag outs, dag ins,
// instruction to replace the current one in case it got picked during generation. // instruction to replace the current one in case it got picked during generation.
string FoldGenRegForm = ?; string FoldGenRegForm = ?;
// Used to prevent an explicit EVEX2VEX override for this instruction. // Used to prevent an explicit EVEX2VEX override for this instruction.
string EVEX2VEXOverride = ?; string EVEX2VEXOverride = ?;
bit isMemoryFoldable = 1; // Is it allowed to memory fold/unfold this instruction? bit isMemoryFoldable = 1; // Is it allowed to memory fold/unfold this instruction?
bit notEVEX2VEXConvertible = 0; // Prevent EVEX->VEX conversion. bit notEVEX2VEXConvertible = 0; // Prevent EVEX->VEX conversion.
bit ExplicitVEXPrefix = 0; // Force the instruction to use VEX encoding.
// TSFlags layout should be kept in sync with X86BaseInfo.h. // TSFlags layout should be kept in sync with X86BaseInfo.h.
let TSFlags{6-0} = FormBits; let TSFlags{6-0} = FormBits;
let TSFlags{8-7} = OpSizeBits; let TSFlags{8-7} = OpSizeBits;
let TSFlags{10-9} = AdSizeBits; let TSFlags{10-9} = AdSizeBits;
// No need for 3rd bit, we don't need to distinguish NoPrfx from PS. // No need for 3rd bit, we don't need to distinguish NoPrfx from PS.
let TSFlags{12-11} = OpPrefixBits{1-0}; let TSFlags{12-11} = OpPrefixBits{1-0};
let TSFlags{15-13} = OpMapBits; let TSFlags{15-13} = OpMapBits;
Show All 12 Linesclass X86Inst<bits<8> opcod, Format f, ImmType i, dag outs, dag ins,
let TSFlags{41} = hasEVEX_K; let TSFlags{41} = hasEVEX_K;
let TSFlags{42} = hasEVEX_Z; let TSFlags{42} = hasEVEX_Z;
let TSFlags{43} = hasEVEX_L2; let TSFlags{43} = hasEVEX_L2;
let TSFlags{44} = hasEVEX_B; let TSFlags{44} = hasEVEX_B;
// If we run out of TSFlags bits, it's possible to encode this in 3 bits. // If we run out of TSFlags bits, it's possible to encode this in 3 bits.
let TSFlags{51-45} = CD8_Scale; let TSFlags{51-45} = CD8_Scale;
let TSFlags{52} = hasEVEX_RC; let TSFlags{52} = hasEVEX_RC;
let TSFlags{53} = hasNoTrackPrefix; let TSFlags{53} = hasNoTrackPrefix;
let TSFlags{54} = ExplicitVEXPrefix;
} }
class PseudoI<dag oops, dag iops, list<dag> pattern> class PseudoI<dag oops, dag iops, list<dag> pattern>
: X86Inst<0, Pseudo, NoImm, oops, iops, ""> { : X86Inst<0, Pseudo, NoImm, oops, iops, ""> {
let Pattern = pattern; let Pattern = pattern;
} }
class I<bits<8> o, Format f, dag outs, dag ins, string asm, class I<bits<8> o, Format f, dag outs, dag ins, string asm,
▲ Show 20 Lines • Show All 620 Lines • Show Last 20 Lines

llvm/lib/Target/X86/X86InstrInfo.cpp

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 2,562 Lines • ▼ Show 20 Linesbool X86InstrInfo::findCommutedOpIndices(const MachineInstr &MI,
case X86::VPTERNLOGQZrmbi: case X86::VPTERNLOGQZrmbi:
case X86::VPTERNLOGDZ128rmbikz: case X86::VPTERNLOGDZ128rmbikz:
case X86::VPTERNLOGDZ256rmbikz: case X86::VPTERNLOGDZ256rmbikz:
case X86::VPTERNLOGDZrmbikz: case X86::VPTERNLOGDZrmbikz:
case X86::VPTERNLOGQZ128rmbikz: case X86::VPTERNLOGQZ128rmbikz:
case X86::VPTERNLOGQZ256rmbikz: case X86::VPTERNLOGQZ256rmbikz:
case X86::VPTERNLOGQZrmbikz: case X86::VPTERNLOGQZrmbikz:
return findThreeSrcCommutedOpIndices(MI, SrcOpIdx1, SrcOpIdx2); return findThreeSrcCommutedOpIndices(MI, SrcOpIdx1, SrcOpIdx2);
case X86::VPDPWSSDYrr:
case X86::VPDPWSSDrr:
case X86::VPDPWSSDSYrr:
case X86::VPDPWSSDSrr:
case X86::VPDPWSSDZ128r: case X86::VPDPWSSDZ128r:
case X86::VPDPWSSDZ128rk: case X86::VPDPWSSDZ128rk:
case X86::VPDPWSSDZ128rkz: case X86::VPDPWSSDZ128rkz:
case X86::VPDPWSSDZ256r: case X86::VPDPWSSDZ256r:
case X86::VPDPWSSDZ256rk: case X86::VPDPWSSDZ256rk:
case X86::VPDPWSSDZ256rkz: case X86::VPDPWSSDZ256rkz:
case X86::VPDPWSSDZr: case X86::VPDPWSSDZr:
case X86::VPDPWSSDZrk: case X86::VPDPWSSDZrk:
▲ Show 20 Lines • Show All 6,442 Lines • Show Last 20 Lines

llvm/lib/Target/X86/X86InstrInfo.td

Show First 20 Lines • Show All 900 Lines • ▼ Show 20 Lines
def HasVLX : Predicate<"Subtarget->hasVLX()">; def HasVLX : Predicate<"Subtarget->hasVLX()">;
def NoVLX : Predicate<"!Subtarget->hasVLX()">; def NoVLX : Predicate<"!Subtarget->hasVLX()">;
def NoVLX_Or_NoBWI : Predicate<"!Subtarget->hasVLX() || !Subtarget->hasBWI()">; def NoVLX_Or_NoBWI : Predicate<"!Subtarget->hasVLX() || !Subtarget->hasBWI()">;
def NoVLX_Or_NoDQI : Predicate<"!Subtarget->hasVLX() || !Subtarget->hasDQI()">; def NoVLX_Or_NoDQI : Predicate<"!Subtarget->hasVLX() || !Subtarget->hasDQI()">;
def PKU : Predicate<"Subtarget->hasPKU()">; def PKU : Predicate<"Subtarget->hasPKU()">;
def HasVNNI : Predicate<"Subtarget->hasVNNI()">; def HasVNNI : Predicate<"Subtarget->hasVNNI()">;
def HasVP2INTERSECT : Predicate<"Subtarget->hasVP2INTERSECT()">; def HasVP2INTERSECT : Predicate<"Subtarget->hasVP2INTERSECT()">;
def HasBF16 : Predicate<"Subtarget->hasBF16()">; def HasBF16 : Predicate<"Subtarget->hasBF16()">;
def HasAVXVNNI : Predicate <"Subtarget->hasAVXVNNI()">;
def NoVLX_Or_NoVNNI : Predicate<"!Subtarget->hasVLX() || !Subtarget->hasVNNI()">;
def HasBITALG : Predicate<"Subtarget->hasBITALG()">; def HasBITALG : Predicate<"Subtarget->hasBITALG()">;
def HasPOPCNT : Predicate<"Subtarget->hasPOPCNT()">; def HasPOPCNT : Predicate<"Subtarget->hasPOPCNT()">;
def HasAES : Predicate<"Subtarget->hasAES()">; def HasAES : Predicate<"Subtarget->hasAES()">;
def HasVAES : Predicate<"Subtarget->hasVAES()">; def HasVAES : Predicate<"Subtarget->hasVAES()">;
def NoVLX_Or_NoVAES : Predicate<"!Subtarget->hasVLX() || !Subtarget->hasVAES()">; def NoVLX_Or_NoVAES : Predicate<"!Subtarget->hasVLX() || !Subtarget->hasVAES()">;
def HasFXSR : Predicate<"Subtarget->hasFXSR()">; def HasFXSR : Predicate<"Subtarget->hasFXSR()">;
def HasXSAVE : Predicate<"Subtarget->hasXSAVE()">; def HasXSAVE : Predicate<"Subtarget->hasXSAVE()">;
▲ Show 20 Lines • Show All 2,756 Lines • Show Last 20 Lines

llvm/lib/Target/X86/X86InstrSSE.td

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 7,159 Lines • ▼ Show 20 Lines
defm VMASKMOVPD : avx_movmask_rm<0x2D, 0x2F, "vmaskmovpd", defm VMASKMOVPD : avx_movmask_rm<0x2D, 0x2F, "vmaskmovpd",
int_x86_avx_maskload_pd, int_x86_avx_maskload_pd,
int_x86_avx_maskload_pd_256, int_x86_avx_maskload_pd_256,
int_x86_avx_maskstore_pd, int_x86_avx_maskstore_pd,
int_x86_avx_maskstore_pd_256, int_x86_avx_maskstore_pd_256,
WriteFMaskMove64, WriteFMaskMove64Y>; WriteFMaskMove64, WriteFMaskMove64Y>;
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// AVX_VNNI
//===----------------------------------------------------------------------===//
let Predicates = [HasAVXVNNI, NoVLX_Or_NoVNNI], Constraints = "$src1 = $dst" in
multiclass avx_vnni_rm<bits<8> opc, string OpcodeStr, SDNode OpNode,
bit IsCommutable> {
let isCommutable = IsCommutable in
def rr : AVX8I<opc, MRMSrcReg, (outs VR128:$dst),
(ins VR128:$src1, VR128:$src2, VR128:$src3),
!strconcat(OpcodeStr, "\t{$src3, $src2, $dst|$dst, $src2, $src3}"),
[(set VR128:$dst, (v4i32 (OpNode VR128:$src1,
VR128:$src2, VR128:$src3)))]>,
VEX_4V, Sched<[SchedWriteVecIMul.XMM]>;
def rm : AVX8I<opc, MRMSrcMem, (outs VR128:$dst),
(ins VR128:$src1, VR128:$src2, i128mem:$src3),
!strconcat(OpcodeStr, "\t{$src3, $src2, $dst|$dst, $src2, $src3}"),
[(set VR128:$dst, (v4i32 (OpNode VR128:$src1, VR128:$src2,
(loadv4i32 addr:$src3))))]>,
VEX_4V, Sched<[SchedWriteVecIMul.XMM]>;
let isCommutable = IsCommutable in
def Yrr : AVX8I<opc, MRMSrcReg, (outs VR256:$dst),
(ins VR256:$src1, VR256:$src2, VR256:$src3),
!strconcat(OpcodeStr, "\t{$src3, $src2, $dst|$dst, $src2, $src3}"),
[(set VR256:$dst, (v8i32 (OpNode VR256:$src1,
VR256:$src2, VR256:$src3)))]>,
VEX_4V, VEX_L, Sched<[SchedWriteVecIMul.XMM]>;
def Yrm : AVX8I<opc, MRMSrcMem, (outs VR256:$dst),
(ins VR256:$src1, VR256:$src2, i256mem:$src3),
!strconcat(OpcodeStr, "\t{$src3, $src2, $dst|$dst, $src2, $src3}"),
[(set VR256:$dst, (v8i32 (OpNode VR256:$src1, VR256:$src2,
(loadv8i32 addr:$src3))))]>,
VEX_4V, VEX_L, Sched<[SchedWriteVecIMul.XMM]>;
}
defm VPDPBUSD : avx_vnni_rm<0x50, "vpdpbusd", X86Vpdpbusd, 0>, ExplicitVEXPrefix;
defm VPDPBUSDS : avx_vnni_rm<0x51, "vpdpbusds", X86Vpdpbusds, 0>, ExplicitVEXPrefix;
defm VPDPWSSD : avx_vnni_rm<0x52, "vpdpwssd", X86Vpdpwssd, 1>, ExplicitVEXPrefix;
defm VPDPWSSDS : avx_vnni_rm<0x53, "vpdpwssds", X86Vpdpwssds, 1>, ExplicitVEXPrefix;
//===----------------------------------------------------------------------===//
// VPERMIL - Permute Single and Double Floating-Point Values // VPERMIL - Permute Single and Double Floating-Point Values
// //
multiclass avx_permil<bits<8> opc_rm, bits<8> opc_rmi, string OpcodeStr, multiclass avx_permil<bits<8> opc_rm, bits<8> opc_rmi, string OpcodeStr,
RegisterClass RC, X86MemOperand x86memop_f, RegisterClass RC, X86MemOperand x86memop_f,
X86MemOperand x86memop_i, X86MemOperand x86memop_i,
ValueType f_vt, ValueType i_vt, ValueType f_vt, ValueType i_vt,
X86FoldableSchedWrite sched, X86FoldableSchedWrite sched,
▲ Show 20 Lines • Show All 793 Lines • Show Last 20 Lines

llvm/lib/Target/X86/X86Subtarget.h

Show First 20 Lines • Show All 349 Lines • ▼ Show 20 Linesclass X86Subtarget final : public X86GenSubtargetInfo {
bool HasVLX = false; bool HasVLX = false;
/// Processor has PKU extenstions /// Processor has PKU extenstions
bool HasPKU = false; bool HasPKU = false;
/// Processor has AVX-512 Vector Neural Network Instructions /// Processor has AVX-512 Vector Neural Network Instructions
bool HasVNNI = false; bool HasVNNI = false;
/// Processor has AVX Vector Neural Network Instructions
bool HasAVXVNNI = false;
/// Processor has AVX-512 bfloat16 floating-point extensions /// Processor has AVX-512 bfloat16 floating-point extensions
bool HasBF16 = false; bool HasBF16 = false;
/// Processor supports ENQCMD instructions /// Processor supports ENQCMD instructions
bool HasENQCMD = false; bool HasENQCMD = false;
/// Processor has AVX-512 Bit Algorithms instructions /// Processor has AVX-512 Bit Algorithms instructions
bool HasBITALG = false; bool HasBITALG = false;
▲ Show 20 Lines • Show All 371 Lines • ▼ Show 20 Linespublic:
bool hasKL() const { return HasKL; } bool hasKL() const { return HasKL; }
bool hasWIDEKL() const { return HasWIDEKL; } bool hasWIDEKL() const { return HasWIDEKL; }
bool hasSERIALIZE() const { return HasSERIALIZE; } bool hasSERIALIZE() const { return HasSERIALIZE; }
bool hasTSXLDTRK() const { return HasTSXLDTRK; } bool hasTSXLDTRK() const { return HasTSXLDTRK; }
bool useRetpolineIndirectCalls() const { return UseRetpolineIndirectCalls; } bool useRetpolineIndirectCalls() const { return UseRetpolineIndirectCalls; }
bool useRetpolineIndirectBranches() const { bool useRetpolineIndirectBranches() const {
return UseRetpolineIndirectBranches; return UseRetpolineIndirectBranches;
} }
bool hasAVXVNNI() const { return HasAVXVNNI; }
bool hasAMXTILE() const { return HasAMXTILE; } bool hasAMXTILE() const { return HasAMXTILE; }
bool hasAMXBF16() const { return HasAMXBF16; } bool hasAMXBF16() const { return HasAMXBF16; }
bool hasAMXINT8() const { return HasAMXINT8; } bool hasAMXINT8() const { return HasAMXINT8; }
bool useRetpolineExternalThunk() const { return UseRetpolineExternalThunk; } bool useRetpolineExternalThunk() const { return UseRetpolineExternalThunk; }
// These are generic getters that OR together all of the thunk types // These are generic getters that OR together all of the thunk types
// supported by the subtarget. Therefore useIndirectThunk*() will return true // supported by the subtarget. Therefore useIndirectThunk*() will return true
// if any respective thunk feature is enabled. // if any respective thunk feature is enabled.
▲ Show 20 Lines • Show All 180 Lines • Show Last 20 Lines

llvm/test/CodeGen/X86/avx-vnni/avx_vnni-intrinsics.ll

  • This file was added.
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
; RUN: llc < %s -mtriple=i686-unknown-unknown -mattr=+avxvnni --show-mc-encoding | FileCheck %s --check-prefixes=CHECK,AVX-X86
pengfeiUnsubmitted
Not Done
ReplyInline Actions

Do we need new directory for the tests?

pengfei: Do we need new directory for the tests?
; RUN: llc < %s -mtriple=i686-unknown-unknown -mattr=+avx512vnni,+avx512vl,+avxvnni --show-mc-encoding | FileCheck %s --check-prefixes=CHECK,AVX512-X86
pengfeiUnsubmitted
Not Done
ReplyInline Actions

I don't see difference between AVX-X86 and AVX-X64, maybe you can use AVX for both, or AVX,AVX-X86 and AVX,AVX-X64 if they do have some differences. And remove CHECK since it never been used.

pengfei: I don't see difference between `AVX-X86` and `AVX-X64`, maybe you can use `AVX` for both, or…
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avxvnni --show-mc-encoding | FileCheck %s --check-prefixes=CHECK,AVX-X64
; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512vnni,+avx512vl,+avxvnni --show-mc-encoding | FileCheck %s --check-prefixes=CHECK,AVX512-X64
declare <8 x i32> @llvm.x86.avx512.vpdpbusd.256(<8 x i32>, <8 x i32>, <8 x i32>)
define <8 x i32>@test_int_x86_avx_vpdpbusd_256(<8 x i32> %x0, <8 x i32> %x1, <8 x i32> %x2) {
; AVX-X86-LABEL: test_int_x86_avx_vpdpbusd_256:
; AVX-X86: # %bb.0:
; AVX-X86-NEXT: {vex} vpdpbusd %ymm2, %ymm1, %ymm0 # encoding: [0xc4,0xe2,0x75,0x50,0xc2]
; AVX-X86-NEXT: retl # encoding: [0xc3]
;
; AVX512-X86-LABEL: test_int_x86_avx_vpdpbusd_256:
; AVX512-X86: # %bb.0:
; AVX512-X86-NEXT: {vex} vpdpbusd %ymm2, %ymm1, %ymm0 # EVEX TO VEX Compression encoding: [0xc4,0xe2,0x75,0x50,0xc2]
; AVX512-X86-NEXT: retl # encoding: [0xc3]
;
; AVX-X64-LABEL: test_int_x86_avx_vpdpbusd_256:
; AVX-X64: # %bb.0:
; AVX-X64-NEXT: {vex} vpdpbusd %ymm2, %ymm1, %ymm0 # encoding: [0xc4,0xe2,0x75,0x50,0xc2]
; AVX-X64-NEXT: retq # encoding: [0xc3]
;
; AVX512-X64-LABEL: test_int_x86_avx_vpdpbusd_256:
; AVX512-X64: # %bb.0:
; AVX512-X64-NEXT: {vex} vpdpbusd %ymm2, %ymm1, %ymm0 # EVEX TO VEX Compression encoding: [0xc4,0xe2,0x75,0x50,0xc2]
; AVX512-X64-NEXT: retq # encoding: [0xc3]
%res = call <8 x i32> @llvm.x86.avx512.vpdpbusd.256(<8 x i32> %x0, <8 x i32> %x1, <8 x i32> %x2)
ret <8 x i32> %res
}
declare <4 x i32> @llvm.x86.avx512.vpdpbusd.128(<4 x i32>, <4 x i32>, <4 x i32>)
define <4 x i32>@test_int_x86_avx_vpdpbusd_128(<4 x i32> %x0, <4 x i32> %x1, <4 x i32> %x2) {
; AVX-X86-LABEL: test_int_x86_avx_vpdpbusd_128:
; AVX-X86: # %bb.0:
; AVX-X86-NEXT: {vex} vpdpbusd %xmm2, %xmm1, %xmm0 # encoding: [0xc4,0xe2,0x71,0x50,0xc2]
; AVX-X86-NEXT: retl # encoding: [0xc3]
;
; AVX512-X86-LABEL: test_int_x86_avx_vpdpbusd_128:
; AVX512-X86: # %bb.0:
; AVX512-X86-NEXT: {vex} vpdpbusd %xmm2, %xmm1, %xmm0 # EVEX TO VEX Compression encoding: [0xc4,0xe2,0x71,0x50,0xc2]
; AVX512-X86-NEXT: retl # encoding: [0xc3]
;
; AVX-X64-LABEL: test_int_x86_avx_vpdpbusd_128:
; AVX-X64: # %bb.0:
; AVX-X64-NEXT: {vex} vpdpbusd %xmm2, %xmm1, %xmm0 # encoding: [0xc4,0xe2,0x71,0x50,0xc2]
; AVX-X64-NEXT: retq # encoding: [0xc3]
;
; AVX512-X64-LABEL: test_int_x86_avx_vpdpbusd_128:
; AVX512-X64: # %bb.0:
; AVX512-X64-NEXT: {vex} vpdpbusd %xmm2, %xmm1, %xmm0 # EVEX TO VEX Compression encoding: [0xc4,0xe2,0x71,0x50,0xc2]
; AVX512-X64-NEXT: retq # encoding: [0xc3]
%res = call <4 x i32> @llvm.x86.avx512.vpdpbusd.128(<4 x i32> %x0, <4 x i32> %x1, <4 x i32> %x2)
ret <4 x i32> %res
}
declare <8 x i32> @llvm.x86.avx512.vpdpbusds.256(<8 x i32>, <8 x i32>, <8 x i32>)
define <8 x i32>@test_int_x86_avx_vpdpbusds_256(<8 x i32> %x0, <8 x i32> %x1, <8 x i32> %x2) {
; AVX-X86-LABEL: test_int_x86_avx_vpdpbusds_256:
; AVX-X86: # %bb.0:
; AVX-X86-NEXT: {vex} vpdpbusds %ymm2, %ymm1, %ymm0 # encoding: [0xc4,0xe2,0x75,0x51,0xc2]
; AVX-X86-NEXT: retl # encoding: [0xc3]
;
; AVX512-X86-LABEL: test_int_x86_avx_vpdpbusds_256:
; AVX512-X86: # %bb.0:
; AVX512-X86-NEXT: {vex} vpdpbusds %ymm2, %ymm1, %ymm0 # EVEX TO VEX Compression encoding: [0xc4,0xe2,0x75,0x51,0xc2]
; AVX512-X86-NEXT: retl # encoding: [0xc3]
;
; AVX-X64-LABEL: test_int_x86_avx_vpdpbusds_256:
; AVX-X64: # %bb.0:
; AVX-X64-NEXT: {vex} vpdpbusds %ymm2, %ymm1, %ymm0 # encoding: [0xc4,0xe2,0x75,0x51,0xc2]
; AVX-X64-NEXT: retq # encoding: [0xc3]
;
; AVX512-X64-LABEL: test_int_x86_avx_vpdpbusds_256:
; AVX512-X64: # %bb.0:
; AVX512-X64-NEXT: {vex} vpdpbusds %ymm2, %ymm1, %ymm0 # EVEX TO VEX Compression encoding: [0xc4,0xe2,0x75,0x51,0xc2]
; AVX512-X64-NEXT: retq # encoding: [0xc3]
%res = call <8 x i32> @llvm.x86.avx512.vpdpbusds.256(<8 x i32> %x0, <8 x i32> %x1, <8 x i32> %x2)
ret <8 x i32> %res
}
declare <4 x i32> @llvm.x86.avx512.vpdpbusds.128(<4 x i32>, <4 x i32>, <4 x i32>)
define <4 x i32>@test_int_x86_avx_vpdpbusds_128(<4 x i32> %x0, <4 x i32> %x1, <4 x i32> %x2) {
; AVX-X86-LABEL: test_int_x86_avx_vpdpbusds_128:
; AVX-X86: # %bb.0:
; AVX-X86-NEXT: {vex} vpdpbusds %xmm2, %xmm1, %xmm0 # encoding: [0xc4,0xe2,0x71,0x51,0xc2]
; AVX-X86-NEXT: retl # encoding: [0xc3]
;
; AVX512-X86-LABEL: test_int_x86_avx_vpdpbusds_128:
; AVX512-X86: # %bb.0:
; AVX512-X86-NEXT: {vex} vpdpbusds %xmm2, %xmm1, %xmm0 # EVEX TO VEX Compression encoding: [0xc4,0xe2,0x71,0x51,0xc2]
; AVX512-X86-NEXT: retl # encoding: [0xc3]
;
; AVX-X64-LABEL: test_int_x86_avx_vpdpbusds_128:
; AVX-X64: # %bb.0:
; AVX-X64-NEXT: {vex} vpdpbusds %xmm2, %xmm1, %xmm0 # encoding: [0xc4,0xe2,0x71,0x51,0xc2]
; AVX-X64-NEXT: retq # encoding: [0xc3]
;
; AVX512-X64-LABEL: test_int_x86_avx_vpdpbusds_128:
; AVX512-X64: # %bb.0:
; AVX512-X64-NEXT: {vex} vpdpbusds %xmm2, %xmm1, %xmm0 # EVEX TO VEX Compression encoding: [0xc4,0xe2,0x71,0x51,0xc2]
; AVX512-X64-NEXT: retq # encoding: [0xc3]
%res = call <4 x i32> @llvm.x86.avx512.vpdpbusds.128(<4 x i32> %x0, <4 x i32> %x1, <4 x i32> %x2)
ret <4 x i32> %res
}
declare <8 x i32> @llvm.x86.avx512.vpdpwssd.256(<8 x i32>, <8 x i32>, <8 x i32>)
define <8 x i32>@test_int_x86_avx_vpdpwssd_256(<8 x i32> %x0, <8 x i32> %x1, <8 x i32> %x2) {
; AVX-X86-LABEL: test_int_x86_avx_vpdpwssd_256:
; AVX-X86: # %bb.0:
; AVX-X86-NEXT: {vex} vpdpwssd %ymm2, %ymm1, %ymm0 # encoding: [0xc4,0xe2,0x75,0x52,0xc2]
; AVX-X86-NEXT: retl # encoding: [0xc3]
;
; AVX512-X86-LABEL: test_int_x86_avx_vpdpwssd_256:
; AVX512-X86: # %bb.0:
; AVX512-X86-NEXT: {vex} vpdpwssd %ymm2, %ymm1, %ymm0 # EVEX TO VEX Compression encoding: [0xc4,0xe2,0x75,0x52,0xc2]
; AVX512-X86-NEXT: retl # encoding: [0xc3]
;
; AVX-X64-LABEL: test_int_x86_avx_vpdpwssd_256:
; AVX-X64: # %bb.0:
; AVX-X64-NEXT: {vex} vpdpwssd %ymm2, %ymm1, %ymm0 # encoding: [0xc4,0xe2,0x75,0x52,0xc2]
; AVX-X64-NEXT: retq # encoding: [0xc3]
;
; AVX512-X64-LABEL: test_int_x86_avx_vpdpwssd_256:
; AVX512-X64: # %bb.0:
; AVX512-X64-NEXT: {vex} vpdpwssd %ymm2, %ymm1, %ymm0 # EVEX TO VEX Compression encoding: [0xc4,0xe2,0x75,0x52,0xc2]
; AVX512-X64-NEXT: retq # encoding: [0xc3]
%res = call <8 x i32> @llvm.x86.avx512.vpdpwssd.256(<8 x i32> %x0, <8 x i32> %x1, <8 x i32> %x2)
ret <8 x i32> %res
}
declare <4 x i32> @llvm.x86.avx512.vpdpwssd.128(<4 x i32>, <4 x i32>, <4 x i32>)
define <4 x i32>@test_int_x86_avx_vpdpwssd_128(<4 x i32> %x0, <4 x i32> %x1, <4 x i32> %x2) {
; AVX-X86-LABEL: test_int_x86_avx_vpdpwssd_128:
; AVX-X86: # %bb.0:
; AVX-X86-NEXT: {vex} vpdpwssd %xmm2, %xmm1, %xmm0 # encoding: [0xc4,0xe2,0x71,0x52,0xc2]
; AVX-X86-NEXT: retl # encoding: [0xc3]
;
; AVX512-X86-LABEL: test_int_x86_avx_vpdpwssd_128:
; AVX512-X86: # %bb.0:
; AVX512-X86-NEXT: {vex} vpdpwssd %xmm2, %xmm1, %xmm0 # EVEX TO VEX Compression encoding: [0xc4,0xe2,0x71,0x52,0xc2]
; AVX512-X86-NEXT: retl # encoding: [0xc3]
;
; AVX-X64-LABEL: test_int_x86_avx_vpdpwssd_128:
; AVX-X64: # %bb.0:
; AVX-X64-NEXT: {vex} vpdpwssd %xmm2, %xmm1, %xmm0 # encoding: [0xc4,0xe2,0x71,0x52,0xc2]
; AVX-X64-NEXT: retq # encoding: [0xc3]
;
; AVX512-X64-LABEL: test_int_x86_avx_vpdpwssd_128:
; AVX512-X64: # %bb.0:
; AVX512-X64-NEXT: {vex} vpdpwssd %xmm2, %xmm1, %xmm0 # EVEX TO VEX Compression encoding: [0xc4,0xe2,0x71,0x52,0xc2]
; AVX512-X64-NEXT: retq # encoding: [0xc3]
%res = call <4 x i32> @llvm.x86.avx512.vpdpwssd.128(<4 x i32> %x0, <4 x i32> %x1, <4 x i32> %x2)
ret <4 x i32> %res
}
declare <8 x i32> @llvm.x86.avx512.vpdpwssds.256(<8 x i32>, <8 x i32>, <8 x i32>)
define <8 x i32>@test_int_x86_avx_vpdpwssds_256(<8 x i32> %x0, <8 x i32> %x1, <8 x i32> %x2) {
; AVX-X86-LABEL: test_int_x86_avx_vpdpwssds_256:
; AVX-X86: # %bb.0:
; AVX-X86-NEXT: {vex} vpdpwssds %ymm2, %ymm1, %ymm0 # encoding: [0xc4,0xe2,0x75,0x53,0xc2]
; AVX-X86-NEXT: retl # encoding: [0xc3]
;
; AVX512-X86-LABEL: test_int_x86_avx_vpdpwssds_256:
; AVX512-X86: # %bb.0:
; AVX512-X86-NEXT: {vex} vpdpwssds %ymm2, %ymm1, %ymm0 # EVEX TO VEX Compression encoding: [0xc4,0xe2,0x75,0x53,0xc2]
; AVX512-X86-NEXT: retl # encoding: [0xc3]
;
; AVX-X64-LABEL: test_int_x86_avx_vpdpwssds_256:
; AVX-X64: # %bb.0:
; AVX-X64-NEXT: {vex} vpdpwssds %ymm2, %ymm1, %ymm0 # encoding: [0xc4,0xe2,0x75,0x53,0xc2]
; AVX-X64-NEXT: retq # encoding: [0xc3]
;
; AVX512-X64-LABEL: test_int_x86_avx_vpdpwssds_256:
; AVX512-X64: # %bb.0:
; AVX512-X64-NEXT: {vex} vpdpwssds %ymm2, %ymm1, %ymm0 # EVEX TO VEX Compression encoding: [0xc4,0xe2,0x75,0x53,0xc2]
; AVX512-X64-NEXT: retq # encoding: [0xc3]
%res = call <8 x i32> @llvm.x86.avx512.vpdpwssds.256(<8 x i32> %x0, <8 x i32> %x1, <8 x i32> %x2)
ret <8 x i32> %res
}
declare <4 x i32> @llvm.x86.avx512.vpdpwssds.128(<4 x i32>, <4 x i32>, <4 x i32>)
define <4 x i32>@test_int_x86_avx_vpdpwssds_128(<4 x i32> %x0, <4 x i32> %x1, <4 x i32> %x2) {
; AVX-X86-LABEL: test_int_x86_avx_vpdpwssds_128:
; AVX-X86: # %bb.0:
; AVX-X86-NEXT: {vex} vpdpwssds %xmm2, %xmm1, %xmm0 # encoding: [0xc4,0xe2,0x71,0x53,0xc2]
; AVX-X86-NEXT: retl # encoding: [0xc3]
;
; AVX512-X86-LABEL: test_int_x86_avx_vpdpwssds_128:
; AVX512-X86: # %bb.0:
; AVX512-X86-NEXT: {vex} vpdpwssds %xmm2, %xmm1, %xmm0 # EVEX TO VEX Compression encoding: [0xc4,0xe2,0x71,0x53,0xc2]
; AVX512-X86-NEXT: retl # encoding: [0xc3]
;
; AVX-X64-LABEL: test_int_x86_avx_vpdpwssds_128:
; AVX-X64: # %bb.0:
; AVX-X64-NEXT: {vex} vpdpwssds %xmm2, %xmm1, %xmm0 # encoding: [0xc4,0xe2,0x71,0x53,0xc2]
; AVX-X64-NEXT: retq # encoding: [0xc3]
;
; AVX512-X64-LABEL: test_int_x86_avx_vpdpwssds_128:
; AVX512-X64: # %bb.0:
; AVX512-X64-NEXT: {vex} vpdpwssds %xmm2, %xmm1, %xmm0 # EVEX TO VEX Compression encoding: [0xc4,0xe2,0x71,0x53,0xc2]
; AVX512-X64-NEXT: retq # encoding: [0xc3]
%res = call <4 x i32> @llvm.x86.avx512.vpdpwssds.128(<4 x i32> %x0, <4 x i32> %x1, <4 x i32> %x2)
ret <4 x i32> %res
}

llvm/test/CodeGen/X86/avx-vnni/stack-folding-int-avxvnni.ll

  • This file was added.
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
; RUN: llc -O3 -disable-peephole -mtriple=x86_64-unknown-unknown -mattr=+avxvnni < %s | FileCheck %s
target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-unknown-unknown"
declare <4 x i32> @llvm.x86.avx512.vpdpwssd.128(<4 x i32>, <4 x i32>, <4 x i32>)
declare <8 x i32> @llvm.x86.avx512.vpdpwssd.256(<8 x i32>, <8 x i32>, <8 x i32>)
declare <4 x i32> @llvm.x86.avx512.vpdpwssds.128(<4 x i32>, <4 x i32>, <4 x i32>)
declare <8 x i32> @llvm.x86.avx512.vpdpwssds.256(<8 x i32>, <8 x i32>, <8 x i32>)
declare <4 x i32> @llvm.x86.avx512.vpdpbusd.128(<4 x i32>, <4 x i32>, <4 x i32>)
declare <8 x i32> @llvm.x86.avx512.vpdpbusd.256(<8 x i32>, <8 x i32>, <8 x i32>)
declare <4 x i32> @llvm.x86.avx512.vpdpbusds.128(<4 x i32>, <4 x i32>, <4 x i32>)
declare <8 x i32> @llvm.x86.avx512.vpdpbusds.256(<8 x i32>, <8 x i32>, <8 x i32>)
define <4 x i32> @stack_fold_vpdpwssd(<4 x i32> %a0, <4 x i32> %a1, <4 x i32> %a2) {
; CHECK-LABEL: stack_fold_vpdpwssd:
; CHECK: # %bb.0:
; CHECK-NEXT: vmovaps %xmm2, {{[-0-9]+}}(%r{{[sb]}}p) # 16-byte Spill
; CHECK-NEXT: #APP
; CHECK-NEXT: nop
; CHECK-NEXT: #NO_APP
; CHECK-NEXT: {vex} vpdpwssd {{[-0-9]+}}(%r{{[sb]}}p), %xmm1, %xmm0 # 16-byte Folded Reload
; CHECK-NEXT: retq
%1 = tail call <2 x i64> asm sideeffect "nop", "=x,~{xmm3},~{xmm4},~{xmm5},~{xmm6},~{xmm7},~{xmm8},~{xmm9},~{xmm10},~{xmm11},~{xmm12},~{xmm13},~{xmm14},~{xmm15},~{flags}"()
%2 = call <4 x i32> @llvm.x86.avx512.vpdpwssd.128(<4 x i32> %a0, <4 x i32> %a1, <4 x i32> %a2)
ret <4 x i32> %2
}
define <4 x i32> @stack_fold_vpdpwssd_commuted(<4 x i32> %a0, <4 x i32> %a1, <4 x i32> %a2) {
; CHECK-LABEL: stack_fold_vpdpwssd_commuted:
; CHECK: # %bb.0:
; CHECK-NEXT: vmovaps %xmm2, {{[-0-9]+}}(%r{{[sb]}}p) # 16-byte Spill
; CHECK-NEXT: #APP
; CHECK-NEXT: nop
; CHECK-NEXT: #NO_APP
; CHECK-NEXT: {vex} vpdpwssd {{[-0-9]+}}(%r{{[sb]}}p), %xmm1, %xmm0 # 16-byte Folded Reload
; CHECK-NEXT: retq
%1 = tail call <2 x i64> asm sideeffect "nop", "=x,~{xmm3},~{xmm4},~{xmm5},~{xmm6},~{xmm7},~{xmm8},~{xmm9},~{xmm10},~{xmm11},~{xmm12},~{xmm13},~{xmm14},~{xmm15},~{flags}"()
%2 = call <4 x i32> @llvm.x86.avx512.vpdpwssd.128(<4 x i32> %a0, <4 x i32> %a2, <4 x i32> %a1)
ret <4 x i32> %2
}
define <8 x i32> @stack_fold_vpdpwssd_256(<8 x i32> %a0, <8 x i32> %a1, <8 x i32> %a2) {
; CHECK-LABEL: stack_fold_vpdpwssd_256:
; CHECK: # %bb.0:
; CHECK-NEXT: vmovups %ymm2, {{[-0-9]+}}(%r{{[sb]}}p) # 32-byte Spill
; CHECK-NEXT: #APP
; CHECK-NEXT: nop
; CHECK-NEXT: #NO_APP
; CHECK-NEXT: {vex} vpdpwssd {{[-0-9]+}}(%r{{[sb]}}p), %ymm1, %ymm0 # 32-byte Folded Reload
; CHECK-NEXT: retq
%1 = tail call <2 x i64> asm sideeffect "nop", "=x,~{xmm3},~{xmm4},~{xmm5},~{xmm6},~{xmm7},~{xmm8},~{xmm9},~{xmm10},~{xmm11},~{xmm12},~{xmm13},~{xmm14},~{xmm15},~{flags}"()
%2 = call <8 x i32> @llvm.x86.avx512.vpdpwssd.256(<8 x i32> %a0, <8 x i32> %a1, <8 x i32> %a2)
ret <8 x i32> %2
}
define <8 x i32> @stack_fold_vpdpwssd_256_commuted(<8 x i32> %a0, <8 x i32> %a1, <8 x i32> %a2) {
; CHECK-LABEL: stack_fold_vpdpwssd_256_commuted:
; CHECK: # %bb.0:
; CHECK-NEXT: vmovups %ymm2, {{[-0-9]+}}(%r{{[sb]}}p) # 32-byte Spill
; CHECK-NEXT: #APP
; CHECK-NEXT: nop
; CHECK-NEXT: #NO_APP
; CHECK-NEXT: {vex} vpdpwssd {{[-0-9]+}}(%r{{[sb]}}p), %ymm1, %ymm0 # 32-byte Folded Reload
; CHECK-NEXT: retq
%1 = tail call <2 x i64> asm sideeffect "nop", "=x,~{xmm3},~{xmm4},~{xmm5},~{xmm6},~{xmm7},~{xmm8},~{xmm9},~{xmm10},~{xmm11},~{xmm12},~{xmm13},~{xmm14},~{xmm15},~{flags}"()
%2 = call <8 x i32> @llvm.x86.avx512.vpdpwssd.256(<8 x i32> %a0, <8 x i32> %a2, <8 x i32> %a1)
ret <8 x i32> %2
}
define <4 x i32> @stack_fold_vpdpwssds(<4 x i32> %a0, <4 x i32> %a1, <4 x i32> %a2) {
; CHECK-LABEL: stack_fold_vpdpwssds:
; CHECK: # %bb.0:
; CHECK-NEXT: vmovaps %xmm2, {{[-0-9]+}}(%r{{[sb]}}p) # 16-byte Spill
; CHECK-NEXT: #APP
; CHECK-NEXT: nop
; CHECK-NEXT: #NO_APP
; CHECK-NEXT: {vex} vpdpwssds {{[-0-9]+}}(%r{{[sb]}}p), %xmm1, %xmm0 # 16-byte Folded Reload
; CHECK-NEXT: retq
%1 = tail call <2 x i64> asm sideeffect "nop", "=x,~{xmm3},~{xmm4},~{xmm5},~{xmm6},~{xmm7},~{xmm8},~{xmm9},~{xmm10},~{xmm11},~{xmm12},~{xmm13},~{xmm14},~{xmm15},~{flags}"()
%2 = call <4 x i32> @llvm.x86.avx512.vpdpwssds.128(<4 x i32> %a0, <4 x i32> %a1, <4 x i32> %a2)
ret <4 x i32> %2
}
define <4 x i32> @stack_fold_vpdpwssds_commuted(<4 x i32> %a0, <4 x i32> %a1, <4 x i32> %a2) {
; CHECK-LABEL: stack_fold_vpdpwssds_commuted:
; CHECK: # %bb.0:
; CHECK-NEXT: vmovaps %xmm2, {{[-0-9]+}}(%r{{[sb]}}p) # 16-byte Spill
; CHECK-NEXT: #APP
; CHECK-NEXT: nop
; CHECK-NEXT: #NO_APP
; CHECK-NEXT: {vex} vpdpwssds {{[-0-9]+}}(%r{{[sb]}}p), %xmm1, %xmm0 # 16-byte Folded Reload
; CHECK-NEXT: retq
%1 = tail call <2 x i64> asm sideeffect "nop", "=x,~{xmm3},~{xmm4},~{xmm5},~{xmm6},~{xmm7},~{xmm8},~{xmm9},~{xmm10},~{xmm11},~{xmm12},~{xmm13},~{xmm14},~{xmm15},~{flags}"()
%2 = call <4 x i32> @llvm.x86.avx512.vpdpwssds.128(<4 x i32> %a0, <4 x i32> %a2, <4 x i32> %a1)
ret <4 x i32> %2
}
define <8 x i32> @stack_fold_vpdpwssds_256(<8 x i32> %a0, <8 x i32> %a1, <8 x i32> %a2) {
; CHECK-LABEL: stack_fold_vpdpwssds_256:
; CHECK: # %bb.0:
; CHECK-NEXT: vmovups %ymm2, {{[-0-9]+}}(%r{{[sb]}}p) # 32-byte Spill
; CHECK-NEXT: #APP
; CHECK-NEXT: nop
; CHECK-NEXT: #NO_APP
; CHECK-NEXT: {vex} vpdpwssds {{[-0-9]+}}(%r{{[sb]}}p), %ymm1, %ymm0 # 32-byte Folded Reload
; CHECK-NEXT: retq
%1 = tail call <2 x i64> asm sideeffect "nop", "=x,~{xmm3},~{xmm4},~{xmm5},~{xmm6},~{xmm7},~{xmm8},~{xmm9},~{xmm10},~{xmm11},~{xmm12},~{xmm13},~{xmm14},~{xmm15},~{flags}"()
%2 = call <8 x i32> @llvm.x86.avx512.vpdpwssds.256(<8 x i32> %a0, <8 x i32> %a1, <8 x i32> %a2)
ret <8 x i32> %2
}
define <8 x i32> @stack_fold_vpdpwssds_256_commuted(<8 x i32> %a0, <8 x i32> %a1, <8 x i32> %a2) {
; CHECK-LABEL: stack_fold_vpdpwssds_256_commuted:
; CHECK: # %bb.0:
; CHECK-NEXT: vmovups %ymm2, {{[-0-9]+}}(%r{{[sb]}}p) # 32-byte Spill
; CHECK-NEXT: #APP
; CHECK-NEXT: nop
; CHECK-NEXT: #NO_APP
; CHECK-NEXT: {vex} vpdpwssds {{[-0-9]+}}(%r{{[sb]}}p), %ymm1, %ymm0 # 32-byte Folded Reload
; CHECK-NEXT: retq
%1 = tail call <2 x i64> asm sideeffect "nop", "=x,~{xmm3},~{xmm4},~{xmm5},~{xmm6},~{xmm7},~{xmm8},~{xmm9},~{xmm10},~{xmm11},~{xmm12},~{xmm13},~{xmm14},~{xmm15},~{flags}"()
%2 = call <8 x i32> @llvm.x86.avx512.vpdpwssds.256(<8 x i32> %a0, <8 x i32> %a2, <8 x i32> %a1)
ret <8 x i32> %2
}
define <4 x i32> @stack_fold_vpdpbusd(<4 x i32> %a0, <4 x i32> %a1, <4 x i32> %a2) {
; CHECK-LABEL: stack_fold_vpdpbusd:
; CHECK: # %bb.0:
; CHECK-NEXT: vmovaps %xmm2, {{[-0-9]+}}(%r{{[sb]}}p) # 16-byte Spill
; CHECK-NEXT: #APP
; CHECK-NEXT: nop
; CHECK-NEXT: #NO_APP
; CHECK-NEXT: {vex} vpdpbusd {{[-0-9]+}}(%r{{[sb]}}p), %xmm1, %xmm0 # 16-byte Folded Reload
; CHECK-NEXT: retq
%1 = tail call <2 x i64> asm sideeffect "nop", "=x,~{xmm3},~{xmm4},~{xmm5},~{xmm6},~{xmm7},~{xmm8},~{xmm9},~{xmm10},~{xmm11},~{xmm12},~{xmm13},~{xmm14},~{xmm15},~{flags}"()
%2 = call <4 x i32> @llvm.x86.avx512.vpdpbusd.128(<4 x i32> %a0, <4 x i32> %a1, <4 x i32> %a2)
ret <4 x i32> %2
}
define <4 x i32> @stack_fold_vpdpbusd_commuted(<4 x i32> %a0, <4 x i32> %a1, <4 x i32> %a2) {
; CHECK-LABEL: stack_fold_vpdpbusd_commuted:
; CHECK: # %bb.0:
; CHECK-NEXT: vmovaps %xmm2, {{[-0-9]+}}(%r{{[sb]}}p) # 16-byte Spill
; CHECK-NEXT: #APP
; CHECK-NEXT: nop
; CHECK-NEXT: #NO_APP
; CHECK-NEXT: vmovdqa {{[-0-9]+}}(%r{{[sb]}}p), %xmm2 # 16-byte Reload
; CHECK-NEXT: {vex} vpdpbusd %xmm1, %xmm2, %xmm0
; CHECK-NEXT: retq
%1 = tail call <2 x i64> asm sideeffect "nop", "=x,~{xmm3},~{xmm4},~{xmm5},~{xmm6},~{xmm7},~{xmm8},~{xmm9},~{xmm10},~{xmm11},~{xmm12},~{xmm13},~{xmm14},~{xmm15},~{flags}"()
%2 = call <4 x i32> @llvm.x86.avx512.vpdpbusd.128(<4 x i32> %a0, <4 x i32> %a2, <4 x i32> %a1)
ret <4 x i32> %2
}
define <8 x i32> @stack_fold_vpdpbusd_256(<8 x i32> %a0, <8 x i32> %a1, <8 x i32> %a2) {
; CHECK-LABEL: stack_fold_vpdpbusd_256:
; CHECK: # %bb.0:
; CHECK-NEXT: vmovups %ymm2, {{[-0-9]+}}(%r{{[sb]}}p) # 32-byte Spill
; CHECK-NEXT: #APP
; CHECK-NEXT: nop
; CHECK-NEXT: #NO_APP
; CHECK-NEXT: {vex} vpdpbusd {{[-0-9]+}}(%r{{[sb]}}p), %ymm1, %ymm0 # 32-byte Folded Reload
; CHECK-NEXT: retq
%1 = tail call <2 x i64> asm sideeffect "nop", "=x,~{xmm3},~{xmm4},~{xmm5},~{xmm6},~{xmm7},~{xmm8},~{xmm9},~{xmm10},~{xmm11},~{xmm12},~{xmm13},~{xmm14},~{xmm15},~{flags}"()
%2 = call <8 x i32> @llvm.x86.avx512.vpdpbusd.256(<8 x i32> %a0, <8 x i32> %a1, <8 x i32> %a2)
ret <8 x i32> %2
}
define <8 x i32> @stack_fold_vpdpbusd_256_commuted(<8 x i32> %a0, <8 x i32> %a1, <8 x i32> %a2) {
; CHECK-LABEL: stack_fold_vpdpbusd_256_commuted:
; CHECK: # %bb.0:
; CHECK-NEXT: vmovups %ymm2, {{[-0-9]+}}(%r{{[sb]}}p) # 32-byte Spill
; CHECK-NEXT: #APP
; CHECK-NEXT: nop
; CHECK-NEXT: #NO_APP
; CHECK-NEXT: vmovdqu {{[-0-9]+}}(%r{{[sb]}}p), %ymm2 # 32-byte Reload
; CHECK-NEXT: {vex} vpdpbusd %ymm1, %ymm2, %ymm0
; CHECK-NEXT: retq
%1 = tail call <2 x i64> asm sideeffect "nop", "=x,~{xmm3},~{xmm4},~{xmm5},~{xmm6},~{xmm7},~{xmm8},~{xmm9},~{xmm10},~{xmm11},~{xmm12},~{xmm13},~{xmm14},~{xmm15},~{flags}"()
%2 = call <8 x i32> @llvm.x86.avx512.vpdpbusd.256(<8 x i32> %a0, <8 x i32> %a2, <8 x i32> %a1)
ret <8 x i32> %2
}
define <4 x i32> @stack_fold_vpdpbusds(<4 x i32> %a0, <4 x i32> %a1, <4 x i32> %a2) {
; CHECK-LABEL: stack_fold_vpdpbusds:
; CHECK: # %bb.0:
; CHECK-NEXT: vmovaps %xmm2, {{[-0-9]+}}(%r{{[sb]}}p) # 16-byte Spill
; CHECK-NEXT: #APP
; CHECK-NEXT: nop
; CHECK-NEXT: #NO_APP
; CHECK-NEXT: {vex} vpdpbusds {{[-0-9]+}}(%r{{[sb]}}p), %xmm1, %xmm0 # 16-byte Folded Reload
; CHECK-NEXT: retq
%1 = tail call <2 x i64> asm sideeffect "nop", "=x,~{xmm3},~{xmm4},~{xmm5},~{xmm6},~{xmm7},~{xmm8},~{xmm9},~{xmm10},~{xmm11},~{xmm12},~{xmm13},~{xmm14},~{xmm15},~{flags}"()
%2 = call <4 x i32> @llvm.x86.avx512.vpdpbusds.128(<4 x i32> %a0, <4 x i32> %a1, <4 x i32> %a2)
ret <4 x i32> %2
}
define <4 x i32> @stack_fold_vpdpbusds_commuted(<4 x i32> %a0, <4 x i32> %a1, <4 x i32> %a2) {
; CHECK-LABEL: stack_fold_vpdpbusds_commuted:
; CHECK: # %bb.0:
; CHECK-NEXT: vmovaps %xmm2, {{[-0-9]+}}(%r{{[sb]}}p) # 16-byte Spill
; CHECK-NEXT: #APP
; CHECK-NEXT: nop
; CHECK-NEXT: #NO_APP
; CHECK-NEXT: vmovdqa {{[-0-9]+}}(%r{{[sb]}}p), %xmm2 # 16-byte Reload
; CHECK-NEXT: {vex} vpdpbusds %xmm1, %xmm2, %xmm0
; CHECK-NEXT: retq
%1 = tail call <2 x i64> asm sideeffect "nop", "=x,~{xmm3},~{xmm4},~{xmm5},~{xmm6},~{xmm7},~{xmm8},~{xmm9},~{xmm10},~{xmm11},~{xmm12},~{xmm13},~{xmm14},~{xmm15},~{flags}"()
%2 = call <4 x i32> @llvm.x86.avx512.vpdpbusds.128(<4 x i32> %a0, <4 x i32> %a2, <4 x i32> %a1)
ret <4 x i32> %2
}
define <8 x i32> @stack_fold_vpdpbusds_256(<8 x i32> %a0, <8 x i32> %a1, <8 x i32> %a2) {
; CHECK-LABEL: stack_fold_vpdpbusds_256:
; CHECK: # %bb.0:
; CHECK-NEXT: vmovups %ymm2, {{[-0-9]+}}(%r{{[sb]}}p) # 32-byte Spill
; CHECK-NEXT: #APP
; CHECK-NEXT: nop
; CHECK-NEXT: #NO_APP
; CHECK-NEXT: {vex} vpdpbusds {{[-0-9]+}}(%r{{[sb]}}p), %ymm1, %ymm0 # 32-byte Folded Reload
; CHECK-NEXT: retq
%1 = tail call <2 x i64> asm sideeffect "nop", "=x,~{xmm3},~{xmm4},~{xmm5},~{xmm6},~{xmm7},~{xmm8},~{xmm9},~{xmm10},~{xmm11},~{xmm12},~{xmm13},~{xmm14},~{xmm15},~{flags}"()
%2 = call <8 x i32> @llvm.x86.avx512.vpdpbusds.256(<8 x i32> %a0, <8 x i32> %a1, <8 x i32> %a2)
ret <8 x i32> %2
}
define <8 x i32> @stack_fold_vpdpbusds_256_commuted(<8 x i32> %a0, <8 x i32> %a1, <8 x i32> %a2) {
; CHECK-LABEL: stack_fold_vpdpbusds_256_commuted:
; CHECK: # %bb.0:
; CHECK-NEXT: vmovups %ymm2, {{[-0-9]+}}(%r{{[sb]}}p) # 32-byte Spill
; CHECK-NEXT: #APP
; CHECK-NEXT: nop
; CHECK-NEXT: #NO_APP
; CHECK-NEXT: vmovdqu {{[-0-9]+}}(%r{{[sb]}}p), %ymm2 # 32-byte Reload
; CHECK-NEXT: {vex} vpdpbusds %ymm1, %ymm2, %ymm0
; CHECK-NEXT: retq
%1 = tail call <2 x i64> asm sideeffect "nop", "=x,~{xmm3},~{xmm4},~{xmm5},~{xmm6},~{xmm7},~{xmm8},~{xmm9},~{xmm10},~{xmm11},~{xmm12},~{xmm13},~{xmm14},~{xmm15},~{flags}"()
%2 = call <8 x i32> @llvm.x86.avx512.vpdpbusds.256(<8 x i32> %a0, <8 x i32> %a2, <8 x i32> %a1)
ret <8 x i32> %2
}

llvm/test/MC/Disassembler/X86/avx_vnni.txt

  • This file was added.
# RUN: llvm-mc --disassemble %s -triple=i686-apple-darwin9 | FileCheck %s
# CHECK: {vex} vpdpbusd %ymm4, %ymm5, %ymm6
0xc4,0xe2,0x55,0x50,0xf4
# CHECK: {vex} vpdpbusd %xmm4, %xmm5, %xmm6
0xc4,0xe2,0x51,0x50,0xf4
# CHECK: {vex} vpdpbusd 268435456(%esp,%esi,8), %ymm5, %ymm6
0xc4,0xe2,0x55,0x50,0xb4,0xf4,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpbusd 291(%edi,%eax,4), %ymm5, %ymm6
0xc4,0xe2,0x55,0x50,0xb4,0x87,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpbusd (%eax), %ymm5, %ymm6
0xc4,0xe2,0x55,0x50,0x30
# CHECK: {vex} vpdpbusd -1024(,%ebp,2), %ymm5, %ymm6
0xc4,0xe2,0x55,0x50,0x34,0x6d,0x00,0xfc,0xff,0xff
# CHECK: {vex} vpdpbusd 4064(%ecx), %ymm5, %ymm6
0xc4,0xe2,0x55,0x50,0xb1,0xe0,0x0f,0x00,0x00
# CHECK: {vex} vpdpbusd -4096(%edx), %ymm5, %ymm6
0xc4,0xe2,0x55,0x50,0xb2,0x00,0xf0,0xff,0xff
# CHECK: {vex} vpdpbusd 268435456(%esp,%esi,8), %xmm5, %xmm6
0xc4,0xe2,0x51,0x50,0xb4,0xf4,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpbusd 291(%edi,%eax,4), %xmm5, %xmm6
0xc4,0xe2,0x51,0x50,0xb4,0x87,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpbusd (%eax), %xmm5, %xmm6
0xc4,0xe2,0x51,0x50,0x30
# CHECK: {vex} vpdpbusd -512(,%ebp,2), %xmm5, %xmm6
0xc4,0xe2,0x51,0x50,0x34,0x6d,0x00,0xfe,0xff,0xff
# CHECK: {vex} vpdpbusd 2032(%ecx), %xmm5, %xmm6
0xc4,0xe2,0x51,0x50,0xb1,0xf0,0x07,0x00,0x00
# CHECK: {vex} vpdpbusd -2048(%edx), %xmm5, %xmm6
0xc4,0xe2,0x51,0x50,0xb2,0x00,0xf8,0xff,0xff
# CHECK: {vex} vpdpbusds %ymm4, %ymm5, %ymm6
0xc4,0xe2,0x55,0x51,0xf4
# CHECK: {vex} vpdpbusds %xmm4, %xmm5, %xmm6
0xc4,0xe2,0x51,0x51,0xf4
# CHECK: {vex} vpdpbusds 268435456(%esp,%esi,8), %ymm5, %ymm6
0xc4,0xe2,0x55,0x51,0xb4,0xf4,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpbusds 291(%edi,%eax,4), %ymm5, %ymm6
0xc4,0xe2,0x55,0x51,0xb4,0x87,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpbusds (%eax), %ymm5, %ymm6
0xc4,0xe2,0x55,0x51,0x30
# CHECK: {vex} vpdpbusds -1024(,%ebp,2), %ymm5, %ymm6
0xc4,0xe2,0x55,0x51,0x34,0x6d,0x00,0xfc,0xff,0xff
# CHECK: {vex} vpdpbusds 4064(%ecx), %ymm5, %ymm6
0xc4,0xe2,0x55,0x51,0xb1,0xe0,0x0f,0x00,0x00
# CHECK: {vex} vpdpbusds -4096(%edx), %ymm5, %ymm6
0xc4,0xe2,0x55,0x51,0xb2,0x00,0xf0,0xff,0xff
# CHECK: {vex} vpdpbusds 268435456(%esp,%esi,8), %xmm5, %xmm6
0xc4,0xe2,0x51,0x51,0xb4,0xf4,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpbusds 291(%edi,%eax,4), %xmm5, %xmm6
0xc4,0xe2,0x51,0x51,0xb4,0x87,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpbusds (%eax), %xmm5, %xmm6
0xc4,0xe2,0x51,0x51,0x30
# CHECK: {vex} vpdpbusds -512(,%ebp,2), %xmm5, %xmm6
0xc4,0xe2,0x51,0x51,0x34,0x6d,0x00,0xfe,0xff,0xff
# CHECK: {vex} vpdpbusds 2032(%ecx), %xmm5, %xmm6
0xc4,0xe2,0x51,0x51,0xb1,0xf0,0x07,0x00,0x00
# CHECK: {vex} vpdpbusds -2048(%edx), %xmm5, %xmm6
0xc4,0xe2,0x51,0x51,0xb2,0x00,0xf8,0xff,0xff
# CHECK: {vex} vpdpwssd %ymm4, %ymm5, %ymm6
0xc4,0xe2,0x55,0x52,0xf4
# CHECK: {vex} vpdpwssd %xmm4, %xmm5, %xmm6
0xc4,0xe2,0x51,0x52,0xf4
# CHECK: {vex} vpdpwssd 268435456(%esp,%esi,8), %ymm5, %ymm6
0xc4,0xe2,0x55,0x52,0xb4,0xf4,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpwssd 291(%edi,%eax,4), %ymm5, %ymm6
0xc4,0xe2,0x55,0x52,0xb4,0x87,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpwssd (%eax), %ymm5, %ymm6
0xc4,0xe2,0x55,0x52,0x30
# CHECK: {vex} vpdpwssd -1024(,%ebp,2), %ymm5, %ymm6
0xc4,0xe2,0x55,0x52,0x34,0x6d,0x00,0xfc,0xff,0xff
# CHECK: {vex} vpdpwssd 4064(%ecx), %ymm5, %ymm6
0xc4,0xe2,0x55,0x52,0xb1,0xe0,0x0f,0x00,0x00
# CHECK: {vex} vpdpwssd -4096(%edx), %ymm5, %ymm6
0xc4,0xe2,0x55,0x52,0xb2,0x00,0xf0,0xff,0xff
# CHECK: {vex} vpdpwssd 268435456(%esp,%esi,8), %xmm5, %xmm6
0xc4,0xe2,0x51,0x52,0xb4,0xf4,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpwssd 291(%edi,%eax,4), %xmm5, %xmm6
0xc4,0xe2,0x51,0x52,0xb4,0x87,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpwssd (%eax), %xmm5, %xmm6
0xc4,0xe2,0x51,0x52,0x30
# CHECK: {vex} vpdpwssd -512(,%ebp,2), %xmm5, %xmm6
0xc4,0xe2,0x51,0x52,0x34,0x6d,0x00,0xfe,0xff,0xff
# CHECK: {vex} vpdpwssd 2032(%ecx), %xmm5, %xmm6
0xc4,0xe2,0x51,0x52,0xb1,0xf0,0x07,0x00,0x00
# CHECK: {vex} vpdpwssd -2048(%edx), %xmm5, %xmm6
0xc4,0xe2,0x51,0x52,0xb2,0x00,0xf8,0xff,0xff
# CHECK: {vex} vpdpwssds %ymm4, %ymm5, %ymm6
0xc4,0xe2,0x55,0x53,0xf4
# CHECK: {vex} vpdpwssds %xmm4, %xmm5, %xmm6
0xc4,0xe2,0x51,0x53,0xf4
# CHECK: {vex} vpdpwssds 268435456(%esp,%esi,8), %ymm5, %ymm6
0xc4,0xe2,0x55,0x53,0xb4,0xf4,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpwssds 291(%edi,%eax,4), %ymm5, %ymm6
0xc4,0xe2,0x55,0x53,0xb4,0x87,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpwssds (%eax), %ymm5, %ymm6
0xc4,0xe2,0x55,0x53,0x30
# CHECK: {vex} vpdpwssds -1024(,%ebp,2), %ymm5, %ymm6
0xc4,0xe2,0x55,0x53,0x34,0x6d,0x00,0xfc,0xff,0xff
# CHECK: {vex} vpdpwssds 4064(%ecx), %ymm5, %ymm6
0xc4,0xe2,0x55,0x53,0xb1,0xe0,0x0f,0x00,0x00
# CHECK: {vex} vpdpwssds -4096(%edx), %ymm5, %ymm6
0xc4,0xe2,0x55,0x53,0xb2,0x00,0xf0,0xff,0xff
# CHECK: {vex} vpdpwssds 268435456(%esp,%esi,8), %xmm5, %xmm6
0xc4,0xe2,0x51,0x53,0xb4,0xf4,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpwssds 291(%edi,%eax,4), %xmm5, %xmm6
0xc4,0xe2,0x51,0x53,0xb4,0x87,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpwssds (%eax), %xmm5, %xmm6
0xc4,0xe2,0x51,0x53,0x30
# CHECK: {vex} vpdpwssds -512(,%ebp,2), %xmm5, %xmm6
0xc4,0xe2,0x51,0x53,0x34,0x6d,0x00,0xfe,0xff,0xff
# CHECK: {vex} vpdpwssds 2032(%ecx), %xmm5, %xmm6
0xc4,0xe2,0x51,0x53,0xb1,0xf0,0x07,0x00,0x00
# CHECK: {vex} vpdpwssds -2048(%edx), %xmm5, %xmm6
0xc4,0xe2,0x51,0x53,0xb2,0x00,0xf8,0xff,0xff

llvm/test/MC/Disassembler/X86/intel-syntax-avx_vnni.txt

  • This file was added.
# RUN: llvm-mc --disassemble %s -triple=i686 -x86-asm-syntax=intel --output-asm-variant=1 | FileCheck %s
# CHECK: {vex} vpdpbusd ymm6, ymm5, ymm4
0xc4,0xe2,0x55,0x50,0xf4
# CHECK: {vex} vpdpbusd xmm6, xmm5, xmm4
0xc4,0xe2,0x51,0x50,0xf4
# CHECK: {vex} vpdpbusd ymm6, ymm5, ymmword ptr [esp + 8*esi + 268435456]
0xc4,0xe2,0x55,0x50,0xb4,0xf4,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpbusd ymm6, ymm5, ymmword ptr [edi + 4*eax + 291]
0xc4,0xe2,0x55,0x50,0xb4,0x87,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpbusd ymm6, ymm5, ymmword ptr [eax]
0xc4,0xe2,0x55,0x50,0x30
# CHECK: {vex} vpdpbusd ymm6, ymm5, ymmword ptr [2*ebp - 1024]
0xc4,0xe2,0x55,0x50,0x34,0x6d,0x00,0xfc,0xff,0xff
# CHECK: {vex} vpdpbusd ymm6, ymm5, ymmword ptr [ecx + 4064]
0xc4,0xe2,0x55,0x50,0xb1,0xe0,0x0f,0x00,0x00
# CHECK: {vex} vpdpbusd ymm6, ymm5, ymmword ptr [edx - 4096]
0xc4,0xe2,0x55,0x50,0xb2,0x00,0xf0,0xff,0xff
# CHECK: {vex} vpdpbusd xmm6, xmm5, xmmword ptr [esp + 8*esi + 268435456]
0xc4,0xe2,0x51,0x50,0xb4,0xf4,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpbusd xmm6, xmm5, xmmword ptr [edi + 4*eax + 291]
0xc4,0xe2,0x51,0x50,0xb4,0x87,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpbusd xmm6, xmm5, xmmword ptr [eax]
0xc4,0xe2,0x51,0x50,0x30
# CHECK: {vex} vpdpbusd xmm6, xmm5, xmmword ptr [2*ebp - 512]
0xc4,0xe2,0x51,0x50,0x34,0x6d,0x00,0xfe,0xff,0xff
# CHECK: {vex} vpdpbusd xmm6, xmm5, xmmword ptr [ecx + 2032]
0xc4,0xe2,0x51,0x50,0xb1,0xf0,0x07,0x00,0x00
# CHECK: {vex} vpdpbusd xmm6, xmm5, xmmword ptr [edx - 2048]
0xc4,0xe2,0x51,0x50,0xb2,0x00,0xf8,0xff,0xff
# CHECK: {vex} vpdpbusds ymm6, ymm5, ymm4
0xc4,0xe2,0x55,0x51,0xf4
# CHECK: {vex} vpdpbusds xmm6, xmm5, xmm4
0xc4,0xe2,0x51,0x51,0xf4
# CHECK: {vex} vpdpbusds ymm6, ymm5, ymmword ptr [esp + 8*esi + 268435456]
0xc4,0xe2,0x55,0x51,0xb4,0xf4,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpbusds ymm6, ymm5, ymmword ptr [edi + 4*eax + 291]
0xc4,0xe2,0x55,0x51,0xb4,0x87,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpbusds ymm6, ymm5, ymmword ptr [eax]
0xc4,0xe2,0x55,0x51,0x30
# CHECK: {vex} vpdpbusds ymm6, ymm5, ymmword ptr [2*ebp - 1024]
0xc4,0xe2,0x55,0x51,0x34,0x6d,0x00,0xfc,0xff,0xff
# CHECK: {vex} vpdpbusds ymm6, ymm5, ymmword ptr [ecx + 4064]
0xc4,0xe2,0x55,0x51,0xb1,0xe0,0x0f,0x00,0x00
# CHECK: {vex} vpdpbusds ymm6, ymm5, ymmword ptr [edx - 4096]
0xc4,0xe2,0x55,0x51,0xb2,0x00,0xf0,0xff,0xff
# CHECK: {vex} vpdpbusds xmm6, xmm5, xmmword ptr [esp + 8*esi + 268435456]
0xc4,0xe2,0x51,0x51,0xb4,0xf4,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpbusds xmm6, xmm5, xmmword ptr [edi + 4*eax + 291]
0xc4,0xe2,0x51,0x51,0xb4,0x87,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpbusds xmm6, xmm5, xmmword ptr [eax]
0xc4,0xe2,0x51,0x51,0x30
# CHECK: {vex} vpdpbusds xmm6, xmm5, xmmword ptr [2*ebp - 512]
0xc4,0xe2,0x51,0x51,0x34,0x6d,0x00,0xfe,0xff,0xff
# CHECK: {vex} vpdpbusds xmm6, xmm5, xmmword ptr [ecx + 2032]
0xc4,0xe2,0x51,0x51,0xb1,0xf0,0x07,0x00,0x00
# CHECK: {vex} vpdpbusds xmm6, xmm5, xmmword ptr [edx - 2048]
0xc4,0xe2,0x51,0x51,0xb2,0x00,0xf8,0xff,0xff
# CHECK: {vex} vpdpwssd ymm6, ymm5, ymm4
0xc4,0xe2,0x55,0x52,0xf4
# CHECK: {vex} vpdpwssd xmm6, xmm5, xmm4
0xc4,0xe2,0x51,0x52,0xf4
# CHECK: {vex} vpdpwssd ymm6, ymm5, ymmword ptr [esp + 8*esi + 268435456]
0xc4,0xe2,0x55,0x52,0xb4,0xf4,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpwssd ymm6, ymm5, ymmword ptr [edi + 4*eax + 291]
0xc4,0xe2,0x55,0x52,0xb4,0x87,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpwssd ymm6, ymm5, ymmword ptr [eax]
0xc4,0xe2,0x55,0x52,0x30
# CHECK: {vex} vpdpwssd ymm6, ymm5, ymmword ptr [2*ebp - 1024]
0xc4,0xe2,0x55,0x52,0x34,0x6d,0x00,0xfc,0xff,0xff
# CHECK: {vex} vpdpwssd ymm6, ymm5, ymmword ptr [ecx + 4064]
0xc4,0xe2,0x55,0x52,0xb1,0xe0,0x0f,0x00,0x00
# CHECK: {vex} vpdpwssd ymm6, ymm5, ymmword ptr [edx - 4096]
0xc4,0xe2,0x55,0x52,0xb2,0x00,0xf0,0xff,0xff
# CHECK: {vex} vpdpwssd xmm6, xmm5, xmmword ptr [esp + 8*esi + 268435456]
0xc4,0xe2,0x51,0x52,0xb4,0xf4,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpwssd xmm6, xmm5, xmmword ptr [edi + 4*eax + 291]
0xc4,0xe2,0x51,0x52,0xb4,0x87,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpwssd xmm6, xmm5, xmmword ptr [eax]
0xc4,0xe2,0x51,0x52,0x30
# CHECK: {vex} vpdpwssd xmm6, xmm5, xmmword ptr [2*ebp - 512]
0xc4,0xe2,0x51,0x52,0x34,0x6d,0x00,0xfe,0xff,0xff
# CHECK: {vex} vpdpwssd xmm6, xmm5, xmmword ptr [ecx + 2032]
0xc4,0xe2,0x51,0x52,0xb1,0xf0,0x07,0x00,0x00
# CHECK: {vex} vpdpwssd xmm6, xmm5, xmmword ptr [edx - 2048]
0xc4,0xe2,0x51,0x52,0xb2,0x00,0xf8,0xff,0xff
# CHECK: {vex} vpdpwssds ymm6, ymm5, ymm4
0xc4,0xe2,0x55,0x53,0xf4
# CHECK: {vex} vpdpwssds xmm6, xmm5, xmm4
0xc4,0xe2,0x51,0x53,0xf4
# CHECK: {vex} vpdpwssds ymm6, ymm5, ymmword ptr [esp + 8*esi + 268435456]
0xc4,0xe2,0x55,0x53,0xb4,0xf4,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpwssds ymm6, ymm5, ymmword ptr [edi + 4*eax + 291]
0xc4,0xe2,0x55,0x53,0xb4,0x87,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpwssds ymm6, ymm5, ymmword ptr [eax]
0xc4,0xe2,0x55,0x53,0x30
# CHECK: {vex} vpdpwssds ymm6, ymm5, ymmword ptr [2*ebp - 1024]
0xc4,0xe2,0x55,0x53,0x34,0x6d,0x00,0xfc,0xff,0xff
# CHECK: {vex} vpdpwssds ymm6, ymm5, ymmword ptr [ecx + 4064]
0xc4,0xe2,0x55,0x53,0xb1,0xe0,0x0f,0x00,0x00
# CHECK: {vex} vpdpwssds ymm6, ymm5, ymmword ptr [edx - 4096]
0xc4,0xe2,0x55,0x53,0xb2,0x00,0xf0,0xff,0xff
# CHECK: {vex} vpdpwssds xmm6, xmm5, xmmword ptr [esp + 8*esi + 268435456]
0xc4,0xe2,0x51,0x53,0xb4,0xf4,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpwssds xmm6, xmm5, xmmword ptr [edi + 4*eax + 291]
0xc4,0xe2,0x51,0x53,0xb4,0x87,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpwssds xmm6, xmm5, xmmword ptr [eax]
0xc4,0xe2,0x51,0x53,0x30
# CHECK: {vex} vpdpwssds xmm6, xmm5, xmmword ptr [2*ebp - 512]
0xc4,0xe2,0x51,0x53,0x34,0x6d,0x00,0xfe,0xff,0xff
# CHECK: {vex} vpdpwssds xmm6, xmm5, xmmword ptr [ecx + 2032]
0xc4,0xe2,0x51,0x53,0xb1,0xf0,0x07,0x00,0x00
# CHECK: {vex} vpdpwssds xmm6, xmm5, xmmword ptr [edx - 2048]
0xc4,0xe2,0x51,0x53,0xb2,0x00,0xf8,0xff,0xff

llvm/test/MC/Disassembler/X86/intel-syntax-x86-64-avx_vnni.txt

  • This file was added.
# RUN: llvm-mc --disassemble %s -triple=x86_64 -x86-asm-syntax=intel --output-asm-variant=1 | FileCheck %s
# CHECK: {vex} vpdpbusd ymm6, ymm5, ymm4
0xc4,0xe2,0x55,0x50,0xf4
# CHECK: {vex} vpdpbusd xmm6, xmm5, xmm4
0xc4,0xe2,0x51,0x50,0xf4
# CHECK: {vex} vpdpbusd ymm6, ymm5, ymmword ptr [rbp + 8*r14 + 268435456]
0xc4,0xa2,0x55,0x50,0xb4,0xf5,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpbusd ymm6, ymm5, ymmword ptr [r8 + 4*rax + 291]
0xc4,0xc2,0x55,0x50,0xb4,0x80,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpbusd ymm6, ymm5, ymmword ptr [rip]
0xc4,0xe2,0x55,0x50,0x35,0x00,0x00,0x00,0x00
# CHECK: {vex} vpdpbusd ymm6, ymm5, ymmword ptr [2*rbp - 1024]
0xc4,0xe2,0x55,0x50,0x34,0x6d,0x00,0xfc,0xff,0xff
# CHECK: {vex} vpdpbusd ymm6, ymm5, ymmword ptr [rcx + 4064]
0xc4,0xe2,0x55,0x50,0xb1,0xe0,0x0f,0x00,0x00
# CHECK: {vex} vpdpbusd ymm6, ymm5, ymmword ptr [rdx - 4096]
0xc4,0xe2,0x55,0x50,0xb2,0x00,0xf0,0xff,0xff
# CHECK: {vex} vpdpbusd xmm6, xmm5, xmmword ptr [rbp + 8*r14 + 268435456]
0xc4,0xa2,0x51,0x50,0xb4,0xf5,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpbusd xmm6, xmm5, xmmword ptr [r8 + 4*rax + 291]
0xc4,0xc2,0x51,0x50,0xb4,0x80,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpbusd xmm6, xmm5, xmmword ptr [rip]
0xc4,0xe2,0x51,0x50,0x35,0x00,0x00,0x00,0x00
# CHECK: {vex} vpdpbusd xmm6, xmm5, xmmword ptr [2*rbp - 512]
0xc4,0xe2,0x51,0x50,0x34,0x6d,0x00,0xfe,0xff,0xff
# CHECK: {vex} vpdpbusd xmm6, xmm5, xmmword ptr [rcx + 2032]
0xc4,0xe2,0x51,0x50,0xb1,0xf0,0x07,0x00,0x00
# CHECK: {vex} vpdpbusd xmm6, xmm5, xmmword ptr [rdx - 2048]
0xc4,0xe2,0x51,0x50,0xb2,0x00,0xf8,0xff,0xff
# CHECK: {vex} vpdpbusds ymm6, ymm5, ymm4
0xc4,0xe2,0x55,0x51,0xf4
# CHECK: {vex} vpdpbusds xmm6, xmm5, xmm4
0xc4,0xe2,0x51,0x51,0xf4
# CHECK: {vex} vpdpbusds ymm6, ymm5, ymmword ptr [rbp + 8*r14 + 268435456]
0xc4,0xa2,0x55,0x51,0xb4,0xf5,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpbusds ymm6, ymm5, ymmword ptr [r8 + 4*rax + 291]
0xc4,0xc2,0x55,0x51,0xb4,0x80,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpbusds ymm6, ymm5, ymmword ptr [rip]
0xc4,0xe2,0x55,0x51,0x35,0x00,0x00,0x00,0x00
# CHECK: {vex} vpdpbusds ymm6, ymm5, ymmword ptr [2*rbp - 1024]
0xc4,0xe2,0x55,0x51,0x34,0x6d,0x00,0xfc,0xff,0xff
# CHECK: {vex} vpdpbusds ymm6, ymm5, ymmword ptr [rcx + 4064]
0xc4,0xe2,0x55,0x51,0xb1,0xe0,0x0f,0x00,0x00
# CHECK: {vex} vpdpbusds ymm6, ymm5, ymmword ptr [rdx - 4096]
0xc4,0xe2,0x55,0x51,0xb2,0x00,0xf0,0xff,0xff
# CHECK: {vex} vpdpbusds xmm6, xmm5, xmmword ptr [rbp + 8*r14 + 268435456]
0xc4,0xa2,0x51,0x51,0xb4,0xf5,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpbusds xmm6, xmm5, xmmword ptr [r8 + 4*rax + 291]
0xc4,0xc2,0x51,0x51,0xb4,0x80,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpbusds xmm6, xmm5, xmmword ptr [rip]
0xc4,0xe2,0x51,0x51,0x35,0x00,0x00,0x00,0x00
# CHECK: {vex} vpdpbusds xmm6, xmm5, xmmword ptr [2*rbp - 512]
0xc4,0xe2,0x51,0x51,0x34,0x6d,0x00,0xfe,0xff,0xff
# CHECK: {vex} vpdpbusds xmm6, xmm5, xmmword ptr [rcx + 2032]
0xc4,0xe2,0x51,0x51,0xb1,0xf0,0x07,0x00,0x00
# CHECK: {vex} vpdpbusds xmm6, xmm5, xmmword ptr [rdx - 2048]
0xc4,0xe2,0x51,0x51,0xb2,0x00,0xf8,0xff,0xff
# CHECK: {vex} vpdpwssd ymm6, ymm5, ymm4
0xc4,0xe2,0x55,0x52,0xf4
# CHECK: {vex} vpdpwssd xmm6, xmm5, xmm4
0xc4,0xe2,0x51,0x52,0xf4
# CHECK: {vex} vpdpwssd ymm6, ymm5, ymmword ptr [rbp + 8*r14 + 268435456]
0xc4,0xa2,0x55,0x52,0xb4,0xf5,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpwssd ymm6, ymm5, ymmword ptr [r8 + 4*rax + 291]
0xc4,0xc2,0x55,0x52,0xb4,0x80,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpwssd ymm6, ymm5, ymmword ptr [rip]
0xc4,0xe2,0x55,0x52,0x35,0x00,0x00,0x00,0x00
# CHECK: {vex} vpdpwssd ymm6, ymm5, ymmword ptr [2*rbp - 1024]
0xc4,0xe2,0x55,0x52,0x34,0x6d,0x00,0xfc,0xff,0xff
# CHECK: {vex} vpdpwssd ymm6, ymm5, ymmword ptr [rcx + 4064]
0xc4,0xe2,0x55,0x52,0xb1,0xe0,0x0f,0x00,0x00
# CHECK: {vex} vpdpwssd ymm6, ymm5, ymmword ptr [rdx - 4096]
0xc4,0xe2,0x55,0x52,0xb2,0x00,0xf0,0xff,0xff
# CHECK: {vex} vpdpwssd xmm6, xmm5, xmmword ptr [rbp + 8*r14 + 268435456]
0xc4,0xa2,0x51,0x52,0xb4,0xf5,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpwssd xmm6, xmm5, xmmword ptr [r8 + 4*rax + 291]
0xc4,0xc2,0x51,0x52,0xb4,0x80,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpwssd xmm6, xmm5, xmmword ptr [rip]
0xc4,0xe2,0x51,0x52,0x35,0x00,0x00,0x00,0x00
# CHECK: {vex} vpdpwssd xmm6, xmm5, xmmword ptr [2*rbp - 512]
0xc4,0xe2,0x51,0x52,0x34,0x6d,0x00,0xfe,0xff,0xff
# CHECK: {vex} vpdpwssd xmm6, xmm5, xmmword ptr [rcx + 2032]
0xc4,0xe2,0x51,0x52,0xb1,0xf0,0x07,0x00,0x00
# CHECK: {vex} vpdpwssd xmm6, xmm5, xmmword ptr [rdx - 2048]
0xc4,0xe2,0x51,0x52,0xb2,0x00,0xf8,0xff,0xff
# CHECK: {vex} vpdpwssds ymm6, ymm5, ymm4
0xc4,0xe2,0x55,0x53,0xf4
# CHECK: {vex} vpdpwssds xmm6, xmm5, xmm4
0xc4,0xe2,0x51,0x53,0xf4
# CHECK: {vex} vpdpwssds ymm6, ymm5, ymmword ptr [rbp + 8*r14 + 268435456]
0xc4,0xa2,0x55,0x53,0xb4,0xf5,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpwssds ymm6, ymm5, ymmword ptr [r8 + 4*rax + 291]
0xc4,0xc2,0x55,0x53,0xb4,0x80,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpwssds ymm6, ymm5, ymmword ptr [rip]
0xc4,0xe2,0x55,0x53,0x35,0x00,0x00,0x00,0x00
# CHECK: {vex} vpdpwssds ymm6, ymm5, ymmword ptr [2*rbp - 1024]
0xc4,0xe2,0x55,0x53,0x34,0x6d,0x00,0xfc,0xff,0xff
# CHECK: {vex} vpdpwssds ymm6, ymm5, ymmword ptr [rcx + 4064]
0xc4,0xe2,0x55,0x53,0xb1,0xe0,0x0f,0x00,0x00
# CHECK: {vex} vpdpwssds ymm6, ymm5, ymmword ptr [rdx - 4096]
0xc4,0xe2,0x55,0x53,0xb2,0x00,0xf0,0xff,0xff
# CHECK: {vex} vpdpwssds xmm6, xmm5, xmmword ptr [rbp + 8*r14 + 268435456]
0xc4,0xa2,0x51,0x53,0xb4,0xf5,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpwssds xmm6, xmm5, xmmword ptr [r8 + 4*rax + 291]
0xc4,0xc2,0x51,0x53,0xb4,0x80,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpwssds xmm6, xmm5, xmmword ptr [rip]
0xc4,0xe2,0x51,0x53,0x35,0x00,0x00,0x00,0x00
# CHECK: {vex} vpdpwssds xmm6, xmm5, xmmword ptr [2*rbp - 512]
0xc4,0xe2,0x51,0x53,0x34,0x6d,0x00,0xfe,0xff,0xff
# CHECK: {vex} vpdpwssds xmm6, xmm5, xmmword ptr [rcx + 2032]
0xc4,0xe2,0x51,0x53,0xb1,0xf0,0x07,0x00,0x00
# CHECK: {vex} vpdpwssds xmm6, xmm5, xmmword ptr [rdx - 2048]
0xc4,0xe2,0x51,0x53,0xb2,0x00,0xf8,0xff,0xff

llvm/test/MC/Disassembler/X86/x86-64-avx_vnni.txt

  • This file was added.
# RUN: llvm-mc --disassemble %s -triple=x86_64-apple-darwin9 | FileCheck %s
# CHECK: {vex} vpdpbusd %ymm4, %ymm5, %ymm6
0xc4,0xe2,0x55,0x50,0xf4
# CHECK: {vex} vpdpbusd %xmm4, %xmm5, %xmm6
0xc4,0xe2,0x51,0x50,0xf4
# CHECK: {vex} vpdpbusd 268435456(%rbp,%r14,8), %ymm5, %ymm6
0xc4,0xa2,0x55,0x50,0xb4,0xf5,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpbusd 291(%r8,%rax,4), %ymm5, %ymm6
0xc4,0xc2,0x55,0x50,0xb4,0x80,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpbusd (%rip), %ymm5, %ymm6
0xc4,0xe2,0x55,0x50,0x35,0x00,0x00,0x00,0x00
# CHECK: {vex} vpdpbusd -1024(,%rbp,2), %ymm5, %ymm6
0xc4,0xe2,0x55,0x50,0x34,0x6d,0x00,0xfc,0xff,0xff
# CHECK: {vex} vpdpbusd 4064(%rcx), %ymm5, %ymm6
0xc4,0xe2,0x55,0x50,0xb1,0xe0,0x0f,0x00,0x00
# CHECK: {vex} vpdpbusd -4096(%rdx), %ymm5, %ymm6
0xc4,0xe2,0x55,0x50,0xb2,0x00,0xf0,0xff,0xff
# CHECK: {vex} vpdpbusd 268435456(%rbp,%r14,8), %xmm5, %xmm6
0xc4,0xa2,0x51,0x50,0xb4,0xf5,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpbusd 291(%r8,%rax,4), %xmm5, %xmm6
0xc4,0xc2,0x51,0x50,0xb4,0x80,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpbusd (%rip), %xmm5, %xmm6
0xc4,0xe2,0x51,0x50,0x35,0x00,0x00,0x00,0x00
# CHECK: {vex} vpdpbusd -512(,%rbp,2), %xmm5, %xmm6
0xc4,0xe2,0x51,0x50,0x34,0x6d,0x00,0xfe,0xff,0xff
# CHECK: {vex} vpdpbusd 2032(%rcx), %xmm5, %xmm6
0xc4,0xe2,0x51,0x50,0xb1,0xf0,0x07,0x00,0x00
# CHECK: {vex} vpdpbusd -2048(%rdx), %xmm5, %xmm6
0xc4,0xe2,0x51,0x50,0xb2,0x00,0xf8,0xff,0xff
# CHECK: {vex} vpdpbusds %ymm4, %ymm5, %ymm6
0xc4,0xe2,0x55,0x51,0xf4
# CHECK: {vex} vpdpbusds %xmm4, %xmm5, %xmm6
0xc4,0xe2,0x51,0x51,0xf4
# CHECK: {vex} vpdpbusds 268435456(%rbp,%r14,8), %ymm5, %ymm6
0xc4,0xa2,0x55,0x51,0xb4,0xf5,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpbusds 291(%r8,%rax,4), %ymm5, %ymm6
0xc4,0xc2,0x55,0x51,0xb4,0x80,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpbusds (%rip), %ymm5, %ymm6
0xc4,0xe2,0x55,0x51,0x35,0x00,0x00,0x00,0x00
# CHECK: {vex} vpdpbusds -1024(,%rbp,2), %ymm5, %ymm6
0xc4,0xe2,0x55,0x51,0x34,0x6d,0x00,0xfc,0xff,0xff
# CHECK: {vex} vpdpbusds 4064(%rcx), %ymm5, %ymm6
0xc4,0xe2,0x55,0x51,0xb1,0xe0,0x0f,0x00,0x00
# CHECK: {vex} vpdpbusds -4096(%rdx), %ymm5, %ymm6
0xc4,0xe2,0x55,0x51,0xb2,0x00,0xf0,0xff,0xff
# CHECK: {vex} vpdpbusds 268435456(%rbp,%r14,8), %xmm5, %xmm6
0xc4,0xa2,0x51,0x51,0xb4,0xf5,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpbusds 291(%r8,%rax,4), %xmm5, %xmm6
0xc4,0xc2,0x51,0x51,0xb4,0x80,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpbusds (%rip), %xmm5, %xmm6
0xc4,0xe2,0x51,0x51,0x35,0x00,0x00,0x00,0x00
# CHECK: {vex} vpdpbusds -512(,%rbp,2), %xmm5, %xmm6
0xc4,0xe2,0x51,0x51,0x34,0x6d,0x00,0xfe,0xff,0xff
# CHECK: {vex} vpdpbusds 2032(%rcx), %xmm5, %xmm6
0xc4,0xe2,0x51,0x51,0xb1,0xf0,0x07,0x00,0x00
# CHECK: {vex} vpdpbusds -2048(%rdx), %xmm5, %xmm6
0xc4,0xe2,0x51,0x51,0xb2,0x00,0xf8,0xff,0xff
# CHECK: {vex} vpdpwssd %ymm4, %ymm5, %ymm6
0xc4,0xe2,0x55,0x52,0xf4
# CHECK: {vex} vpdpwssd %xmm4, %xmm5, %xmm6
0xc4,0xe2,0x51,0x52,0xf4
# CHECK: {vex} vpdpwssd 268435456(%rbp,%r14,8), %ymm5, %ymm6
0xc4,0xa2,0x55,0x52,0xb4,0xf5,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpwssd 291(%r8,%rax,4), %ymm5, %ymm6
0xc4,0xc2,0x55,0x52,0xb4,0x80,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpwssd (%rip), %ymm5, %ymm6
0xc4,0xe2,0x55,0x52,0x35,0x00,0x00,0x00,0x00
# CHECK: {vex} vpdpwssd -1024(,%rbp,2), %ymm5, %ymm6
0xc4,0xe2,0x55,0x52,0x34,0x6d,0x00,0xfc,0xff,0xff
# CHECK: {vex} vpdpwssd 4064(%rcx), %ymm5, %ymm6
0xc4,0xe2,0x55,0x52,0xb1,0xe0,0x0f,0x00,0x00
# CHECK: {vex} vpdpwssd -4096(%rdx), %ymm5, %ymm6
0xc4,0xe2,0x55,0x52,0xb2,0x00,0xf0,0xff,0xff
# CHECK: {vex} vpdpwssd 268435456(%rbp,%r14,8), %xmm5, %xmm6
0xc4,0xa2,0x51,0x52,0xb4,0xf5,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpwssd 291(%r8,%rax,4), %xmm5, %xmm6
0xc4,0xc2,0x51,0x52,0xb4,0x80,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpwssd (%rip), %xmm5, %xmm6
0xc4,0xe2,0x51,0x52,0x35,0x00,0x00,0x00,0x00
# CHECK: {vex} vpdpwssd -512(,%rbp,2), %xmm5, %xmm6
0xc4,0xe2,0x51,0x52,0x34,0x6d,0x00,0xfe,0xff,0xff
# CHECK: {vex} vpdpwssd 2032(%rcx), %xmm5, %xmm6
0xc4,0xe2,0x51,0x52,0xb1,0xf0,0x07,0x00,0x00
# CHECK: {vex} vpdpwssd -2048(%rdx), %xmm5, %xmm6
0xc4,0xe2,0x51,0x52,0xb2,0x00,0xf8,0xff,0xff
# CHECK: {vex} vpdpwssds %ymm4, %ymm5, %ymm6
0xc4,0xe2,0x55,0x53,0xf4
# CHECK: {vex} vpdpwssds %xmm4, %xmm5, %xmm6
0xc4,0xe2,0x51,0x53,0xf4
# CHECK: {vex} vpdpwssds 268435456(%rbp,%r14,8), %ymm5, %ymm6
0xc4,0xa2,0x55,0x53,0xb4,0xf5,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpwssds 291(%r8,%rax,4), %ymm5, %ymm6
0xc4,0xc2,0x55,0x53,0xb4,0x80,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpwssds (%rip), %ymm5, %ymm6
0xc4,0xe2,0x55,0x53,0x35,0x00,0x00,0x00,0x00
# CHECK: {vex} vpdpwssds -1024(,%rbp,2), %ymm5, %ymm6
0xc4,0xe2,0x55,0x53,0x34,0x6d,0x00,0xfc,0xff,0xff
# CHECK: {vex} vpdpwssds 4064(%rcx), %ymm5, %ymm6
0xc4,0xe2,0x55,0x53,0xb1,0xe0,0x0f,0x00,0x00
# CHECK: {vex} vpdpwssds -4096(%rdx), %ymm5, %ymm6
0xc4,0xe2,0x55,0x53,0xb2,0x00,0xf0,0xff,0xff
# CHECK: {vex} vpdpwssds 268435456(%rbp,%r14,8), %xmm5, %xmm6
0xc4,0xa2,0x51,0x53,0xb4,0xf5,0x00,0x00,0x00,0x10
# CHECK: {vex} vpdpwssds 291(%r8,%rax,4), %xmm5, %xmm6
0xc4,0xc2,0x51,0x53,0xb4,0x80,0x23,0x01,0x00,0x00
# CHECK: {vex} vpdpwssds (%rip), %xmm5, %xmm6
0xc4,0xe2,0x51,0x53,0x35,0x00,0x00,0x00,0x00
# CHECK: {vex} vpdpwssds -512(,%rbp,2), %xmm5, %xmm6
0xc4,0xe2,0x51,0x53,0x34,0x6d,0x00,0xfe,0xff,0xff
# CHECK: {vex} vpdpwssds 2032(%rcx), %xmm5, %xmm6
0xc4,0xe2,0x51,0x53,0xb1,0xf0,0x07,0x00,0x00
# CHECK: {vex} vpdpwssds -2048(%rdx), %xmm5, %xmm6
0xc4,0xe2,0x51,0x53,0xb2,0x00,0xf8,0xff,0xff

llvm/test/MC/X86/avx_vnni-encoding.s

  • This file was added.
// RUN: llvm-mc -triple i686-unknown-unknown -mattr=+avxvnni --show-encoding < %s | FileCheck %s
// CHECK: {vex} vpdpbusd %ymm4, %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x50,0xf4]
{vex} vpdpbusd %ymm4, %ymm5, %ymm6
// CHECK: {vex} vpdpbusd %xmm4, %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x50,0xf4]
{vex} vpdpbusd %xmm4, %xmm5, %xmm6
// CHECK: {vex} vpdpbusd 268435456(%esp,%esi,8), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x50,0xb4,0xf4,0x00,0x00,0x00,0x10]
{vex} vpdpbusd 268435456(%esp,%esi,8), %ymm5, %ymm6
// CHECK: {vex} vpdpbusd 291(%edi,%eax,4), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x50,0xb4,0x87,0x23,0x01,0x00,0x00]
{vex} vpdpbusd 291(%edi,%eax,4), %ymm5, %ymm6
// CHECK: {vex} vpdpbusd (%eax), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x50,0x30]
{vex} vpdpbusd (%eax), %ymm5, %ymm6
// CHECK: {vex} vpdpbusd -1024(,%ebp,2), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x50,0x34,0x6d,0x00,0xfc,0xff,0xff]
{vex} vpdpbusd -1024(,%ebp,2), %ymm5, %ymm6
// CHECK: {vex} vpdpbusd 4064(%ecx), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x50,0xb1,0xe0,0x0f,0x00,0x00]
{vex} vpdpbusd 4064(%ecx), %ymm5, %ymm6
// CHECK: {vex} vpdpbusd -4096(%edx), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x50,0xb2,0x00,0xf0,0xff,0xff]
{vex} vpdpbusd -4096(%edx), %ymm5, %ymm6
// CHECK: {vex} vpdpbusd 268435456(%esp,%esi,8), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x50,0xb4,0xf4,0x00,0x00,0x00,0x10]
{vex} vpdpbusd 268435456(%esp,%esi,8), %xmm5, %xmm6
// CHECK: {vex} vpdpbusd 291(%edi,%eax,4), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x50,0xb4,0x87,0x23,0x01,0x00,0x00]
{vex} vpdpbusd 291(%edi,%eax,4), %xmm5, %xmm6
// CHECK: {vex} vpdpbusd (%eax), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x50,0x30]
{vex} vpdpbusd (%eax), %xmm5, %xmm6
// CHECK: {vex} vpdpbusd -512(,%ebp,2), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x50,0x34,0x6d,0x00,0xfe,0xff,0xff]
{vex} vpdpbusd -512(,%ebp,2), %xmm5, %xmm6
// CHECK: {vex} vpdpbusd 2032(%ecx), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x50,0xb1,0xf0,0x07,0x00,0x00]
{vex} vpdpbusd 2032(%ecx), %xmm5, %xmm6
// CHECK: {vex} vpdpbusd -2048(%edx), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x50,0xb2,0x00,0xf8,0xff,0xff]
{vex} vpdpbusd -2048(%edx), %xmm5, %xmm6
// CHECK: {vex} vpdpbusds %ymm4, %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x51,0xf4]
{vex} vpdpbusds %ymm4, %ymm5, %ymm6
// CHECK: {vex} vpdpbusds %xmm4, %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x51,0xf4]
{vex} vpdpbusds %xmm4, %xmm5, %xmm6
// CHECK: {vex} vpdpbusds 268435456(%esp,%esi,8), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x51,0xb4,0xf4,0x00,0x00,0x00,0x10]
{vex} vpdpbusds 268435456(%esp,%esi,8), %ymm5, %ymm6
// CHECK: {vex} vpdpbusds 291(%edi,%eax,4), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x51,0xb4,0x87,0x23,0x01,0x00,0x00]
{vex} vpdpbusds 291(%edi,%eax,4), %ymm5, %ymm6
// CHECK: {vex} vpdpbusds (%eax), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x51,0x30]
{vex} vpdpbusds (%eax), %ymm5, %ymm6
// CHECK: {vex} vpdpbusds -1024(,%ebp,2), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x51,0x34,0x6d,0x00,0xfc,0xff,0xff]
{vex} vpdpbusds -1024(,%ebp,2), %ymm5, %ymm6
// CHECK: {vex} vpdpbusds 4064(%ecx), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x51,0xb1,0xe0,0x0f,0x00,0x00]
{vex} vpdpbusds 4064(%ecx), %ymm5, %ymm6
// CHECK: {vex} vpdpbusds -4096(%edx), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x51,0xb2,0x00,0xf0,0xff,0xff]
{vex} vpdpbusds -4096(%edx), %ymm5, %ymm6
// CHECK: {vex} vpdpbusds 268435456(%esp,%esi,8), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x51,0xb4,0xf4,0x00,0x00,0x00,0x10]
{vex} vpdpbusds 268435456(%esp,%esi,8), %xmm5, %xmm6
// CHECK: {vex} vpdpbusds 291(%edi,%eax,4), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x51,0xb4,0x87,0x23,0x01,0x00,0x00]
{vex} vpdpbusds 291(%edi,%eax,4), %xmm5, %xmm6
// CHECK: {vex} vpdpbusds (%eax), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x51,0x30]
{vex} vpdpbusds (%eax), %xmm5, %xmm6
// CHECK: {vex} vpdpbusds -512(,%ebp,2), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x51,0x34,0x6d,0x00,0xfe,0xff,0xff]
{vex} vpdpbusds -512(,%ebp,2), %xmm5, %xmm6
// CHECK: {vex} vpdpbusds 2032(%ecx), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x51,0xb1,0xf0,0x07,0x00,0x00]
{vex} vpdpbusds 2032(%ecx), %xmm5, %xmm6
// CHECK: {vex} vpdpbusds -2048(%edx), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x51,0xb2,0x00,0xf8,0xff,0xff]
{vex} vpdpbusds -2048(%edx), %xmm5, %xmm6
// CHECK: vpdpwssd %ymm4, %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x52,0xf4]
{vex} vpdpwssd %ymm4, %ymm5, %ymm6
// CHECK: vpdpwssd %xmm4, %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x52,0xf4]
{vex} vpdpwssd %xmm4, %xmm5, %xmm6
// CHECK: vpdpwssd 268435456(%esp,%esi,8), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x52,0xb4,0xf4,0x00,0x00,0x00,0x10]
{vex} vpdpwssd 268435456(%esp,%esi,8), %ymm5, %ymm6
// CHECK: vpdpwssd 291(%edi,%eax,4), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x52,0xb4,0x87,0x23,0x01,0x00,0x00]
{vex} vpdpwssd 291(%edi,%eax,4), %ymm5, %ymm6
// CHECK: vpdpwssd (%eax), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x52,0x30]
{vex} vpdpwssd (%eax), %ymm5, %ymm6
// CHECK: vpdpwssd -1024(,%ebp,2), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x52,0x34,0x6d,0x00,0xfc,0xff,0xff]
{vex} vpdpwssd -1024(,%ebp,2), %ymm5, %ymm6
// CHECK: vpdpwssd 4064(%ecx), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x52,0xb1,0xe0,0x0f,0x00,0x00]
{vex} vpdpwssd 4064(%ecx), %ymm5, %ymm6
// CHECK: vpdpwssd -4096(%edx), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x52,0xb2,0x00,0xf0,0xff,0xff]
{vex} vpdpwssd -4096(%edx), %ymm5, %ymm6
// CHECK: vpdpwssd 268435456(%esp,%esi,8), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x52,0xb4,0xf4,0x00,0x00,0x00,0x10]
{vex} vpdpwssd 268435456(%esp,%esi,8), %xmm5, %xmm6
// CHECK: vpdpwssd 291(%edi,%eax,4), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x52,0xb4,0x87,0x23,0x01,0x00,0x00]
{vex} vpdpwssd 291(%edi,%eax,4), %xmm5, %xmm6
// CHECK: vpdpwssd (%eax), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x52,0x30]
{vex} vpdpwssd (%eax), %xmm5, %xmm6
// CHECK: vpdpwssd -512(,%ebp,2), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x52,0x34,0x6d,0x00,0xfe,0xff,0xff]
{vex} vpdpwssd -512(,%ebp,2), %xmm5, %xmm6
// CHECK: vpdpwssd 2032(%ecx), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x52,0xb1,0xf0,0x07,0x00,0x00]
{vex} vpdpwssd 2032(%ecx), %xmm5, %xmm6
// CHECK: vpdpwssd -2048(%edx), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x52,0xb2,0x00,0xf8,0xff,0xff]
{vex} vpdpwssd -2048(%edx), %xmm5, %xmm6
// CHECK: vpdpwssds %ymm4, %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x53,0xf4]
{vex} vpdpwssds %ymm4, %ymm5, %ymm6
// CHECK: vpdpwssds %xmm4, %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x53,0xf4]
{vex} vpdpwssds %xmm4, %xmm5, %xmm6
// CHECK: vpdpwssds 268435456(%esp,%esi,8), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x53,0xb4,0xf4,0x00,0x00,0x00,0x10]
{vex} vpdpwssds 268435456(%esp,%esi,8), %ymm5, %ymm6
// CHECK: vpdpwssds 291(%edi,%eax,4), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x53,0xb4,0x87,0x23,0x01,0x00,0x00]
{vex} vpdpwssds 291(%edi,%eax,4), %ymm5, %ymm6
// CHECK: vpdpwssds (%eax), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x53,0x30]
{vex} vpdpwssds (%eax), %ymm5, %ymm6
// CHECK: vpdpwssds -1024(,%ebp,2), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x53,0x34,0x6d,0x00,0xfc,0xff,0xff]
{vex} vpdpwssds -1024(,%ebp,2), %ymm5, %ymm6
// CHECK: vpdpwssds 4064(%ecx), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x53,0xb1,0xe0,0x0f,0x00,0x00]
{vex} vpdpwssds 4064(%ecx), %ymm5, %ymm6
// CHECK: vpdpwssds -4096(%edx), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x53,0xb2,0x00,0xf0,0xff,0xff]
{vex} vpdpwssds -4096(%edx), %ymm5, %ymm6
// CHECK: vpdpwssds 268435456(%esp,%esi,8), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x53,0xb4,0xf4,0x00,0x00,0x00,0x10]
{vex} vpdpwssds 268435456(%esp,%esi,8), %xmm5, %xmm6
// CHECK: vpdpwssds 291(%edi,%eax,4), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x53,0xb4,0x87,0x23,0x01,0x00,0x00]
{vex} vpdpwssds 291(%edi,%eax,4), %xmm5, %xmm6
// CHECK: vpdpwssds (%eax), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x53,0x30]
{vex} vpdpwssds (%eax), %xmm5, %xmm6
// CHECK: vpdpwssds -512(,%ebp,2), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x53,0x34,0x6d,0x00,0xfe,0xff,0xff]
{vex} vpdpwssds -512(,%ebp,2), %xmm5, %xmm6
// CHECK: vpdpwssds 2032(%ecx), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x53,0xb1,0xf0,0x07,0x00,0x00]
{vex} vpdpwssds 2032(%ecx), %xmm5, %xmm6
// CHECK: vpdpwssds -2048(%edx), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x53,0xb2,0x00,0xf8,0xff,0xff]
{vex} vpdpwssds -2048(%edx), %xmm5, %xmm6

llvm/test/MC/X86/intel-syntax-avx_vnni.s

  • This file was added.
// RUN: llvm-mc -triple i686-unknown-unknown -mattr=+avxvnni -x86-asm-syntax=intel -output-asm-variant=1 --show-encoding %s | FileCheck %s
// CHECK: {vex} vpdpbusd ymm6, ymm5, ymm4
// CHECK: encoding: [0xc4,0xe2,0x55,0x50,0xf4]
{vex} vpdpbusd ymm6, ymm5, ymm4
// CHECK: {vex} vpdpbusd xmm6, xmm5, xmm4
// CHECK: encoding: [0xc4,0xe2,0x51,0x50,0xf4]
{vex} vpdpbusd xmm6, xmm5, xmm4
// CHECK: {vex} vpdpbusd ymm6, ymm5, ymmword ptr [esp + 8*esi + 268435456]
// CHECK: encoding: [0xc4,0xe2,0x55,0x50,0xb4,0xf4,0x00,0x00,0x00,0x10]
{vex} vpdpbusd ymm6, ymm5, ymmword ptr [esp + 8*esi + 268435456]
// CHECK: {vex} vpdpbusd ymm6, ymm5, ymmword ptr [edi + 4*eax + 291]
// CHECK: encoding: [0xc4,0xe2,0x55,0x50,0xb4,0x87,0x23,0x01,0x00,0x00]
{vex} vpdpbusd ymm6, ymm5, ymmword ptr [edi + 4*eax + 291]
// CHECK: {vex} vpdpbusd ymm6, ymm5, ymmword ptr [eax]
// CHECK: encoding: [0xc4,0xe2,0x55,0x50,0x30]
{vex} vpdpbusd ymm6, ymm5, ymmword ptr [eax]
// CHECK: {vex} vpdpbusd ymm6, ymm5, ymmword ptr [2*ebp - 1024]
// CHECK: encoding: [0xc4,0xe2,0x55,0x50,0x34,0x6d,0x00,0xfc,0xff,0xff]
{vex} vpdpbusd ymm6, ymm5, ymmword ptr [2*ebp - 1024]
// CHECK: {vex} vpdpbusd ymm6, ymm5, ymmword ptr [ecx + 4064]
// CHECK: encoding: [0xc4,0xe2,0x55,0x50,0xb1,0xe0,0x0f,0x00,0x00]
{vex} vpdpbusd ymm6, ymm5, ymmword ptr [ecx + 4064]
// CHECK: {vex} vpdpbusd ymm6, ymm5, ymmword ptr [edx - 4096]
// CHECK: encoding: [0xc4,0xe2,0x55,0x50,0xb2,0x00,0xf0,0xff,0xff]
{vex} vpdpbusd ymm6, ymm5, ymmword ptr [edx - 4096]
// CHECK: {vex} vpdpbusd xmm6, xmm5, xmmword ptr [esp + 8*esi + 268435456]
// CHECK: encoding: [0xc4,0xe2,0x51,0x50,0xb4,0xf4,0x00,0x00,0x00,0x10]
{vex} vpdpbusd xmm6, xmm5, xmmword ptr [esp + 8*esi + 268435456]
// CHECK: {vex} vpdpbusd xmm6, xmm5, xmmword ptr [edi + 4*eax + 291]
// CHECK: encoding: [0xc4,0xe2,0x51,0x50,0xb4,0x87,0x23,0x01,0x00,0x00]
{vex} vpdpbusd xmm6, xmm5, xmmword ptr [edi + 4*eax + 291]
// CHECK: {vex} vpdpbusd xmm6, xmm5, xmmword ptr [eax]
// CHECK: encoding: [0xc4,0xe2,0x51,0x50,0x30]
{vex} vpdpbusd xmm6, xmm5, xmmword ptr [eax]
// CHECK: {vex} vpdpbusd xmm6, xmm5, xmmword ptr [2*ebp - 512]
// CHECK: encoding: [0xc4,0xe2,0x51,0x50,0x34,0x6d,0x00,0xfe,0xff,0xff]
{vex} vpdpbusd xmm6, xmm5, xmmword ptr [2*ebp - 512]
// CHECK: {vex} vpdpbusd xmm6, xmm5, xmmword ptr [ecx + 2032]
// CHECK: encoding: [0xc4,0xe2,0x51,0x50,0xb1,0xf0,0x07,0x00,0x00]
{vex} vpdpbusd xmm6, xmm5, xmmword ptr [ecx + 2032]
// CHECK: {vex} vpdpbusd xmm6, xmm5, xmmword ptr [edx - 2048]
// CHECK: encoding: [0xc4,0xe2,0x51,0x50,0xb2,0x00,0xf8,0xff,0xff]
{vex} vpdpbusd xmm6, xmm5, xmmword ptr [edx - 2048]
// CHECK: {vex} vpdpbusds ymm6, ymm5, ymm4
// CHECK: encoding: [0xc4,0xe2,0x55,0x51,0xf4]
{vex} vpdpbusds ymm6, ymm5, ymm4
// CHECK: {vex} vpdpbusds xmm6, xmm5, xmm4
// CHECK: encoding: [0xc4,0xe2,0x51,0x51,0xf4]
{vex} vpdpbusds xmm6, xmm5, xmm4
// CHECK: {vex} vpdpbusds ymm6, ymm5, ymmword ptr [esp + 8*esi + 268435456]
// CHECK: encoding: [0xc4,0xe2,0x55,0x51,0xb4,0xf4,0x00,0x00,0x00,0x10]
{vex} vpdpbusds ymm6, ymm5, ymmword ptr [esp + 8*esi + 268435456]
// CHECK: {vex} vpdpbusds ymm6, ymm5, ymmword ptr [edi + 4*eax + 291]
// CHECK: encoding: [0xc4,0xe2,0x55,0x51,0xb4,0x87,0x23,0x01,0x00,0x00]
{vex} vpdpbusds ymm6, ymm5, ymmword ptr [edi + 4*eax + 291]
// CHECK: {vex} vpdpbusds ymm6, ymm5, ymmword ptr [eax]
// CHECK: encoding: [0xc4,0xe2,0x55,0x51,0x30]
{vex} vpdpbusds ymm6, ymm5, ymmword ptr [eax]
// CHECK: {vex} vpdpbusds ymm6, ymm5, ymmword ptr [2*ebp - 1024]
// CHECK: encoding: [0xc4,0xe2,0x55,0x51,0x34,0x6d,0x00,0xfc,0xff,0xff]
{vex} vpdpbusds ymm6, ymm5, ymmword ptr [2*ebp - 1024]
// CHECK: {vex} vpdpbusds ymm6, ymm5, ymmword ptr [ecx + 4064]
// CHECK: encoding: [0xc4,0xe2,0x55,0x51,0xb1,0xe0,0x0f,0x00,0x00]
{vex} vpdpbusds ymm6, ymm5, ymmword ptr [ecx + 4064]
// CHECK: {vex} vpdpbusds ymm6, ymm5, ymmword ptr [edx - 4096]
// CHECK: encoding: [0xc4,0xe2,0x55,0x51,0xb2,0x00,0xf0,0xff,0xff]
{vex} vpdpbusds ymm6, ymm5, ymmword ptr [edx - 4096]
// CHECK: {vex} vpdpbusds xmm6, xmm5, xmmword ptr [esp + 8*esi + 268435456]
// CHECK: encoding: [0xc4,0xe2,0x51,0x51,0xb4,0xf4,0x00,0x00,0x00,0x10]
{vex} vpdpbusds xmm6, xmm5, xmmword ptr [esp + 8*esi + 268435456]
// CHECK: {vex} vpdpbusds xmm6, xmm5, xmmword ptr [edi + 4*eax + 291]
// CHECK: encoding: [0xc4,0xe2,0x51,0x51,0xb4,0x87,0x23,0x01,0x00,0x00]
{vex} vpdpbusds xmm6, xmm5, xmmword ptr [edi + 4*eax + 291]
// CHECK: {vex} vpdpbusds xmm6, xmm5, xmmword ptr [eax]
// CHECK: encoding: [0xc4,0xe2,0x51,0x51,0x30]
{vex} vpdpbusds xmm6, xmm5, xmmword ptr [eax]
// CHECK: {vex} vpdpbusds xmm6, xmm5, xmmword ptr [2*ebp - 512]
// CHECK: encoding: [0xc4,0xe2,0x51,0x51,0x34,0x6d,0x00,0xfe,0xff,0xff]
{vex} vpdpbusds xmm6, xmm5, xmmword ptr [2*ebp - 512]
// CHECK: {vex} vpdpbusds xmm6, xmm5, xmmword ptr [ecx + 2032]
// CHECK: encoding: [0xc4,0xe2,0x51,0x51,0xb1,0xf0,0x07,0x00,0x00]
{vex} vpdpbusds xmm6, xmm5, xmmword ptr [ecx + 2032]
// CHECK: {vex} vpdpbusds xmm6, xmm5, xmmword ptr [edx - 2048]
// CHECK: encoding: [0xc4,0xe2,0x51,0x51,0xb2,0x00,0xf8,0xff,0xff]
{vex} vpdpbusds xmm6, xmm5, xmmword ptr [edx - 2048]
// CHECK: {vex} vpdpwssd ymm6, ymm5, ymm4
// CHECK: encoding: [0xc4,0xe2,0x55,0x52,0xf4]
{vex} vpdpwssd ymm6, ymm5, ymm4
// CHECK: {vex} vpdpwssd xmm6, xmm5, xmm4
// CHECK: encoding: [0xc4,0xe2,0x51,0x52,0xf4]
{vex} vpdpwssd xmm6, xmm5, xmm4
// CHECK: {vex} vpdpwssd ymm6, ymm5, ymmword ptr [esp + 8*esi + 268435456]
// CHECK: encoding: [0xc4,0xe2,0x55,0x52,0xb4,0xf4,0x00,0x00,0x00,0x10]
{vex} vpdpwssd ymm6, ymm5, ymmword ptr [esp + 8*esi + 268435456]
// CHECK: {vex} vpdpwssd ymm6, ymm5, ymmword ptr [edi + 4*eax + 291]
// CHECK: encoding: [0xc4,0xe2,0x55,0x52,0xb4,0x87,0x23,0x01,0x00,0x00]
{vex} vpdpwssd ymm6, ymm5, ymmword ptr [edi + 4*eax + 291]
// CHECK: {vex} vpdpwssd ymm6, ymm5, ymmword ptr [eax]
// CHECK: encoding: [0xc4,0xe2,0x55,0x52,0x30]
{vex} vpdpwssd ymm6, ymm5, ymmword ptr [eax]
// CHECK: {vex} vpdpwssd ymm6, ymm5, ymmword ptr [2*ebp - 1024]
// CHECK: encoding: [0xc4,0xe2,0x55,0x52,0x34,0x6d,0x00,0xfc,0xff,0xff]
{vex} vpdpwssd ymm6, ymm5, ymmword ptr [2*ebp - 1024]
// CHECK: {vex} vpdpwssd ymm6, ymm5, ymmword ptr [ecx + 4064]
// CHECK: encoding: [0xc4,0xe2,0x55,0x52,0xb1,0xe0,0x0f,0x00,0x00]
{vex} vpdpwssd ymm6, ymm5, ymmword ptr [ecx + 4064]
// CHECK: {vex} vpdpwssd ymm6, ymm5, ymmword ptr [edx - 4096]
// CHECK: encoding: [0xc4,0xe2,0x55,0x52,0xb2,0x00,0xf0,0xff,0xff]
{vex} vpdpwssd ymm6, ymm5, ymmword ptr [edx - 4096]
// CHECK: {vex} vpdpwssd xmm6, xmm5, xmmword ptr [esp + 8*esi + 268435456]
// CHECK: encoding: [0xc4,0xe2,0x51,0x52,0xb4,0xf4,0x00,0x00,0x00,0x10]
{vex} vpdpwssd xmm6, xmm5, xmmword ptr [esp + 8*esi + 268435456]
// CHECK: {vex} vpdpwssd xmm6, xmm5, xmmword ptr [edi + 4*eax + 291]
// CHECK: encoding: [0xc4,0xe2,0x51,0x52,0xb4,0x87,0x23,0x01,0x00,0x00]
{vex} vpdpwssd xmm6, xmm5, xmmword ptr [edi + 4*eax + 291]
// CHECK: {vex} vpdpwssd xmm6, xmm5, xmmword ptr [eax]
// CHECK: encoding: [0xc4,0xe2,0x51,0x52,0x30]
{vex} vpdpwssd xmm6, xmm5, xmmword ptr [eax]
// CHECK: {vex} vpdpwssd xmm6, xmm5, xmmword ptr [2*ebp - 512]
// CHECK: encoding: [0xc4,0xe2,0x51,0x52,0x34,0x6d,0x00,0xfe,0xff,0xff]
{vex} vpdpwssd xmm6, xmm5, xmmword ptr [2*ebp - 512]
// CHECK: {vex} vpdpwssd xmm6, xmm5, xmmword ptr [ecx + 2032]
// CHECK: encoding: [0xc4,0xe2,0x51,0x52,0xb1,0xf0,0x07,0x00,0x00]
{vex} vpdpwssd xmm6, xmm5, xmmword ptr [ecx + 2032]
// CHECK: {vex} vpdpwssd xmm6, xmm5, xmmword ptr [edx - 2048]
// CHECK: encoding: [0xc4,0xe2,0x51,0x52,0xb2,0x00,0xf8,0xff,0xff]
{vex} vpdpwssd xmm6, xmm5, xmmword ptr [edx - 2048]
// CHECK: {vex} vpdpwssds ymm6, ymm5, ymm4
// CHECK: encoding: [0xc4,0xe2,0x55,0x53,0xf4]
{vex} vpdpwssds ymm6, ymm5, ymm4
// CHECK: {vex} vpdpwssds xmm6, xmm5, xmm4
// CHECK: encoding: [0xc4,0xe2,0x51,0x53,0xf4]
{vex} vpdpwssds xmm6, xmm5, xmm4
// CHECK: {vex} vpdpwssds ymm6, ymm5, ymmword ptr [esp + 8*esi + 268435456]
// CHECK: encoding: [0xc4,0xe2,0x55,0x53,0xb4,0xf4,0x00,0x00,0x00,0x10]
{vex} vpdpwssds ymm6, ymm5, ymmword ptr [esp + 8*esi + 268435456]
// CHECK: {vex} vpdpwssds ymm6, ymm5, ymmword ptr [edi + 4*eax + 291]
// CHECK: encoding: [0xc4,0xe2,0x55,0x53,0xb4,0x87,0x23,0x01,0x00,0x00]
{vex} vpdpwssds ymm6, ymm5, ymmword ptr [edi + 4*eax + 291]
// CHECK: {vex} vpdpwssds ymm6, ymm5, ymmword ptr [eax]
// CHECK: encoding: [0xc4,0xe2,0x55,0x53,0x30]
{vex} vpdpwssds ymm6, ymm5, ymmword ptr [eax]
// CHECK: {vex} vpdpwssds ymm6, ymm5, ymmword ptr [2*ebp - 1024]
// CHECK: encoding: [0xc4,0xe2,0x55,0x53,0x34,0x6d,0x00,0xfc,0xff,0xff]
{vex} vpdpwssds ymm6, ymm5, ymmword ptr [2*ebp - 1024]
// CHECK: {vex} vpdpwssds ymm6, ymm5, ymmword ptr [ecx + 4064]
// CHECK: encoding: [0xc4,0xe2,0x55,0x53,0xb1,0xe0,0x0f,0x00,0x00]
{vex} vpdpwssds ymm6, ymm5, ymmword ptr [ecx + 4064]
// CHECK: {vex} vpdpwssds ymm6, ymm5, ymmword ptr [edx - 4096]
// CHECK: encoding: [0xc4,0xe2,0x55,0x53,0xb2,0x00,0xf0,0xff,0xff]
{vex} vpdpwssds ymm6, ymm5, ymmword ptr [edx - 4096]
// CHECK: {vex} vpdpwssds xmm6, xmm5, xmmword ptr [esp + 8*esi + 268435456]
// CHECK: encoding: [0xc4,0xe2,0x51,0x53,0xb4,0xf4,0x00,0x00,0x00,0x10]
{vex} vpdpwssds xmm6, xmm5, xmmword ptr [esp + 8*esi + 268435456]
// CHECK: {vex} vpdpwssds xmm6, xmm5, xmmword ptr [edi + 4*eax + 291]
// CHECK: encoding: [0xc4,0xe2,0x51,0x53,0xb4,0x87,0x23,0x01,0x00,0x00]
{vex} vpdpwssds xmm6, xmm5, xmmword ptr [edi + 4*eax + 291]
// CHECK: {vex} vpdpwssds xmm6, xmm5, xmmword ptr [eax]
// CHECK: encoding: [0xc4,0xe2,0x51,0x53,0x30]
{vex} vpdpwssds xmm6, xmm5, xmmword ptr [eax]
// CHECK: {vex} vpdpwssds xmm6, xmm5, xmmword ptr [2*ebp - 512]
// CHECK: encoding: [0xc4,0xe2,0x51,0x53,0x34,0x6d,0x00,0xfe,0xff,0xff]
{vex} vpdpwssds xmm6, xmm5, xmmword ptr [2*ebp - 512]
// CHECK: {vex} vpdpwssds xmm6, xmm5, xmmword ptr [ecx + 2032]
// CHECK: encoding: [0xc4,0xe2,0x51,0x53,0xb1,0xf0,0x07,0x00,0x00]
{vex} vpdpwssds xmm6, xmm5, xmmword ptr [ecx + 2032]
// CHECK: {vex} vpdpwssds xmm6, xmm5, xmmword ptr [edx - 2048]
// CHECK: encoding: [0xc4,0xe2,0x51,0x53,0xb2,0x00,0xf8,0xff,0xff]
{vex} vpdpwssds xmm6, xmm5, xmmword ptr [edx - 2048]

llvm/test/MC/X86/intel-syntax-x86-64-avx_vnni.s

  • This file was added.
// RUN: llvm-mc -triple x86_64-unknown-unknown -mattr=+avxvnni -x86-asm-syntax=intel -output-asm-variant=1 --show-encoding %s | FileCheck %s
// CHECK: {vex} vpdpbusd ymm6, ymm5, ymm4
// CHECK: encoding: [0xc4,0xe2,0x55,0x50,0xf4]
{vex} vpdpbusd ymm6, ymm5, ymm4
// CHECK: {vex} vpdpbusd xmm6, xmm5, xmm4
// CHECK: encoding: [0xc4,0xe2,0x51,0x50,0xf4]
{vex} vpdpbusd xmm6, xmm5, xmm4
// CHECK: {vex} vpdpbusd ymm6, ymm5, ymmword ptr [rbp + 8*r14 + 268435456]
// CHECK: encoding: [0xc4,0xa2,0x55,0x50,0xb4,0xf5,0x00,0x00,0x00,0x10]
{vex} vpdpbusd ymm6, ymm5, ymmword ptr [rbp + 8*r14 + 268435456]
// CHECK: {vex} vpdpbusd ymm6, ymm5, ymmword ptr [r8 + 4*rax + 291]
// CHECK: encoding: [0xc4,0xc2,0x55,0x50,0xb4,0x80,0x23,0x01,0x00,0x00]
{vex} vpdpbusd ymm6, ymm5, ymmword ptr [r8 + 4*rax + 291]
// CHECK: {vex} vpdpbusd ymm6, ymm5, ymmword ptr [rip]
// CHECK: encoding: [0xc4,0xe2,0x55,0x50,0x35,0x00,0x00,0x00,0x00]
{vex} vpdpbusd ymm6, ymm5, ymmword ptr [rip]
// CHECK: {vex} vpdpbusd ymm6, ymm5, ymmword ptr [2*rbp - 1024]
// CHECK: encoding: [0xc4,0xe2,0x55,0x50,0x34,0x6d,0x00,0xfc,0xff,0xff]
{vex} vpdpbusd ymm6, ymm5, ymmword ptr [2*rbp - 1024]
// CHECK: {vex} vpdpbusd ymm6, ymm5, ymmword ptr [rcx + 4064]
// CHECK: encoding: [0xc4,0xe2,0x55,0x50,0xb1,0xe0,0x0f,0x00,0x00]
{vex} vpdpbusd ymm6, ymm5, ymmword ptr [rcx + 4064]
// CHECK: {vex} vpdpbusd ymm6, ymm5, ymmword ptr [rdx - 4096]
// CHECK: encoding: [0xc4,0xe2,0x55,0x50,0xb2,0x00,0xf0,0xff,0xff]
{vex} vpdpbusd ymm6, ymm5, ymmword ptr [rdx - 4096]
// CHECK: {vex} vpdpbusd xmm6, xmm5, xmmword ptr [rbp + 8*r14 + 268435456]
// CHECK: encoding: [0xc4,0xa2,0x51,0x50,0xb4,0xf5,0x00,0x00,0x00,0x10]
{vex} vpdpbusd xmm6, xmm5, xmmword ptr [rbp + 8*r14 + 268435456]
// CHECK: {vex} vpdpbusd xmm6, xmm5, xmmword ptr [r8 + 4*rax + 291]
// CHECK: encoding: [0xc4,0xc2,0x51,0x50,0xb4,0x80,0x23,0x01,0x00,0x00]
{vex} vpdpbusd xmm6, xmm5, xmmword ptr [r8 + 4*rax + 291]
// CHECK: {vex} vpdpbusd xmm6, xmm5, xmmword ptr [rip]
// CHECK: encoding: [0xc4,0xe2,0x51,0x50,0x35,0x00,0x00,0x00,0x00]
{vex} vpdpbusd xmm6, xmm5, xmmword ptr [rip]
// CHECK: {vex} vpdpbusd xmm6, xmm5, xmmword ptr [2*rbp - 512]
// CHECK: encoding: [0xc4,0xe2,0x51,0x50,0x34,0x6d,0x00,0xfe,0xff,0xff]
{vex} vpdpbusd xmm6, xmm5, xmmword ptr [2*rbp - 512]
// CHECK: {vex} vpdpbusd xmm6, xmm5, xmmword ptr [rcx + 2032]
// CHECK: encoding: [0xc4,0xe2,0x51,0x50,0xb1,0xf0,0x07,0x00,0x00]
{vex} vpdpbusd xmm6, xmm5, xmmword ptr [rcx + 2032]
// CHECK: {vex} vpdpbusd xmm6, xmm5, xmmword ptr [rdx - 2048]
// CHECK: encoding: [0xc4,0xe2,0x51,0x50,0xb2,0x00,0xf8,0xff,0xff]
{vex} vpdpbusd xmm6, xmm5, xmmword ptr [rdx - 2048]
// CHECK: {vex} vpdpbusds ymm6, ymm5, ymm4
// CHECK: encoding: [0xc4,0xe2,0x55,0x51,0xf4]
{vex} vpdpbusds ymm6, ymm5, ymm4
// CHECK: {vex} vpdpbusds xmm6, xmm5, xmm4
// CHECK: encoding: [0xc4,0xe2,0x51,0x51,0xf4]
{vex} vpdpbusds xmm6, xmm5, xmm4
// CHECK: {vex} vpdpbusds ymm6, ymm5, ymmword ptr [rbp + 8*r14 + 268435456]
// CHECK: encoding: [0xc4,0xa2,0x55,0x51,0xb4,0xf5,0x00,0x00,0x00,0x10]
{vex} vpdpbusds ymm6, ymm5, ymmword ptr [rbp + 8*r14 + 268435456]
// CHECK: {vex} vpdpbusds ymm6, ymm5, ymmword ptr [r8 + 4*rax + 291]
// CHECK: encoding: [0xc4,0xc2,0x55,0x51,0xb4,0x80,0x23,0x01,0x00,0x00]
{vex} vpdpbusds ymm6, ymm5, ymmword ptr [r8 + 4*rax + 291]
// CHECK: {vex} vpdpbusds ymm6, ymm5, ymmword ptr [rip]
// CHECK: encoding: [0xc4,0xe2,0x55,0x51,0x35,0x00,0x00,0x00,0x00]
{vex} vpdpbusds ymm6, ymm5, ymmword ptr [rip]
// CHECK: {vex} vpdpbusds ymm6, ymm5, ymmword ptr [2*rbp - 1024]
// CHECK: encoding: [0xc4,0xe2,0x55,0x51,0x34,0x6d,0x00,0xfc,0xff,0xff]
{vex} vpdpbusds ymm6, ymm5, ymmword ptr [2*rbp - 1024]
// CHECK: {vex} vpdpbusds ymm6, ymm5, ymmword ptr [rcx + 4064]
// CHECK: encoding: [0xc4,0xe2,0x55,0x51,0xb1,0xe0,0x0f,0x00,0x00]
{vex} vpdpbusds ymm6, ymm5, ymmword ptr [rcx + 4064]
// CHECK: {vex} vpdpbusds ymm6, ymm5, ymmword ptr [rdx - 4096]
// CHECK: encoding: [0xc4,0xe2,0x55,0x51,0xb2,0x00,0xf0,0xff,0xff]
{vex} vpdpbusds ymm6, ymm5, ymmword ptr [rdx - 4096]
// CHECK: {vex} vpdpbusds xmm6, xmm5, xmmword ptr [rbp + 8*r14 + 268435456]
// CHECK: encoding: [0xc4,0xa2,0x51,0x51,0xb4,0xf5,0x00,0x00,0x00,0x10]
{vex} vpdpbusds xmm6, xmm5, xmmword ptr [rbp + 8*r14 + 268435456]
// CHECK: {vex} vpdpbusds xmm6, xmm5, xmmword ptr [r8 + 4*rax + 291]
// CHECK: encoding: [0xc4,0xc2,0x51,0x51,0xb4,0x80,0x23,0x01,0x00,0x00]
{vex} vpdpbusds xmm6, xmm5, xmmword ptr [r8 + 4*rax + 291]
// CHECK: {vex} vpdpbusds xmm6, xmm5, xmmword ptr [rip]
// CHECK: encoding: [0xc4,0xe2,0x51,0x51,0x35,0x00,0x00,0x00,0x00]
{vex} vpdpbusds xmm6, xmm5, xmmword ptr [rip]
// CHECK: {vex} vpdpbusds xmm6, xmm5, xmmword ptr [2*rbp - 512]
// CHECK: encoding: [0xc4,0xe2,0x51,0x51,0x34,0x6d,0x00,0xfe,0xff,0xff]
{vex} vpdpbusds xmm6, xmm5, xmmword ptr [2*rbp - 512]
// CHECK: {vex} vpdpbusds xmm6, xmm5, xmmword ptr [rcx + 2032]
// CHECK: encoding: [0xc4,0xe2,0x51,0x51,0xb1,0xf0,0x07,0x00,0x00]
{vex} vpdpbusds xmm6, xmm5, xmmword ptr [rcx + 2032]
// CHECK: {vex} vpdpbusds xmm6, xmm5, xmmword ptr [rdx - 2048]
// CHECK: encoding: [0xc4,0xe2,0x51,0x51,0xb2,0x00,0xf8,0xff,0xff]
{vex} vpdpbusds xmm6, xmm5, xmmword ptr [rdx - 2048]
// CHECK: {vex} vpdpwssd ymm6, ymm5, ymm4
// CHECK: encoding: [0xc4,0xe2,0x55,0x52,0xf4]
{vex} vpdpwssd ymm6, ymm5, ymm4
// CHECK: {vex} vpdpwssd xmm6, xmm5, xmm4
// CHECK: encoding: [0xc4,0xe2,0x51,0x52,0xf4]
{vex} vpdpwssd xmm6, xmm5, xmm4
// CHECK: {vex} vpdpwssd ymm6, ymm5, ymmword ptr [rbp + 8*r14 + 268435456]
// CHECK: encoding: [0xc4,0xa2,0x55,0x52,0xb4,0xf5,0x00,0x00,0x00,0x10]
{vex} vpdpwssd ymm6, ymm5, ymmword ptr [rbp + 8*r14 + 268435456]
// CHECK: {vex} vpdpwssd ymm6, ymm5, ymmword ptr [r8 + 4*rax + 291]
// CHECK: encoding: [0xc4,0xc2,0x55,0x52,0xb4,0x80,0x23,0x01,0x00,0x00]
{vex} vpdpwssd ymm6, ymm5, ymmword ptr [r8 + 4*rax + 291]
// CHECK: {vex} vpdpwssd ymm6, ymm5, ymmword ptr [rip]
// CHECK: encoding: [0xc4,0xe2,0x55,0x52,0x35,0x00,0x00,0x00,0x00]
{vex} vpdpwssd ymm6, ymm5, ymmword ptr [rip]
// CHECK: {vex} vpdpwssd ymm6, ymm5, ymmword ptr [2*rbp - 1024]
// CHECK: encoding: [0xc4,0xe2,0x55,0x52,0x34,0x6d,0x00,0xfc,0xff,0xff]
{vex} vpdpwssd ymm6, ymm5, ymmword ptr [2*rbp - 1024]
// CHECK: {vex} vpdpwssd ymm6, ymm5, ymmword ptr [rcx + 4064]
// CHECK: encoding: [0xc4,0xe2,0x55,0x52,0xb1,0xe0,0x0f,0x00,0x00]
{vex} vpdpwssd ymm6, ymm5, ymmword ptr [rcx + 4064]
// CHECK: {vex} vpdpwssd ymm6, ymm5, ymmword ptr [rdx - 4096]
// CHECK: encoding: [0xc4,0xe2,0x55,0x52,0xb2,0x00,0xf0,0xff,0xff]
{vex} vpdpwssd ymm6, ymm5, ymmword ptr [rdx - 4096]
// CHECK: {vex} vpdpwssd xmm6, xmm5, xmmword ptr [rbp + 8*r14 + 268435456]
// CHECK: encoding: [0xc4,0xa2,0x51,0x52,0xb4,0xf5,0x00,0x00,0x00,0x10]
{vex} vpdpwssd xmm6, xmm5, xmmword ptr [rbp + 8*r14 + 268435456]
// CHECK: {vex} vpdpwssd xmm6, xmm5, xmmword ptr [r8 + 4*rax + 291]
// CHECK: encoding: [0xc4,0xc2,0x51,0x52,0xb4,0x80,0x23,0x01,0x00,0x00]
{vex} vpdpwssd xmm6, xmm5, xmmword ptr [r8 + 4*rax + 291]
// CHECK: {vex} vpdpwssd xmm6, xmm5, xmmword ptr [rip]
// CHECK: encoding: [0xc4,0xe2,0x51,0x52,0x35,0x00,0x00,0x00,0x00]
{vex} vpdpwssd xmm6, xmm5, xmmword ptr [rip]
// CHECK: {vex} vpdpwssd xmm6, xmm5, xmmword ptr [2*rbp - 512]
// CHECK: encoding: [0xc4,0xe2,0x51,0x52,0x34,0x6d,0x00,0xfe,0xff,0xff]
{vex} vpdpwssd xmm6, xmm5, xmmword ptr [2*rbp - 512]
// CHECK: {vex} vpdpwssd xmm6, xmm5, xmmword ptr [rcx + 2032]
// CHECK: encoding: [0xc4,0xe2,0x51,0x52,0xb1,0xf0,0x07,0x00,0x00]
{vex} vpdpwssd xmm6, xmm5, xmmword ptr [rcx + 2032]
// CHECK: {vex} vpdpwssd xmm6, xmm5, xmmword ptr [rdx - 2048]
// CHECK: encoding: [0xc4,0xe2,0x51,0x52,0xb2,0x00,0xf8,0xff,0xff]
{vex} vpdpwssd xmm6, xmm5, xmmword ptr [rdx - 2048]
// CHECK: {vex} vpdpwssds ymm6, ymm5, ymm4
// CHECK: encoding: [0xc4,0xe2,0x55,0x53,0xf4]
{vex} vpdpwssds ymm6, ymm5, ymm4
// CHECK: {vex} vpdpwssds xmm6, xmm5, xmm4
// CHECK: encoding: [0xc4,0xe2,0x51,0x53,0xf4]
{vex} vpdpwssds xmm6, xmm5, xmm4
// CHECK: {vex} vpdpwssds ymm6, ymm5, ymmword ptr [rbp + 8*r14 + 268435456]
// CHECK: encoding: [0xc4,0xa2,0x55,0x53,0xb4,0xf5,0x00,0x00,0x00,0x10]
{vex} vpdpwssds ymm6, ymm5, ymmword ptr [rbp + 8*r14 + 268435456]
// CHECK: {vex} vpdpwssds ymm6, ymm5, ymmword ptr [r8 + 4*rax + 291]
// CHECK: encoding: [0xc4,0xc2,0x55,0x53,0xb4,0x80,0x23,0x01,0x00,0x00]
{vex} vpdpwssds ymm6, ymm5, ymmword ptr [r8 + 4*rax + 291]
// CHECK: {vex} vpdpwssds ymm6, ymm5, ymmword ptr [rip]
// CHECK: encoding: [0xc4,0xe2,0x55,0x53,0x35,0x00,0x00,0x00,0x00]
{vex} vpdpwssds ymm6, ymm5, ymmword ptr [rip]
// CHECK: {vex} vpdpwssds ymm6, ymm5, ymmword ptr [2*rbp - 1024]
// CHECK: encoding: [0xc4,0xe2,0x55,0x53,0x34,0x6d,0x00,0xfc,0xff,0xff]
{vex} vpdpwssds ymm6, ymm5, ymmword ptr [2*rbp - 1024]
// CHECK: {vex} vpdpwssds ymm6, ymm5, ymmword ptr [rcx + 4064]
// CHECK: encoding: [0xc4,0xe2,0x55,0x53,0xb1,0xe0,0x0f,0x00,0x00]
{vex} vpdpwssds ymm6, ymm5, ymmword ptr [rcx + 4064]
// CHECK: {vex} vpdpwssds ymm6, ymm5, ymmword ptr [rdx - 4096]
// CHECK: encoding: [0xc4,0xe2,0x55,0x53,0xb2,0x00,0xf0,0xff,0xff]
{vex} vpdpwssds ymm6, ymm5, ymmword ptr [rdx - 4096]
// CHECK: {vex} vpdpwssds xmm6, xmm5, xmmword ptr [rbp + 8*r14 + 268435456]
// CHECK: encoding: [0xc4,0xa2,0x51,0x53,0xb4,0xf5,0x00,0x00,0x00,0x10]
{vex} vpdpwssds xmm6, xmm5, xmmword ptr [rbp + 8*r14 + 268435456]
// CHECK: {vex} vpdpwssds xmm6, xmm5, xmmword ptr [r8 + 4*rax + 291]
// CHECK: encoding: [0xc4,0xc2,0x51,0x53,0xb4,0x80,0x23,0x01,0x00,0x00]
{vex} vpdpwssds xmm6, xmm5, xmmword ptr [r8 + 4*rax + 291]
// CHECK: {vex} vpdpwssds xmm6, xmm5, xmmword ptr [rip]
// CHECK: encoding: [0xc4,0xe2,0x51,0x53,0x35,0x00,0x00,0x00,0x00]
{vex} vpdpwssds xmm6, xmm5, xmmword ptr [rip]
// CHECK: {vex} vpdpwssds xmm6, xmm5, xmmword ptr [2*rbp - 512]
// CHECK: encoding: [0xc4,0xe2,0x51,0x53,0x34,0x6d,0x00,0xfe,0xff,0xff]
{vex} vpdpwssds xmm6, xmm5, xmmword ptr [2*rbp - 512]
// CHECK: {vex} vpdpwssds xmm6, xmm5, xmmword ptr [rcx + 2032]
// CHECK: encoding: [0xc4,0xe2,0x51,0x53,0xb1,0xf0,0x07,0x00,0x00]
{vex} vpdpwssds xmm6, xmm5, xmmword ptr [rcx + 2032]
// CHECK: {vex} vpdpwssds xmm6, xmm5, xmmword ptr [rdx - 2048]
// CHECK: encoding: [0xc4,0xe2,0x51,0x53,0xb2,0x00,0xf8,0xff,0xff]
{vex} vpdpwssds xmm6, xmm5, xmmword ptr [rdx - 2048]

llvm/test/MC/X86/x86-64-avx_vnni-encoding.s

  • This file was added.
// RUN: llvm-mc -triple x86_64-unknown-unknown -mattr=+avxvnni --show-encoding < %s | FileCheck %s
// CHECK: {vex} vpdpbusd %ymm4, %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x50,0xf4]
{vex} vpdpbusd %ymm4, %ymm5, %ymm6
// CHECK: {vex} vpdpbusd %xmm4, %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x50,0xf4]
{vex} vpdpbusd %xmm4, %xmm5, %xmm6
// CHECK: {vex} vpdpbusd 268435456(%rbp,%r14,8), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xa2,0x55,0x50,0xb4,0xf5,0x00,0x00,0x00,0x10]
{vex} vpdpbusd 268435456(%rbp,%r14,8), %ymm5, %ymm6
// CHECK: {vex} vpdpbusd 291(%r8,%rax,4), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xc2,0x55,0x50,0xb4,0x80,0x23,0x01,0x00,0x00]
{vex} vpdpbusd 291(%r8,%rax,4), %ymm5, %ymm6
// CHECK: {vex} vpdpbusd (%rip), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x50,0x35,0x00,0x00,0x00,0x00]
{vex} vpdpbusd (%rip), %ymm5, %ymm6
// CHECK: {vex} vpdpbusd -1024(,%rbp,2), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x50,0x34,0x6d,0x00,0xfc,0xff,0xff]
{vex} vpdpbusd -1024(,%rbp,2), %ymm5, %ymm6
// CHECK: {vex} vpdpbusd 4064(%rcx), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x50,0xb1,0xe0,0x0f,0x00,0x00]
{vex} vpdpbusd 4064(%rcx), %ymm5, %ymm6
// CHECK: {vex} vpdpbusd -4096(%rdx), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x50,0xb2,0x00,0xf0,0xff,0xff]
{vex} vpdpbusd -4096(%rdx), %ymm5, %ymm6
// CHECK: {vex} vpdpbusd 268435456(%rbp,%r14,8), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xa2,0x51,0x50,0xb4,0xf5,0x00,0x00,0x00,0x10]
{vex} vpdpbusd 268435456(%rbp,%r14,8), %xmm5, %xmm6
// CHECK: {vex} vpdpbusd 291(%r8,%rax,4), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xc2,0x51,0x50,0xb4,0x80,0x23,0x01,0x00,0x00]
{vex} vpdpbusd 291(%r8,%rax,4), %xmm5, %xmm6
// CHECK: {vex} vpdpbusd (%rip), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x50,0x35,0x00,0x00,0x00,0x00]
{vex} vpdpbusd (%rip), %xmm5, %xmm6
// CHECK: {vex} vpdpbusd -512(,%rbp,2), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x50,0x34,0x6d,0x00,0xfe,0xff,0xff]
{vex} vpdpbusd -512(,%rbp,2), %xmm5, %xmm6
// CHECK: {vex} vpdpbusd 2032(%rcx), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x50,0xb1,0xf0,0x07,0x00,0x00]
{vex} vpdpbusd 2032(%rcx), %xmm5, %xmm6
// CHECK: {vex} vpdpbusd -2048(%rdx), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x50,0xb2,0x00,0xf8,0xff,0xff]
{vex} vpdpbusd -2048(%rdx), %xmm5, %xmm6
// CHECK: {vex} vpdpbusds %ymm4, %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x51,0xf4]
{vex} vpdpbusds %ymm4, %ymm5, %ymm6
// CHECK: {vex} vpdpbusds %xmm4, %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x51,0xf4]
{vex} vpdpbusds %xmm4, %xmm5, %xmm6
// CHECK: {vex} vpdpbusds 268435456(%rbp,%r14,8), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xa2,0x55,0x51,0xb4,0xf5,0x00,0x00,0x00,0x10]
{vex} vpdpbusds 268435456(%rbp,%r14,8), %ymm5, %ymm6
// CHECK: {vex} vpdpbusds 291(%r8,%rax,4), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xc2,0x55,0x51,0xb4,0x80,0x23,0x01,0x00,0x00]
{vex} vpdpbusds 291(%r8,%rax,4), %ymm5, %ymm6
// CHECK: {vex} vpdpbusds (%rip), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x51,0x35,0x00,0x00,0x00,0x00]
{vex} vpdpbusds (%rip), %ymm5, %ymm6
// CHECK: {vex} vpdpbusds -1024(,%rbp,2), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x51,0x34,0x6d,0x00,0xfc,0xff,0xff]
{vex} vpdpbusds -1024(,%rbp,2), %ymm5, %ymm6
// CHECK: {vex} vpdpbusds 4064(%rcx), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x51,0xb1,0xe0,0x0f,0x00,0x00]
{vex} vpdpbusds 4064(%rcx), %ymm5, %ymm6
// CHECK: {vex} vpdpbusds -4096(%rdx), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x51,0xb2,0x00,0xf0,0xff,0xff]
{vex} vpdpbusds -4096(%rdx), %ymm5, %ymm6
// CHECK: {vex} vpdpbusds 268435456(%rbp,%r14,8), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xa2,0x51,0x51,0xb4,0xf5,0x00,0x00,0x00,0x10]
{vex} vpdpbusds 268435456(%rbp,%r14,8), %xmm5, %xmm6
// CHECK: {vex} vpdpbusds 291(%r8,%rax,4), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xc2,0x51,0x51,0xb4,0x80,0x23,0x01,0x00,0x00]
{vex} vpdpbusds 291(%r8,%rax,4), %xmm5, %xmm6
// CHECK: {vex} vpdpbusds (%rip), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x51,0x35,0x00,0x00,0x00,0x00]
{vex} vpdpbusds (%rip), %xmm5, %xmm6
// CHECK: {vex} vpdpbusds -512(,%rbp,2), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x51,0x34,0x6d,0x00,0xfe,0xff,0xff]
{vex} vpdpbusds -512(,%rbp,2), %xmm5, %xmm6
// CHECK: {vex} vpdpbusds 2032(%rcx), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x51,0xb1,0xf0,0x07,0x00,0x00]
{vex} vpdpbusds 2032(%rcx), %xmm5, %xmm6
// CHECK: {vex} vpdpbusds -2048(%rdx), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x51,0xb2,0x00,0xf8,0xff,0xff]
{vex} vpdpbusds -2048(%rdx), %xmm5, %xmm6
// CHECK: {vex} vpdpwssd %ymm4, %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x52,0xf4]
{vex} vpdpwssd %ymm4, %ymm5, %ymm6
// CHECK: {vex} vpdpwssd %xmm4, %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x52,0xf4]
{vex} vpdpwssd %xmm4, %xmm5, %xmm6
// CHECK: {vex} vpdpwssd 268435456(%rbp,%r14,8), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xa2,0x55,0x52,0xb4,0xf5,0x00,0x00,0x00,0x10]
{vex} vpdpwssd 268435456(%rbp,%r14,8), %ymm5, %ymm6
// CHECK: {vex} vpdpwssd 291(%r8,%rax,4), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xc2,0x55,0x52,0xb4,0x80,0x23,0x01,0x00,0x00]
{vex} vpdpwssd 291(%r8,%rax,4), %ymm5, %ymm6
// CHECK: {vex} vpdpwssd (%rip), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x52,0x35,0x00,0x00,0x00,0x00]
{vex} vpdpwssd (%rip), %ymm5, %ymm6
// CHECK: {vex} vpdpwssd -1024(,%rbp,2), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x52,0x34,0x6d,0x00,0xfc,0xff,0xff]
{vex} vpdpwssd -1024(,%rbp,2), %ymm5, %ymm6
// CHECK: {vex} vpdpwssd 4064(%rcx), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x52,0xb1,0xe0,0x0f,0x00,0x00]
{vex} vpdpwssd 4064(%rcx), %ymm5, %ymm6
// CHECK: {vex} vpdpwssd -4096(%rdx), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x52,0xb2,0x00,0xf0,0xff,0xff]
{vex} vpdpwssd -4096(%rdx), %ymm5, %ymm6
// CHECK: {vex} vpdpwssd 268435456(%rbp,%r14,8), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xa2,0x51,0x52,0xb4,0xf5,0x00,0x00,0x00,0x10]
{vex} vpdpwssd 268435456(%rbp,%r14,8), %xmm5, %xmm6
// CHECK: {vex} vpdpwssd 291(%r8,%rax,4), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xc2,0x51,0x52,0xb4,0x80,0x23,0x01,0x00,0x00]
{vex} vpdpwssd 291(%r8,%rax,4), %xmm5, %xmm6
// CHECK: {vex} vpdpwssd (%rip), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x52,0x35,0x00,0x00,0x00,0x00]
{vex} vpdpwssd (%rip), %xmm5, %xmm6
// CHECK: {vex} vpdpwssd -512(,%rbp,2), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x52,0x34,0x6d,0x00,0xfe,0xff,0xff]
{vex} vpdpwssd -512(,%rbp,2), %xmm5, %xmm6
// CHECK: {vex} vpdpwssd 2032(%rcx), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x52,0xb1,0xf0,0x07,0x00,0x00]
{vex} vpdpwssd 2032(%rcx), %xmm5, %xmm6
// CHECK: {vex} vpdpwssd -2048(%rdx), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x52,0xb2,0x00,0xf8,0xff,0xff]
{vex} vpdpwssd -2048(%rdx), %xmm5, %xmm6
// CHECK: {vex} vpdpwssds %ymm4, %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x53,0xf4]
{vex} vpdpwssds %ymm4, %ymm5, %ymm6
// CHECK: {vex} vpdpwssds %xmm4, %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x53,0xf4]
{vex} vpdpwssds %xmm4, %xmm5, %xmm6
// CHECK: {vex} vpdpwssds 268435456(%rbp,%r14,8), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xa2,0x55,0x53,0xb4,0xf5,0x00,0x00,0x00,0x10]
{vex} vpdpwssds 268435456(%rbp,%r14,8), %ymm5, %ymm6
// CHECK: {vex} vpdpwssds 291(%r8,%rax,4), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xc2,0x55,0x53,0xb4,0x80,0x23,0x01,0x00,0x00]
{vex} vpdpwssds 291(%r8,%rax,4), %ymm5, %ymm6
// CHECK: {vex} vpdpwssds (%rip), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x53,0x35,0x00,0x00,0x00,0x00]
{vex} vpdpwssds (%rip), %ymm5, %ymm6
// CHECK: {vex} vpdpwssds -1024(,%rbp,2), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x53,0x34,0x6d,0x00,0xfc,0xff,0xff]
{vex} vpdpwssds -1024(,%rbp,2), %ymm5, %ymm6
// CHECK: {vex} vpdpwssds 4064(%rcx), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x53,0xb1,0xe0,0x0f,0x00,0x00]
{vex} vpdpwssds 4064(%rcx), %ymm5, %ymm6
// CHECK: {vex} vpdpwssds -4096(%rdx), %ymm5, %ymm6
// CHECK: encoding: [0xc4,0xe2,0x55,0x53,0xb2,0x00,0xf0,0xff,0xff]
{vex} vpdpwssds -4096(%rdx), %ymm5, %ymm6
// CHECK: {vex} vpdpwssds 268435456(%rbp,%r14,8), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xa2,0x51,0x53,0xb4,0xf5,0x00,0x00,0x00,0x10]
{vex} vpdpwssds 268435456(%rbp,%r14,8), %xmm5, %xmm6
// CHECK: {vex} vpdpwssds 291(%r8,%rax,4), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xc2,0x51,0x53,0xb4,0x80,0x23,0x01,0x00,0x00]
{vex} vpdpwssds 291(%r8,%rax,4), %xmm5, %xmm6
// CHECK: {vex} vpdpwssds (%rip), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x53,0x35,0x00,0x00,0x00,0x00]
{vex} vpdpwssds (%rip), %xmm5, %xmm6
// CHECK: {vex} vpdpwssds -512(,%rbp,2), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x53,0x34,0x6d,0x00,0xfe,0xff,0xff]
{vex} vpdpwssds -512(,%rbp,2), %xmm5, %xmm6
// CHECK: {vex} vpdpwssds 2032(%rcx), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x53,0xb1,0xf0,0x07,0x00,0x00]
{vex} vpdpwssds 2032(%rcx), %xmm5, %xmm6
// CHECK: {vex} vpdpwssds -2048(%rdx), %xmm5, %xmm6
// CHECK: encoding: [0xc4,0xe2,0x51,0x53,0xb2,0x00,0xf8,0xff,0xff]
{vex} vpdpwssds -2048(%rdx), %xmm5, %xmm6