This is an archive of the discontinued LLVM Phabricator instance.

Differential D76434

[SCEV] Query expanded immediate cost at minsize
ClosedPublic

Authored by samparker on Mar 19 2020, 9:17 AM.

Details

Reviewers
lebedev.ri
reames
echristo
mkazantsev
hfinkel
craig.topper
arsenm
dmgreen
uweigand
SjoerdMeijer
Commits
rG0bdf8c912724: [SCEV] Constant expansion cost at minsize
Summary

As code size is the only thing we care about at minsize, query the cost of materialising immediates when calculating the cost of a SCEV expansion. We also modify the CostKind to TCK_CodeSize for minsize, instead of RecipThroughput. This gives -0.1% geomean reduction of the llvm test suite at -Oz for both thumbv7a and aarch64.

Diff Detail

Event Timeline

samparker created this revision.Mar 19 2020, 9:17 AM
Herald added subscribers: danielkiss, zzheng, hiraditya and 3 others. · View Herald TranscriptMar 19 2020, 9:18 AM
lebedev.ri added a comment.Mar 19 2020, 12:52 PM

Hmm, i'm not a ventillator. Some initial thoughts.

  1. This is rather pessimistic. If we really want to do this, we need to use TargetTransformInfo::getIntImmCostInst().
  2. What cost does that model? I'm under impression that TargetTransformInfo::getIntImmCost*() model TargetCostKind::TCK_CodeSize, since it is mainly used in ConstantHoistingPass. Here we model TargetCostKind::TCK_RecipThroughput.
  3. Assuming that TargetTransformInfo::getIntImmCost*() actually models TargetCostKind::TCK_RecipThroughput, i believe, all X86 changes should not be here. Either most of them will get fixed via using TargetTransformInfo::getIntImmCostInst(), or the llvm::SCEVCheapExpansionBudget will need to be bumped.
Herald added a subscriber: • wuzish. · View Herald TranscriptMar 19 2020, 12:52 PM
lebedev.ri added a comment.Mar 19 2020, 4:03 PM

@samparker are you trying to mitigate perf impact for in-order CPU?
I wonder if D73501 simply is counter-productive then.
Though yes, vectorizers need to be taught that trick.

samparker added a subscriber: spatel.Mar 20 2020, 1:29 AM

Thanks for taking a look.

This is rather pessimistic. If we really want to do this, we need to use TargetTransformInfo::getIntImmCostInst().

This sounds good and what I was also considering.

What cost does that model?

I was under the impression that throughput/codesize costs for immediates would be highly correlated, where a 'high cost' constant would introduce instruction(s) to generate it, increasing code size and reducing throughput. As @spatel said in D76124, the lines have become blurred but we should be modelling at least something here.

I wonder if D73501 simply is counter-productive then.

I would be lying if I said that the patch didn't cause a whole world of pain :) But I would like to try to resolve this, if we can, by modelling costs better. The SCEV changes may have just broken our we do unrolling for our little microcontrollers, so I'll be looking at ARM TTI too.

Herald added a reviewer: aartbik. · View Herald TranscriptMar 20 2020, 1:29 AM
lebedev.ri added a comment.Mar 20 2020, 4:34 AM

Do we model rthrougput for constants elsewhere?
I may be wrong, but i don't recall seeing any such modelling previously.
Let's take a step back here. How about we simply revert D73501?

samparker added a comment.Mar 20 2020, 5:22 AM

Do we model rthrougput for constants elsewhere?

I guess that depends on who 'we' are because, again, different backends will be modelling different things. Why the focus on throughput now anyway? When this code was using getOperationCost, it would have been getting some performance/code size cost, which I expect most backends will be modelling for constants. And I'm not really sure why throughput is specifically important here, we're not concerning ourselves with the throughput of casts and compares, right?

Let's take a step back here. How about we simply revert D73501?

Sounds good to me, although I still need to prod around here to see if codegen risk being erratic in the future.

lebedev.ri added a comment.Mar 20 2020, 7:27 AM
In D76434#1933359, @samparker wrote:

Do we model rthrougput for constants elsewhere?

I guess that depends on who 'we' are because, again, different backends will be modelling different things.

Sorry, i meant llvm transforms in general, not backends.

Why the focus on throughput now anyway? When this code was using getOperationCost,
it would have been getting some performance/code size cost,
which I expect most backends will be modelling for constants.
And I'm not really sure why throughput is specifically important here,
we're not concerning ourselves with the throughput of casts and compares, right?

We have 3 cost models - latency, size and rthroughput.
My aspirational goal in this scevexpander budget was: "how much more computations are we willing to do without it being too much of a burden?"
Size model isn't really applicable here - we *could* lower any sequence into a libcall, regardless of it's native instruction count.
Likewise i'm not sure we're really after latency here, which leaves us with rthroughput.
But we can't subtract oranges from cucumbers, so all cost modelling should be consistently using rthroughput cost model.
Thus i'm asking, what does getIntImmCost() model? rthroughput or size?

Let's take a step back here. How about we simply revert D73501?

Sounds good to me, although I still need to prod around here to see if codegen risk being erratic in the future.

samparker added a comment.Mar 20 2020, 7:49 AM

I'm pretty sure there's only two APIs that were designed around throughput: 'getInstructionThroughput' and 'getArithmeticInstrCost' and maybe the latter should be named more explicitly. The code size calls are generally a bit more explicit, so there's 'getIntImmCodeSizeCost', which is not the one I've used.

"how much more computations are we willing to do without it being too much of a burden?"

And this also depends what we're considering as a 'burden', and another angle that I'll probably need to look at in the near future is the burden of code size... Either way, this code shouldn't assume that constants are free (in any sense of the term).

samparker updated this revision to Diff 252830.Mar 26 2020, 7:05 AM
samparker edited the summary of this revision. (Show Details)

Introduced a lambda to look at an expressions operand and, if it's a constant, query for cost using getIntImmCostInst.

Herald added a subscriber: javed.absar. · View Herald TranscriptMar 26 2020, 7:05 AM
lebedev.ri requested changes to this revision.Mar 26 2020, 7:53 AM
In D76434#1933498, @lebedev.ri wrote:

<...>
Thus i'm asking, what does getIntImmCost() model? rthroughput or size?

This revision now requires changes to proceed.Mar 26 2020, 7:53 AM
samparker added a comment.Mar 26 2020, 9:06 AM

The API is there to figure out whether the constant will be folded into the given instruction, otherwise there will be some 'cost' to materialize it. Having to generate instruction(s) for the materialization is likely to increase code size, but more importantly, reduce throughput and increase latency - which is why it's sometimes beneficial to hoist expensive constants out of loops.

samparker added a comment.Apr 1 2020, 8:15 AM

Ping.

samparker updated this revision to Diff 256260.Apr 9 2020, 5:44 AM
samparker added reviewers: dmgreen, uweigand.

Rebased. Thanks for reverting rewriteLoopExitValues @lebedev.ri, but I'm still seeing value in this patch. I'm not seeing any changes when running the test suite on my X86 box, but for Arm's DSP suite this affects 42/166 benchmarks for Thumb1 and 30/166 for Thumb2. Out of those changes I'm seeing a 1.6% geomean improvement for both targets.

aartbik removed a reviewer: aartbik.Apr 14 2020, 4:14 PM
samparker updated this revision to Diff 285000.EditedAug 12 2020, 1:06 AM
samparker retitled this revision from [SCEV] Query for immediate cost in Expander to [SCEV] Query expanded immediate cost at minsize.
samparker edited the summary of this revision. (Show Details)

Now only performing the checks when optimising for minsize.
thumbv7a results:

Metric: size..text

Program                                                                                              master  scev-expander diff
                                    test-suite :: SingleSource/Benchmarks/Adobe-C++/loop_unroll.test   47596   46528       -2.2%
                                   test-suite :: MultiSource/Benchmarks/ASC_Sequoia/AMGmk/AMGmk.test    6484    6340       -2.2%
                                        test-suite :: MultiSource/Benchmarks/VersaBench/bmm/bmm.test     924     908       -1.7%
                                 test-suite :: MultiSource/Benchmarks/Rodinia/backprop/backprop.test    2916    2868       -1.6%
                                          test-suite :: MultiSource/Benchmarks/McCat/05-eks/eks.test    4748    4696       -1.1%
                        test-suite :: MultiSource/Benchmarks/Trimaran/netbench-url/netbench-url.test    3120    3088       -1.0%
                                        test-suite :: SingleSource/Benchmarks/Misc/himenobmtxpa.test    2408    2384       -1.0%
                          test-suite :: MultiSource/Applications/ALAC/decode/alacconvert-decode.test   19920   19748       -0.9%
                          test-suite :: MultiSource/Applications/ALAC/encode/alacconvert-encode.test   19920   19748       -0.9%
                         test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C++/PENNANT/PENNANT.test   32516   32236       -0.9%
                                  test-suite :: SingleSource/Benchmarks/Shootout/Shootout-lists.test     956     964        0.8%
                                             test-suite :: MultiSource/Benchmarks/Ptrdist/ft/ft.test    2920    2896       -0.8%
                 test-suite :: MultiSource/Benchmarks/MiBench/automotive-susan/automotive-susan.test   14600   14480       -0.8%
                                            test-suite :: SingleSource/Benchmarks/McGill/queens.test     976     968       -0.8%
                           test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C/miniAMR/miniAMR.test   31508   31252       -0.8%
                             test-suite :: MultiSource/Benchmarks/MallocBench/espresso/espresso.test   68296   67752       -0.8%
                                           test-suite :: SingleSource/Benchmarks/Misc/whetstone.test    1604    1592       -0.7%
                                       test-suite :: MultiSource/Benchmarks/SciMark2-C/scimark2.test    4936    4904       -0.6%
                                    test-suite :: MultiSource/Benchmarks/Prolangs-C/agrep/agrep.test   23836   23684       -0.6%
                                  test-suite :: MultiSource/Benchmarks/Prolangs-C/bison/mybison.test   27552   27384       -0.6%
                                     test-suite :: SingleSource/Benchmarks/BenchmarkGame/puzzle.test    1320    1312       -0.6%
                                           test-suite :: SingleSource/Benchmarks/Stanford/Oscar.test    1348    1340       -0.6%
                                          test-suite :: SingleSource/Benchmarks/Stanford/Puzzle.test    1376    1368       -0.6%
                                         test-suite :: MultiSource/Applications/minisat/minisat.test    9980    9924       -0.6%
                                         test-suite :: SingleSource/Benchmarks/Misc/ReedSolomon.test    3008    3024        0.5%
                       test-suite :: MultiSource/Benchmarks/MiBench/consumer-lame/consumer-lame.test   69828   69460       -0.5%
                                 test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C/CoMD/CoMD.test   20116   20012       -0.5%
                                          test-suite :: SingleSource/Benchmarks/Misc-C++/bigfib.test    3332    3316       -0.5%
                                      test-suite :: MultiSource/Benchmarks/mafft/pairlocalalign.test  168600  167800       -0.5%
                           test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C/miniGMG/miniGMG.test   26104   25984       -0.5%
                                   test-suite :: MultiSource/Benchmarks/ASC_Sequoia/IRSmk/IRSmk.test    3592    3576       -0.4%
                                           test-suite :: MultiSource/Applications/oggenc/oggenc.test   90092   89700       -0.4%
                          test-suite :: MultiSource/Benchmarks/VersaBench/beamformer/beamformer.test    2788    2776       -0.4%
                                              test-suite :: SingleSource/Benchmarks/McGill/misr.test    1864    1872        0.4%
                                          test-suite :: MultiSource/Benchmarks/McCat/18-imp/imp.test    7540    7508       -0.4%
                                        test-suite :: MultiSource/Applications/JM/lencod/lencod.test  318860  317620       -0.4%
                           test-suite :: MultiSource/Benchmarks/ASC_Sequoia/CrystalMk/CrystalMk.test    2176    2168       -0.4%
                                   test-suite :: MultiSource/Benchmarks/Fhourstones/fhourstones.test    4828    4812       -0.3%
                           test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C++/miniFE/miniFE.test   29164   29068       -0.3%
                                     test-suite :: MultiSource/Benchmarks/FreeBench/pifft/pifft.test   24400   24320       -0.3%
                                            test-suite :: SingleSource/Benchmarks/Misc/oourafft.test    5344    5328       -0.3%
                                             test-suite :: MultiSource/Applications/spiff/spiff.test   12184   12152       -0.3%
                                             test-suite :: MultiSource/Applications/sgefa/sgefa.test    6096    6080       -0.3%
                                             test-suite :: MultiSource/Applications/SPASS/SPASS.test  201512  201000       -0.3%
                                        test-suite :: MultiSource/Applications/JM/ldecod/ldecod.test  128336  128016       -0.2%
                           test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C/RSBench/rsbench.test    9640    9616       -0.2%
                                         test-suite :: MultiSource/Applications/ClamAV/clamscan.test  246116  245556       -0.2%
                                                 test-suite :: MultiSource/Applications/lua/lua.test   61520   61392       -0.2%
                                               test-suite :: MultiSource/Benchmarks/PAQ8p/paq8p.test   40916   40832       -0.2%
               test-suite :: MultiSource/Benchmarks/MiBench/security-rijndael/security-rijndael.test    7796    7780       -0.2%
                                               test-suite :: SingleSource/Benchmarks/Misc/flops.test    3936    3928       -0.2%
                                         test-suite :: MultiSource/Applications/obsequi/Obsequi.test   17468   17436       -0.2%
                               test-suite :: MultiSource/Benchmarks/ASCI_Purple/SMG2000/smg2000.test   89716   89552       -0.2%
                                       test-suite :: MultiSource/Benchmarks/7zip/7zip-benchmark.test  324352  323820       -0.2%
                                             test-suite :: MultiSource/Benchmarks/Bullet/bullet.test  296700  296220       -0.2%
                                     test-suite :: SingleSource/Benchmarks/CoyoteBench/fftbench.test    2524    2520       -0.2%
                                             test-suite :: MultiSource/Benchmarks/Ptrdist/bc/bc.test   20760   20728       -0.2%
                                             test-suite :: MultiSource/Benchmarks/nbench/nbench.test   16872   16848       -0.1%
                            test-suite :: MultiSource/Benchmarks/Fhourstones-3.1/fhourstones3.1.test    2948    2952        0.1%
                           test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C/XSBench/XSBench.test    6084    6076       -0.1%
                                          test-suite :: MultiSource/Applications/d/make_dparser.test   43812   43756       -0.1%
                                         test-suite :: MultiSource/Applications/SIBsim4/SIBsim4.test   20152   20176        0.1%
                                       test-suite :: MultiSource/Applications/hexxagon/hexxagon.test    7252    7244       -0.1%
                       test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C/SimpleMOC/SimpleMOC.test   14828   14812       -0.1%
                            test-suite :: MicroBenchmarks/LCALS/SubsetCLambdaLoops/lcalsCLambda.test   92020   91924       -0.1%
                                      test-suite :: MultiSource/Benchmarks/VersaBench/dbms/dbms.test    8228    8220       -0.1%
                                                   test-suite :: MultiSource/Benchmarks/sim/sim.test    8704    8696       -0.1%
                                         test-suite :: MultiSource/Applications/sqlite3/sqlite3.test  168288  168136       -0.1%
                                  test-suite :: MicroBenchmarks/LCALS/SubsetCRawLoops/lcalsCRaw.test   91924   91844       -0.1%
                             test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C++/CLAMR/CLAMR.test  218764  218580       -0.1%
                                                    test-suite :: MicroBenchmarks/harris/harris.test   62584   62536       -0.1%
                                       test-suite :: MultiSource/Benchmarks/Ptrdist/yacr2/yacr2.test   11224   11216       -0.1%
                            test-suite :: MicroBenchmarks/LCALS/SubsetALambdaLoops/lcalsALambda.test   90788   90724       -0.1%
                                   test-suite :: MultiSource/Benchmarks/MallocBench/cfrac/cfrac.test   12352   12344       -0.1%
                             test-suite :: MultiSource/Benchmarks/mediabench/jpeg/jpeg-6a/cjpeg.test   62016   61976       -0.1%
                             test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C++/HPCCG/HPCCG.test    7120    7116       -0.1%
               test-suite :: MicroBenchmarks/ImageProcessing/BilateralFiltering/BilateralFilter.test   61768   61736       -0.1%
        test-suite :: MicroBenchmarks/ImageProcessing/AnisotropicDiffusion/AnisotropicDiffusion.test   61800   61768       -0.1%
                                        test-suite :: MicroBenchmarks/ImageProcessing/Blur/blur.test   62136   62104       -0.1%
                                    test-suite :: MicroBenchmarks/ImageProcessing/Dither/Dither.test   62440   62408       -0.1%
                                               test-suite :: MultiSource/Applications/siod/siod.test   47092   47068       -0.1%
                       test-suite :: MultiSource/Benchmarks/MiBench/consumer-jpeg/consumer-jpeg.test   60088   60064       -0.0%

aarch64 results:

Metric: size..text

Program                                                                                              master  scev-expander diff
                                    test-suite :: SingleSource/Benchmarks/Adobe-C++/loop_unroll.test   72100   68220       -5.4%
                                           test-suite :: SingleSource/Benchmarks/Misc/whetstone.test    2092    2004       -4.2%
                                 test-suite :: MultiSource/Benchmarks/Rodinia/backprop/backprop.test    2588    2500       -3.4%
                                   test-suite :: MultiSource/Benchmarks/ASC_Sequoia/AMGmk/AMGmk.test    8420    8196       -2.7%
                                          test-suite :: SingleSource/Benchmarks/Stanford/Puzzle.test    2212    2156       -2.5%
                                           test-suite :: SingleSource/Benchmarks/Stanford/Oscar.test    1652    1620       -1.9%
                        test-suite :: MultiSource/Benchmarks/Trimaran/netbench-url/netbench-url.test    2948    2908       -1.4%
                         test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C++/PENNANT/PENNANT.test   43972   43428       -1.2%
                                             test-suite :: MultiSource/Benchmarks/Ptrdist/ft/ft.test    3668    3628       -1.1%
                                          test-suite :: MultiSource/Benchmarks/McCat/05-eks/eks.test    5876    5812       -1.1%
                                 test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C/CoMD/CoMD.test   22276   22068       -0.9%
                                           test-suite :: MultiSource/Benchmarks/Olden/em3d/em3d.test    2812    2836        0.9%
                                         test-suite :: MultiSource/Applications/SIBsim4/SIBsim4.test   26804   26580       -0.8%
                                     test-suite :: SingleSource/Benchmarks/BenchmarkGame/puzzle.test     980     972       -0.8%
                                          test-suite :: MultiSource/Benchmarks/McCat/18-imp/imp.test    9132    9060       -0.8%
                          test-suite :: MultiSource/Applications/ALAC/encode/alacconvert-encode.test   26220   26020       -0.8%
                          test-suite :: MultiSource/Applications/ALAC/decode/alacconvert-decode.test   26220   26020       -0.8%
                                        test-suite :: SingleSource/Benchmarks/Misc/himenobmtxpa.test    3156    3132       -0.8%
                                   test-suite :: MultiSource/Benchmarks/ASC_Sequoia/IRSmk/IRSmk.test    3156    3132       -0.8%
                                       test-suite :: MultiSource/Benchmarks/SciMark2-C/scimark2.test    5460    5420       -0.7%
                           test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C/miniAMR/miniAMR.test   34852   34628       -0.6%
                             test-suite :: MultiSource/Benchmarks/MallocBench/espresso/espresso.test   99124   98540       -0.6%
                                           test-suite :: MultiSource/Applications/oggenc/oggenc.test  107024  106456       -0.5%
                                            test-suite :: SingleSource/Benchmarks/McGill/queens.test    1516    1508       -0.5%
                                  test-suite :: MultiSource/Benchmarks/Prolangs-C/bison/mybison.test   36012   35836       -0.5%
                       test-suite :: MultiSource/Benchmarks/MiBench/consumer-lame/consumer-lame.test   82264   81864       -0.5%
                                              test-suite :: SingleSource/Benchmarks/McGill/misr.test    1668    1660       -0.5%
                            test-suite :: MultiSource/Benchmarks/Fhourstones-3.1/fhourstones3.1.test    3372    3356       -0.5%
                                         test-suite :: MultiSource/Applications/minisat/minisat.test   13836   13772       -0.5%
                                         test-suite :: SingleSource/Benchmarks/Misc/ReedSolomon.test    3460    3444       -0.5%
                                   test-suite :: MultiSource/Benchmarks/Fhourstones/fhourstones.test    3996    3980       -0.4%
                                               test-suite :: SingleSource/Benchmarks/Misc/flops.test    4116    4100       -0.4%
                                     test-suite :: MultiSource/Benchmarks/FreeBench/pifft/pifft.test   27396   27292       -0.4%
                           test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C/RSBench/rsbench.test    8852    8820       -0.4%
                                    test-suite :: MultiSource/Benchmarks/Prolangs-C/agrep/agrep.test   32168   32056       -0.3%
                                          test-suite :: SingleSource/Benchmarks/Misc-C++/bigfib.test    4892    4876       -0.3%
                 test-suite :: MultiSource/Benchmarks/MiBench/automotive-susan/automotive-susan.test   19708   19644       -0.3%
                                             test-suite :: MultiSource/Benchmarks/nbench/nbench.test   20916   20852       -0.3%
                                             test-suite :: MultiSource/Applications/spiff/spiff.test   16028   15980       -0.3%
                                      test-suite :: MultiSource/Benchmarks/mafft/pairlocalalign.test  203964  203460       -0.2%
                               test-suite :: SingleSource/Benchmarks/Misc-C++/Large/sphereflake.test    3380    3372       -0.2%
                                             test-suite :: MultiSource/Applications/SPASS/SPASS.test  288136  287464       -0.2%
                                     test-suite :: SingleSource/Benchmarks/CoyoteBench/fftbench.test    3596    3588       -0.2%
                                        test-suite :: MultiSource/Applications/JM/ldecod/ldecod.test  161852  161492       -0.2%
                                                   test-suite :: MultiSource/Benchmarks/sim/sim.test   11044   11020       -0.2%
                           test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C++/miniFE/miniFE.test   41156   41068       -0.2%
                                        test-suite :: MultiSource/Applications/JM/lencod/lencod.test  383100  382340       -0.2%
                           test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C/miniGMG/miniGMG.test   28852   28796       -0.2%
                                             test-suite :: MultiSource/Benchmarks/Ptrdist/bc/bc.test   27236   27188       -0.2%
               test-suite :: MultiSource/Benchmarks/MiBench/security-rijndael/security-rijndael.test    9132    9116       -0.2%
                                            test-suite :: SingleSource/Benchmarks/Misc/oourafft.test    5444    5436       -0.1%
                                   test-suite :: MultiSource/Benchmarks/MallocBench/cfrac/cfrac.test   16428   16404       -0.1%
                                       test-suite :: MultiSource/Benchmarks/7zip/7zip-benchmark.test  471400  470712       -0.1%
                                      test-suite :: MultiSource/Benchmarks/VersaBench/dbms/dbms.test   11324   11308       -0.1%
                                             test-suite :: MultiSource/Benchmarks/Bullet/bullet.test  378596  378116       -0.1%
                                                    test-suite :: MicroBenchmarks/harris/harris.test  104836  104708       -0.1%
                                             test-suite :: MultiSource/Applications/sgefa/sgefa.test    6812    6804       -0.1%
                                         test-suite :: MultiSource/Applications/sqlite3/sqlite3.test  244840  244568       -0.1%
                                         test-suite :: MultiSource/Applications/obsequi/Obsequi.test   21980   21956       -0.1%
                                       test-suite :: MultiSource/Benchmarks/Ptrdist/yacr2/yacr2.test   15124   15108       -0.1%
                               test-suite :: MultiSource/Benchmarks/ASCI_Purple/SMG2000/smg2000.test  116900  116780       -0.1%
                                                 test-suite :: MultiSource/Applications/lua/lua.test   90508   90420       -0.1%
                                         test-suite :: MultiSource/Applications/ClamAV/clamscan.test  331984  331680       -0.1%
                                               test-suite :: MultiSource/Benchmarks/PAQ8p/paq8p.test   54364   54316       -0.1%
                                               test-suite :: MultiSource/Applications/siod/siod.test   64448   64392       -0.1%
                       test-suite :: MultiSource/Benchmarks/MiBench/consumer-jpeg/consumer-jpeg.test   83348   83276       -0.1%
                             test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C++/HPCCG/HPCCG.test    9548    9540       -0.1%
                             test-suite :: MultiSource/Benchmarks/mediabench/jpeg/jpeg-6a/cjpeg.test   86428   86356       -0.1%
                                       test-suite :: MultiSource/Applications/hexxagon/hexxagon.test   10180   10172       -0.1%
                                  test-suite :: MicroBenchmarks/LCALS/SubsetCRawLoops/lcalsCRaw.test  142752  142656       -0.1%
                            test-suite :: MicroBenchmarks/LCALS/SubsetCLambdaLoops/lcalsCLambda.test  142792  142712       -0.1%
                                          test-suite :: MultiSource/Applications/d/make_dparser.test   62912   62880       -0.1%
                                  test-suite :: MicroBenchmarks/LCALS/SubsetARawLoops/lcalsARaw.test  139296  139232       -0.0%
Herald added a project: Restricted Project. · View Herald TranscriptAug 12 2020, 1:06 AM
SjoerdMeijer added a subscriber: SjoerdMeijer.Aug 12 2020, 1:17 AM

That are really nice code size savings!

As only the improvements are shown, just curious if there are no regressions? Thus, this is overall an improvement too?
And for extra confidence and as the numbers are easy to obtain, probably best to get numbers for x86 too?

samparker added a comment.Aug 12 2020, 1:23 AM

Thanks!. That's the full set of results, there's a few little regressions for thumb and I think only one for aarch64... I'll get the X86 numbers now.

samparker added a comment.Aug 12 2020, 1:33 AM

Similar story on X86 as well:

Metric: size..text

Program                                                                                              master  scev-expander diff 
                                           test-suite :: SingleSource/Benchmarks/Misc/whetstone.test    2437    2325       -4.6%
                                           test-suite :: SingleSource/Benchmarks/Stanford/Oscar.test    1637    1573       -3.9%
                                    test-suite :: SingleSource/Benchmarks/Adobe-C++/loop_unroll.test   58885   56645       -3.8%
                                     test-suite :: SingleSource/Benchmarks/BenchmarkGame/puzzle.test     821     805       -1.9%
                                          test-suite :: MultiSource/Benchmarks/McCat/05-eks/eks.test    5845    5733       -1.9%
                                   test-suite :: MultiSource/Benchmarks/ASC_Sequoia/AMGmk/AMGmk.test    8597    8437       -1.9%
                        test-suite :: MultiSource/Benchmarks/Trimaran/netbench-url/netbench-url.test    2661    2613       -1.8%
                                          test-suite :: SingleSource/Benchmarks/Stanford/Puzzle.test    1781    1749       -1.8%
                            test-suite :: MultiSource/Benchmarks/Fhourstones-3.1/fhourstones3.1.test    2917    2869       -1.6%
                                        test-suite :: SingleSource/Benchmarks/Misc/himenobmtxpa.test    3253    3205       -1.5%
                         test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C++/PENNANT/PENNANT.test   47093   46469       -1.3%
                                 test-suite :: MultiSource/Benchmarks/Rodinia/backprop/backprop.test    2485    2453       -1.3%
                                            test-suite :: SingleSource/Benchmarks/McGill/queens.test    1301    1285       -1.2%
                 test-suite :: MultiSource/Benchmarks/MiBench/automotive-susan/automotive-susan.test   20485   20245       -1.2%
                                              test-suite :: SingleSource/Benchmarks/McGill/misr.test    1525    1509       -1.0%
                                          test-suite :: MultiSource/Benchmarks/McCat/18-imp/imp.test    9221    9125       -1.0%
                                             test-suite :: MultiSource/Benchmarks/Ptrdist/ft/ft.test    3205    3173       -1.0%
                                   test-suite :: MultiSource/Benchmarks/Fhourstones/fhourstones.test    3365    3333       -1.0%
                       test-suite :: MultiSource/Benchmarks/MiBench/consumer-lame/consumer-lame.test   88756   88148       -0.7%
                          test-suite :: MultiSource/Applications/ALAC/decode/alacconvert-decode.test   28709   28517       -0.7%
                          test-suite :: MultiSource/Applications/ALAC/encode/alacconvert-encode.test   28709   28517       -0.7%
                           test-suite :: MultiSource/Benchmarks/ASC_Sequoia/CrystalMk/CrystalMk.test    2501    2517        0.6%
                                     test-suite :: MultiSource/Benchmarks/FreeBench/pifft/pifft.test   33429   33221       -0.6%
                                       test-suite :: MultiSource/Benchmarks/SciMark2-C/scimark2.test    5445    5413       -0.6%
                                  test-suite :: MultiSource/Benchmarks/Prolangs-C/bison/mybison.test   34293   34101       -0.6%
                                               test-suite :: SingleSource/Benchmarks/Misc/flops.test    6117    6085       -0.5%
                                             test-suite :: MultiSource/Applications/spiff/spiff.test   15413   15333       -0.5%
                           test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C/miniAMR/miniAMR.test   37109   36917       -0.5%
                                         test-suite :: SingleSource/Benchmarks/Misc/ReedSolomon.test    3109    3125        0.5%
                                            test-suite :: SingleSource/Benchmarks/Misc/oourafft.test    6453    6421       -0.5%
                                    test-suite :: MultiSource/Benchmarks/Prolangs-C/agrep/agrep.test   29668   29524       -0.5%
                          test-suite :: MultiSource/Benchmarks/VersaBench/beamformer/beamformer.test    3461    3445       -0.5%
                                     test-suite :: SingleSource/Benchmarks/CoyoteBench/fftbench.test    3493    3477       -0.5%
                             test-suite :: MultiSource/Benchmarks/MallocBench/espresso/espresso.test   95733   95349       -0.4%
                               test-suite :: SingleSource/Benchmarks/Misc-C++/Large/sphereflake.test    4181    4165       -0.4%
                                         test-suite :: MultiSource/Applications/minisat/minisat.test   13413   13365       -0.4%
                                        test-suite :: MultiSource/Applications/JM/lencod/lencod.test  417589  416293       -0.3%
                                      test-suite :: MultiSource/Benchmarks/VersaBench/dbms/dbms.test   10645   10613       -0.3%
                           test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C/RSBench/rsbench.test   11173   11205        0.3%
                                           test-suite :: MultiSource/Applications/oggenc/oggenc.test  115716  115396       -0.3%
                                        test-suite :: MultiSource/Applications/JM/ldecod/ldecod.test  171989  171525       -0.3%
                                                   test-suite :: MultiSource/Benchmarks/sim/sim.test   11957   11925       -0.3%
                                      test-suite :: MultiSource/Benchmarks/mafft/pairlocalalign.test  221205  220629       -0.3%
                                             test-suite :: MultiSource/Benchmarks/Ptrdist/bc/bc.test   25061   24997       -0.3%
                                         test-suite :: MultiSource/Applications/obsequi/Obsequi.test   21141   21093       -0.2%
                           test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C++/miniFE/miniFE.test   43445   43349       -0.2%
                                       test-suite :: MultiSource/Benchmarks/7zip/7zip-benchmark.test  444644  443812       -0.2%
                                         test-suite :: MultiSource/Applications/SIBsim4/SIBsim4.test   27477   27525        0.2%
                                                 test-suite :: MultiSource/Applications/lua/lua.test   83317   83173       -0.2%
               test-suite :: MultiSource/Benchmarks/MiBench/security-rijndael/security-rijndael.test   10389   10373       -0.2%
                                             test-suite :: MultiSource/Benchmarks/nbench/nbench.test   20981   20949       -0.2%
                             test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C++/HPCCG/HPCCG.test   10821   10837        0.1%
                                 test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C/CoMD/CoMD.test   23077   23045       -0.1%
                                             test-suite :: MultiSource/Applications/SPASS/SPASS.test  284754  284386       -0.1%
                                             test-suite :: MultiSource/Benchmarks/Bullet/bullet.test  400821  400325       -0.1%
                                       test-suite :: MultiSource/Benchmarks/Ptrdist/yacr2/yacr2.test   14341   14325       -0.1%
                                         test-suite :: MultiSource/Applications/ClamAV/clamscan.test  352852  352484       -0.1%
                            test-suite :: MicroBenchmarks/LCALS/SubsetCLambdaLoops/lcalsCLambda.test  141716  141572       -0.1%
                                   test-suite :: MultiSource/Benchmarks/MallocBench/cfrac/cfrac.test   16037   16021       -0.1%
                                  test-suite :: MicroBenchmarks/LCALS/SubsetCRawLoops/lcalsCRaw.test  141636  141508       -0.1%
                                               test-suite :: MultiSource/Benchmarks/PAQ8p/paq8p.test   56085   56037       -0.1%
                                               test-suite :: MultiSource/Applications/siod/siod.test   60580   60532       -0.1%
                                         test-suite :: MultiSource/Applications/sqlite3/sqlite3.test  241315  241139       -0.1%
                                                    test-suite :: MicroBenchmarks/harris/harris.test   99381   99317       -0.1%
                            test-suite :: MicroBenchmarks/LCALS/SubsetALambdaLoops/lcalsALambda.test  139956  139876       -0.1%
                                          test-suite :: MultiSource/Applications/d/make_dparser.test   61347   61315       -0.1%
                               test-suite :: MultiSource/Benchmarks/ASCI_Purple/SMG2000/smg2000.test  138581  138645        0.0%
                                  test-suite :: MicroBenchmarks/LCALS/SubsetARawLoops/lcalsARaw.test  138932  138868       -0.0%
                           test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C/miniGMG/miniGMG.test   36805   36789       -0.0%
                                        test-suite :: MicroBenchmarks/ImageProcessing/Blur/blur.test   98981   98949       -0.0%
                                    test-suite :: MicroBenchmarks/ImageProcessing/Dither/Dither.test   99285   99253       -0.0%
                             test-suite :: MultiSource/Benchmarks/DOE-ProxyApps-C++/CLAMR/CLAMR.test  321508  321412       -0.0%
                                  test-suite :: MicroBenchmarks/LCALS/SubsetBRawLoops/lcalsBRaw.test  134756  134724       -0.0%
                             test-suite :: MultiSource/Benchmarks/mediabench/jpeg/jpeg-6a/cjpeg.test   84661   84677        0.0%
        test-suite :: MicroBenchmarks/ImageProcessing/AnisotropicDiffusion/AnisotropicDiffusion.test   98453   98437       -0.0%
                      test-suite :: MicroBenchmarks/ImageProcessing/Interpolation/Interpolation.test   99397   99381       -0.0%
                                         test-suite :: MultiSource/Benchmarks/MallocBench/gs/gs.test  102437  102421       -0.0%
                 test-suite :: MultiSource/Benchmarks/MiBench/consumer-typeset/consumer-typeset.test  391620  391572       -0.0%
                            test-suite :: MicroBenchmarks/LCALS/SubsetBLambdaLoops/lcalsBLambda.test  134788  134772       -0.0%
                                        test-suite :: MicroBenchmarks/MemFunctions/MemFunctions.test  177893  177877       -0.0%
                                     test-suite :: MultiSource/Benchmarks/tramp3d-v4/tramp3d-v4.test  296165  296149       -0.0%
                                            test-suite :: MultiSource/Applications/kimwitu++/kc.test  297476  297460       -0.0%
                                                                                  Geomean difference                       -0.1%
SjoerdMeijer added a comment.Aug 12 2020, 1:40 AM

Nice one!

llvm/lib/Transforms/Utils/ScalarEvolutionExpander.cpp
2408 ↗(On Diff #285000)

This change here looks like an exact duplication of the change above (lines 2355 - 2362). Can this be in a helper?

samparker added inline comments.Aug 12 2020, 1:41 AM
llvm/lib/Transforms/Utils/ScalarEvolutionExpander.cpp
2408 ↗(On Diff #285000)

Yeah, that would be nicer.

lebedev.ri requested changes to this revision.Aug 12 2020, 1:50 AM

Apologies if i'm missing the point here.

llvm/lib/Transforms/Utils/ScalarEvolutionExpander.cpp
2204 ↗(On Diff #285000)

s/get/accountFor/

2214–2217 ↗(On Diff #285000)

I'm not convinced this modelling is correct.
(which is why i didn't respond, but apparently i forgot to actually post that)

If we have SCEV x + y + 42, 42 will be modelled as-if it's at index 2,
but we should model this as (x + y) + 42, because there's usually some
form of an add that takes an immediate as second param.

2218–2221 ↗(On Diff #285000)

Same

2222–2229 ↗(On Diff #285000)

And again, if we have umin(x, y, 42), it's lowered as z = (x u< y) ? x : y ; z u< 42 ? z : 42,
so you can't possibly have third operand here.

2356–2357 ↗(On Diff #285000)

enumerate(NAry->operands())

2409–2410 ↗(On Diff #285000)

enumerate(NAry->operands())

This revision now requires changes to proceed.Aug 12 2020, 1:50 AM
samparker added inline comments.Aug 12 2020, 2:04 AM
llvm/lib/Transforms/Utils/ScalarEvolutionExpander.cpp
2214–2217 ↗(On Diff #285000)

Ah! Right, I'll try to fix that.

samparker updated this revision to Diff 285028.Aug 12 2020, 3:27 AM

Changed how operands are visited and costed, hopefully now translating the SCEV operand index correctly to an Instruction index.

SjoerdMeijer accepted this revision.Aug 14 2020, 12:53 AM

With @lebedev.ri last comment fixed, this now looks good to me. Please wait a day or so just in case there are more comments.

lebedev.ri requested changes to this revision.EditedAug 14 2020, 2:30 AM

I think this is going in the wrong direction.
I think the worklist needs to be changed, to be a struct { unsigned ParentOpcode; int OperandIdx; SCEV* S; },
then everything should suddenly become less convoluted/hand-wavey.

This revision now requires changes to proceed.Aug 14 2020, 2:30 AM
samparker added a comment.Aug 14 2020, 2:55 AM

Yeah, that sounds better, I'll put together a separate patch. Are you happy with the operand index clamping though?

lebedev.ri added a comment.Aug 14 2020, 3:02 AM
In D76434#2217823, @samparker wrote:

Yeah, that sounds better, I'll put together a separate patch.

Are you happy with the operand index clamping though?

Looks about right i guess.

llvm/lib/Transforms/Utils/ScalarEvolutionExpander.cpp
2195 ↗(On Diff #285028)

Can we still get a constant here?

2279–2282 ↗(On Diff #285028)

We are not taxing constants for right-shifts.

samparker added inline comments.Aug 14 2020, 6:59 AM
llvm/lib/Transforms/Utils/ScalarEvolutionExpander.cpp
2364 ↗(On Diff #285028)

Now I notice that I wouldn't have been handling AddRec expressions... So should these operands be added to the worklist for both Add and Mul or would just Add be okay?

lebedev.ri added a comment.Aug 14 2020, 7:17 AM

All this is (becoming?) so incredibly fragile...
Have you checked what happens if you simply make isHighCostExpansionHelper() always return true for -Oz? :)
This really needs refactoring/generalization.

llvm/lib/Transforms/Utils/ScalarEvolutionExpander.cpp
2364 ↗(On Diff #285028)

Given A + B*x, you'd want to model A as being at index 1,
and B as being part of multipler (again, at index 1.

And for higher orders A + B*x + C*x^2, again, B and C are part of multiply,
and it should be modelled as (B*x + C*x^2) + A.

So i think the generalization is that all nary operands except the first one are at index 1 of mul,
and the first nary operand is at index 1 of add.

samparker added a comment.Aug 14 2020, 7:37 AM

This really needs refactoring/generalization.

I will try...

Have you checked what happens if you simply make isHighCostExpansionHelper() always return true for -Oz? :)

I thought I did, but actually it was just for rewriting loop exit values... I will run the numbers though! But I'm also interested in whether this can still be beneficial for execution speed too.

llvm/lib/Transforms/Utils/ScalarEvolutionExpander.cpp
2364 ↗(On Diff #285028)

Okay, thanks.

samparker added a comment.Aug 17 2020, 12:28 AM

I just ran some numbers for when -Oz == HighCost and it's interesting... For Arm it's not good:

              master  minsize-high-cost-expand        diff                                                                                                                                        
count  310.000000     310.000000                310.000000                                                                                                                                        
mean   19710.077419   19662.567742              0.000666                                                                                                                                          
std    48882.246111   48773.870505              0.010779                                                                                                                                          
min    292.000000     292.000000               -0.042781                                                                                                                                          
25%    1090.000000    1084.000000              -0.002887                                                                                                                                          
50%    2776.000000    2784.000000               0.000000                                                                                                                                          
75%    10185.000000   10138.000000              0.002424                                                                                                                                          
max    324060.000000  323508.000000             0.058974
Geomean difference                                   0.1%

For AArch64 it's okay, but there are still plenty of large regressions:

              master  minsize-high-cost-expand        diff                                                                                                                                        
count  310.000000     310.000000                310.000000                                                                                                                                        
mean   26712.283871   26650.709677             -0.000930                                                                                                                                          
std    66147.455977   66033.469245              0.010996                                                                                                                                          
min    476.000000     476.000000               -0.050374                                                                                                                                          
25%    1294.000000    1304.000000              -0.003676                                                                                                                                          
50%    3168.000000    3168.000000               0.000000                                                                                                                                          
75%    11252.000000   11216.000000              0.001597                                                                                                                                          
max    471648.000000  470888.000000             0.051485
Geomean difference                                  -0.1%

But for X86, it's great:

              master  minsize-high-cost-expand        diff                                                                                                                                        
count  313.000000     313.000000                313.000000                                                                                                                                        
mean   27472.361022   27354.431310             -0.005084                                                                                                                                          
std    68350.278069   68101.639463              0.013028                                                                                                                                          
min    389.000000     389.000000               -0.065654                                                                                                                                          
25%    1157.000000    1157.000000              -0.007393                                                                                                                                          
50%    2933.000000    2949.000000               0.000000                                                                                                                                          
75%    11093.000000   11077.000000              0.000000                                                                                                                                          
max    444676.000000  442660.000000             0.039900
Geomean difference                                  -0.5%
samparker mentioned this in D86050: [SCEV] Refactor isHighCostExpansionHelper.Aug 17 2020, 1:41 AM
samparker mentioned this in D86072: [SCEV] Cost Add and Mul Expr consistently.Aug 17 2020, 7:09 AM
lebedev.ri added inline comments.Aug 17 2020, 7:15 AM
llvm/lib/Transforms/Utils/ScalarEvolutionExpander.cpp
2406 ↗(On Diff #285028)
In D86072#2221367, @samparker wrote:

In D76434 you highlighted by SCEVNAry expressions can have more than two operands, which would expand to a chain of operations, and the existing costs for AddRecExprs tries to account for that. But this was missing for normal Add and Mul expressions. Have I misunderstood you?

Doesn't look missing to me?

samparker added inline comments.Aug 17 2020, 7:26 AM
llvm/lib/Transforms/Utils/ScalarEvolutionExpander.cpp
2406 ↗(On Diff #285028)

Ah, thanks! I'm getting lost amongst the patches.

samparker updated this revision to Diff 286253.Aug 18 2020, 4:49 AM

Rebased on top of D86050 and introduced a struct to map operands for the worklist, instead of just storing the opcode.

samparker added a parent revision: D86050: [SCEV] Refactor isHighCostExpansionHelper.Aug 18 2020, 4:49 AM
samparker added a comment.Aug 24 2020, 11:48 PM

Ping

samparker updated this revision to Diff 290239.Sep 7 2020, 4:09 AM

Rebased now that the refactor patch is in.

This revision was not accepted when it landed; it landed in state Needs Review.Sep 10 2020, 12:23 AM
Closed by commit rG0bdf8c912724: [SCEV] Constant expansion cost at minsize (authored by samparker). · Explain Why
This revision was automatically updated to reflect the committed changes.
samparker added a commit: rG0bdf8c912724: [SCEV] Constant expansion cost at minsize.
samparker added a comment.Sep 10 2020, 12:24 AM

Let's discuss any further desired tweaks in a post-commit review.

lebedev.ri added a comment.Sep 10 2020, 12:30 AM

@samparker why did you commit this?
I have not finished reviewing this, and i've requested changes to the previous revisions.

llvm/lib/Transforms/Utils/ScalarEvolutionExpander.cpp
2187 ↗(On Diff #290893)

This needs a better comment.
Min/Max Idx are pretty magical on the first look.

2189 ↗(On Diff #290893)

Not llvm coding style in any case.
These should likely be Min/Max

2344 ↗(On Diff #290893)

Consider constants to be free unless we are optforsize

2347–2348 ↗(On Diff #290893)

These are only used in a single place

2352 ↗(On Diff #290893)

This is inconsistent with every other return here.

2192–2193 ↗(On Diff #290239)

SCEVOperand::OperandIdx is int

2195 ↗(On Diff #285028)

Please do mark done comments as such.

samparker marked 5 inline comments as done.Sep 10 2020, 12:56 AM

We had gone through many revisions where I've addressed all of your (helpful) comments and I hadn't heard anything else for three weeks. I sincerely expected the remaining issues to be style changes which seem appropriate for a post-commit review.

llvm/lib/Transforms/Utils/ScalarEvolutionExpander.cpp
2192–2193 ↗(On Diff #290239)

But when we enumerate the SCEV operands, the index is size_t and the types need to be compatible for std::max and min.

lebedev.ri added inline comments.Sep 10 2020, 1:01 AM
llvm/lib/Transforms/Utils/ScalarEvolutionExpander.cpp
2192–2193 ↗(On Diff #290239)
int MinIdx = std::max((int)SCEVOp.index(), CostOp.MinIdx);
int OpIdx = std::min(MinIdx, CostOp.MaxIdx);
samparker updated this revision to Diff 290902.Sep 10 2020, 1:07 AM

Addressed comments.

Revision Contents

PathSize
llvm/
include/
llvm/
Analysis/
2 lines
lib/
Analysis/
10 lines
test/
CodeGen/
ARM/
134 lines
411 lines
Transforms/
IndVarSimplify/
56 lines
16 lines
22 lines
46 lines
LoopUnroll/
PowerPC/
121 lines
LoopVectorize/
X86/
264 lines
SpeculateAroundPHIs/
7 lines

Diff 251403

llvm/include/llvm/Analysis/TargetTransformInfoImpl.h

Show First 20 Lines • Show All 358 Lines • ▼ Show 20 Linespublic:
unsigned getFPOpCost(Type *Ty) { return TargetTransformInfo::TCC_Basic; } unsigned getFPOpCost(Type *Ty) { return TargetTransformInfo::TCC_Basic; }
int getIntImmCodeSizeCost(unsigned Opcode, unsigned Idx, const APInt &Imm, int getIntImmCodeSizeCost(unsigned Opcode, unsigned Idx, const APInt &Imm,
Type *Ty) { Type *Ty) {
return 0; return 0;
} }
unsigned getIntImmCost(const APInt &Imm, Type *Ty) { return TTI::TCC_Basic; } unsigned getIntImmCost(const APInt &Imm, Type *Ty) { return TTI::TCC_Free; }
unsigned getIntImmCostInst(unsigned Opcode, unsigned Idx, const APInt &Imm, unsigned getIntImmCostInst(unsigned Opcode, unsigned Idx, const APInt &Imm,
Type *Ty) { Type *Ty) {
return TTI::TCC_Free; return TTI::TCC_Free;
} }
unsigned getIntImmCostIntrin(Intrinsic::ID IID, unsigned Idx, unsigned getIntImmCostIntrin(Intrinsic::ID IID, unsigned Idx,
const APInt &Imm, Type *Ty) { const APInt &Imm, Type *Ty) {
▲ Show 20 Lines • Show All 492 Lines • Show Last 20 Lines

llvm/lib/Analysis/ScalarEvolutionExpander.cpp

Show First 20 Lines • Show All 492 Lines • ▼ Show 20 Lines
if (!Processed.insert(S).second) if (!Processed.insert(S).second)
return false; // We have already accounted for this expression. return false; // We have already accounted for this expression.
// If we can find an existing value for this scev available at the point "At" // If we can find an existing value for this scev available at the point "At"
// then consider the expression cheap. // then consider the expression cheap.
if (getRelatedExistingExpansion(S, &At, L)) if (getRelatedExistingExpansion(S, &At, L))
return false; // Consider the expression to be free. return false; // Consider the expression to be free.
switch (S->getSCEVType()) { if (isa<SCEVUnknown>(S))
case scUnknown: return false;
case scConstant:
return false; // Assume to be zero-cost. if (auto *Const = dyn_cast<SCEVConstant>(S)) {
BudgetRemaining -= TTI.getIntImmCost(Const->getAPInt(), S->getType());
return BudgetRemaining < 0;
} }
if (auto *CastExpr = dyn_cast<SCEVCastExpr>(S)) { if (auto *CastExpr = dyn_cast<SCEVCastExpr>(S)) {
unsigned Opcode; unsigned Opcode;
switch (S->getSCEVType()) { switch (S->getSCEVType()) {
case scTruncate: case scTruncate:
Opcode = Instruction::Trunc; Opcode = Instruction::Trunc;
break; break;
▲ Show 20 Lines • Show All 389 Lines • Show Last 20 Lines

llvm/test/CodeGen/ARM/indvar-cost.ll

; NOTE: Assertions have been autogenerated by utils/update_test_checks.py ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
; RUN: opt -indvars -mtriple=thumbv8m.base -S %s -o - | FileCheck %s --check-prefix=CHECK-T1 ; RUN: opt -indvars -mtriple=thumbv8m.base -S %s -o - | FileCheck %s --check-prefix=CHECK-T1
; RUN: opt -indvars -mtriple=thumbv8m.main -S %s -o - | FileCheck %s --check-prefix=CHECK-T2 ; RUN: opt -indvars -mtriple=thumbv8m.main -S %s -o - | FileCheck %s --check-prefix=CHECK-T2
define dso_local arm_aapcscc void @arm_conv_fast_q15(i16* %pSrcA, i32 %srcALen, i16* %pSrcB, i32 %srcBLen, i16* %pDst, i16** %store.px, i16** %store.py, i32* %store.res) local_unnamed_addr { define dso_local arm_aapcscc void @arm_conv_fast_q15(i16* %pSrcA, i32 %srcALen, i16* %pSrcB, i32 %srcBLen, i16* %pDst, i16** %store.px, i16** %store.py, i32* %store.res) local_unnamed_addr {
; CHECK-T1-LABEL: @arm_conv_fast_q15( ; CHECK-T1-LABEL: @arm_conv_fast_q15(
; CHECK-T1-NEXT: entry: ; CHECK-T1-NEXT: entry:
; CHECK-T1-NEXT: [[CMP:%.*]] = icmp ult i32 [[SRCALEN:%.*]], [[SRCBLEN:%.*]] ; CHECK-T1-NEXT: [[CMP:%.*]] = icmp ult i32 [[SRCALEN:%.*]], [[SRCBLEN:%.*]]
; CHECK-T1-NEXT: [[SRCALEN_SRCBLEN:%.*]] = select i1 [[CMP]], i32 [[SRCALEN]], i32 [[SRCBLEN]] ; CHECK-T1-NEXT: [[SRCALEN_SRCBLEN:%.*]] = select i1 [[CMP]], i32 [[SRCALEN]], i32 [[SRCBLEN]]
; CHECK-T1-NEXT: [[PSRCB_PSRCA:%.*]] = select i1 [[CMP]], i16* [[PSRCB:%.*]], i16* [[PSRCA:%.*]] ; CHECK-T1-NEXT: [[PSRCB_PSRCA:%.*]] = select i1 [[CMP]], i16* [[PSRCB:%.*]], i16* [[PSRCA:%.*]]
; CHECK-T1-NEXT: [[PSRCA_PSRCB:%.*]] = select i1 [[CMP]], i16* [[PSRCA]], i16* [[PSRCB]] ; CHECK-T1-NEXT: [[PSRCA_PSRCB:%.*]] = select i1 [[CMP]], i16* [[PSRCA]], i16* [[PSRCB]]
; CHECK-T1-NEXT: [[SUB:%.*]] = add i32 [[SRCALEN_SRCBLEN]], -1 ; CHECK-T1-NEXT: [[SUB:%.*]] = add i32 [[SRCALEN_SRCBLEN]], -1
; CHECK-T1-NEXT: [[CMP41080:%.*]] = icmp eq i32 [[SUB]], 0 ; CHECK-T1-NEXT: [[CMP41080:%.*]] = icmp eq i32 [[SUB]], 0
; CHECK-T1-NEXT: br i1 [[CMP41080]], label [[WHILE_END13:%.*]], label [[WHILE_COND5_PREHEADER_PREHEADER:%.*]] ; CHECK-T1-NEXT: br i1 [[CMP41080]], label [[WHILE_END13:%.*]], label [[WHILE_COND5_PREHEADER_PREHEADER:%.*]]
; CHECK-T1: while.cond5.preheader.preheader: ; CHECK-T1: while.cond5.preheader.preheader:
; CHECK-T1-NEXT: [[TMP0:%.*]] = add i32 [[SRCALEN_SRCBLEN]], -2
; CHECK-T1-NEXT: [[TMP1:%.*]] = icmp ult i32 [[TMP0]], 2
; CHECK-T1-NEXT: [[UMIN:%.*]] = select i1 [[TMP1]], i32 [[TMP0]], i32 2
; CHECK-T1-NEXT: br label [[WHILE_COND5_PREHEADER:%.*]] ; CHECK-T1-NEXT: br label [[WHILE_COND5_PREHEADER:%.*]]
; CHECK-T1: while.cond5.preheader: ; CHECK-T1: while.cond5.preheader:
; CHECK-T1-NEXT: [[COUNT_01084:%.*]] = phi i32 [ [[INC:%.*]], [[WHILE_END:%.*]] ], [ 1, [[WHILE_COND5_PREHEADER_PREHEADER]] ] ; CHECK-T1-NEXT: [[COUNT_01084:%.*]] = phi i32 [ [[INC:%.*]], [[WHILE_END:%.*]] ], [ 1, [[WHILE_COND5_PREHEADER_PREHEADER]] ]
; CHECK-T1-NEXT: [[BLOCKSIZE1_01083:%.*]] = phi i32 [ [[DEC12:%.*]], [[WHILE_END]] ], [ [[SUB]], [[WHILE_COND5_PREHEADER_PREHEADER]] ] ; CHECK-T1-NEXT: [[BLOCKSIZE1_01083:%.*]] = phi i32 [ [[DEC12:%.*]], [[WHILE_END]] ], [ [[SUB]], [[WHILE_COND5_PREHEADER_PREHEADER]] ]
; CHECK-T1-NEXT: [[PY_01082:%.*]] = phi i16* [ [[ADD_PTR:%.*]], [[WHILE_END]] ], [ [[PSRCA_PSRCB]], [[WHILE_COND5_PREHEADER_PREHEADER]] ] ; CHECK-T1-NEXT: [[PY_01082:%.*]] = phi i16* [ [[ADD_PTR:%.*]], [[WHILE_END]] ], [ [[PSRCA_PSRCB]], [[WHILE_COND5_PREHEADER_PREHEADER]] ]
; CHECK-T1-NEXT: [[POUT_01081:%.*]] = phi i16* [ [[INCDEC_PTR11:%.*]], [[WHILE_END]] ], [ [[PDST:%.*]], [[WHILE_COND5_PREHEADER_PREHEADER]] ] ; CHECK-T1-NEXT: [[POUT_01081:%.*]] = phi i16* [ [[INCDEC_PTR11:%.*]], [[WHILE_END]] ], [ [[PDST:%.*]], [[WHILE_COND5_PREHEADER_PREHEADER]] ]
; CHECK-T1-NEXT: br label [[WHILE_BODY7:%.*]] ; CHECK-T1-NEXT: br label [[WHILE_BODY7:%.*]]
; CHECK-T1: while.body7: ; CHECK-T1: while.body7:
; CHECK-T1-NEXT: [[K_01078:%.*]] = phi i32 [ [[DEC:%.*]], [[WHILE_BODY7]] ], [ [[COUNT_01084]], [[WHILE_COND5_PREHEADER]] ] ; CHECK-T1-NEXT: [[K_01078:%.*]] = phi i32 [ [[DEC:%.*]], [[WHILE_BODY7]] ], [ [[COUNT_01084]], [[WHILE_COND5_PREHEADER]] ]
; CHECK-T1-NEXT: [[SUM_01077:%.*]] = phi i32 [ [[ADD6_I:%.*]], [[WHILE_BODY7]] ], [ 0, [[WHILE_COND5_PREHEADER]] ] ; CHECK-T1-NEXT: [[SUM_01077:%.*]] = phi i32 [ [[ADD6_I:%.*]], [[WHILE_BODY7]] ], [ 0, [[WHILE_COND5_PREHEADER]] ]
; CHECK-T1-NEXT: [[PY_11076:%.*]] = phi i16* [ [[INCDEC_PTR8:%.*]], [[WHILE_BODY7]] ], [ [[PY_01082]], [[WHILE_COND5_PREHEADER]] ] ; CHECK-T1-NEXT: [[PY_11076:%.*]] = phi i16* [ [[INCDEC_PTR8:%.*]], [[WHILE_BODY7]] ], [ [[PY_01082]], [[WHILE_COND5_PREHEADER]] ]
; CHECK-T1-NEXT: [[PX_11075:%.*]] = phi i16* [ [[INCDEC_PTR:%.*]], [[WHILE_BODY7]] ], [ [[PSRCB_PSRCA]], [[WHILE_COND5_PREHEADER]] ] ; CHECK-T1-NEXT: [[PX_11075:%.*]] = phi i16* [ [[INCDEC_PTR:%.*]], [[WHILE_BODY7]] ], [ [[PSRCB_PSRCA]], [[WHILE_COND5_PREHEADER]] ]
; CHECK-T1-NEXT: [[INCDEC_PTR]] = getelementptr inbounds i16, i16* [[PX_11075]], i32 1 ; CHECK-T1-NEXT: [[INCDEC_PTR]] = getelementptr inbounds i16, i16* [[PX_11075]], i32 1
; CHECK-T1-NEXT: [[TMP2:%.*]] = load i16, i16* [[PX_11075]], align 2 ; CHECK-T1-NEXT: [[TMP0:%.*]] = load i16, i16* [[PX_11075]], align 2
; CHECK-T1-NEXT: [[CONV:%.*]] = sext i16 [[TMP2]] to i32 ; CHECK-T1-NEXT: [[CONV:%.*]] = sext i16 [[TMP0]] to i32
; CHECK-T1-NEXT: [[INCDEC_PTR8]] = getelementptr inbounds i16, i16* [[PY_11076]], i32 -1 ; CHECK-T1-NEXT: [[INCDEC_PTR8]] = getelementptr inbounds i16, i16* [[PY_11076]], i32 -1
; CHECK-T1-NEXT: [[TMP3:%.*]] = load i16, i16* [[PY_11076]], align 2 ; CHECK-T1-NEXT: [[TMP1:%.*]] = load i16, i16* [[PY_11076]], align 2
; CHECK-T1-NEXT: [[CONV9:%.*]] = sext i16 [[TMP3]] to i32 ; CHECK-T1-NEXT: [[CONV9:%.*]] = sext i16 [[TMP1]] to i32
; CHECK-T1-NEXT: [[MUL_I:%.*]] = mul nsw i32 [[CONV9]], [[CONV]] ; CHECK-T1-NEXT: [[MUL_I:%.*]] = mul nsw i32 [[CONV9]], [[CONV]]
; CHECK-T1-NEXT: [[SHR3_I:%.*]] = ashr i32 [[CONV]], 16 ; CHECK-T1-NEXT: [[SHR3_I:%.*]] = ashr i32 [[CONV]], 16
; CHECK-T1-NEXT: [[SHR4_I:%.*]] = ashr i32 [[CONV9]], 16 ; CHECK-T1-NEXT: [[SHR4_I:%.*]] = ashr i32 [[CONV9]], 16
; CHECK-T1-NEXT: [[MUL5_I:%.*]] = mul nsw i32 [[SHR4_I]], [[SHR3_I]] ; CHECK-T1-NEXT: [[MUL5_I:%.*]] = mul nsw i32 [[SHR4_I]], [[SHR3_I]]
; CHECK-T1-NEXT: [[ADD_I:%.*]] = add i32 [[MUL_I]], [[SUM_01077]] ; CHECK-T1-NEXT: [[ADD_I:%.*]] = add i32 [[MUL_I]], [[SUM_01077]]
; CHECK-T1-NEXT: [[ADD6_I]] = add i32 [[ADD_I]], [[MUL5_I]] ; CHECK-T1-NEXT: [[ADD6_I]] = add i32 [[ADD_I]], [[MUL5_I]]
; CHECK-T1-NEXT: [[DEC]] = add nsw i32 [[K_01078]], -1 ; CHECK-T1-NEXT: [[DEC]] = add nsw i32 [[K_01078]], -1
; CHECK-T1-NEXT: [[CMP6:%.*]] = icmp eq i32 [[DEC]], 0 ; CHECK-T1-NEXT: [[CMP6:%.*]] = icmp eq i32 [[DEC]], 0
; CHECK-T1-NEXT: br i1 [[CMP6]], label [[WHILE_END]], label [[WHILE_BODY7]] ; CHECK-T1-NEXT: br i1 [[CMP6]], label [[WHILE_END]], label [[WHILE_BODY7]]
; CHECK-T1: while.end: ; CHECK-T1: while.end:
; CHECK-T1-NEXT: [[ADD6_I_LCSSA:%.*]] = phi i32 [ [[ADD6_I]], [[WHILE_BODY7]] ] ; CHECK-T1-NEXT: [[ADD6_I_LCSSA:%.*]] = phi i32 [ [[ADD6_I]], [[WHILE_BODY7]] ]
; CHECK-T1-NEXT: [[TMP4:%.*]] = lshr i32 [[ADD6_I_LCSSA]], 15 ; CHECK-T1-NEXT: [[TMP2:%.*]] = lshr i32 [[ADD6_I_LCSSA]], 15
; CHECK-T1-NEXT: [[CONV10:%.*]] = trunc i32 [[TMP4]] to i16 ; CHECK-T1-NEXT: [[CONV10:%.*]] = trunc i32 [[TMP2]] to i16
; CHECK-T1-NEXT: [[INCDEC_PTR11]] = getelementptr inbounds i16, i16* [[POUT_01081]], i32 1 ; CHECK-T1-NEXT: [[INCDEC_PTR11]] = getelementptr inbounds i16, i16* [[POUT_01081]], i32 1
; CHECK-T1-NEXT: store i16 [[CONV10]], i16* [[POUT_01081]], align 2 ; CHECK-T1-NEXT: store i16 [[CONV10]], i16* [[POUT_01081]], align 2
; CHECK-T1-NEXT: [[ADD_PTR]] = getelementptr inbounds i16, i16* [[PSRCA_PSRCB]], i32 [[COUNT_01084]] ; CHECK-T1-NEXT: [[ADD_PTR]] = getelementptr inbounds i16, i16* [[PSRCA_PSRCB]], i32 [[COUNT_01084]]
; CHECK-T1-NEXT: [[INC]] = add nuw nsw i32 [[COUNT_01084]], 1 ; CHECK-T1-NEXT: [[INC]] = add nuw nsw i32 [[COUNT_01084]], 1
; CHECK-T1-NEXT: [[DEC12]] = add i32 [[BLOCKSIZE1_01083]], -1 ; CHECK-T1-NEXT: [[DEC12]] = add i32 [[BLOCKSIZE1_01083]], -1
; CHECK-T1-NEXT: [[CMP3:%.*]] = icmp ult i32 [[COUNT_01084]], 3 ; CHECK-T1-NEXT: [[CMP3:%.*]] = icmp ult i32 [[COUNT_01084]], 3
; CHECK-T1-NEXT: [[CMP4:%.*]] = icmp ne i32 [[DEC12]], 0 ; CHECK-T1-NEXT: [[CMP4:%.*]] = icmp ne i32 [[DEC12]], 0
; CHECK-T1-NEXT: [[TMP5:%.*]] = and i1 [[CMP4]], [[CMP3]] ; CHECK-T1-NEXT: [[TMP3:%.*]] = and i1 [[CMP4]], [[CMP3]]
; CHECK-T1-NEXT: br i1 [[TMP5]], label [[WHILE_COND5_PREHEADER]], label [[WHILE_END13_LOOPEXIT:%.*]] ; CHECK-T1-NEXT: br i1 [[TMP3]], label [[WHILE_COND5_PREHEADER]], label [[WHILE_END13_LOOPEXIT:%.*]]
; CHECK-T1: while.end13.loopexit: ; CHECK-T1: while.end13.loopexit:
; CHECK-T1-NEXT: [[INCDEC_PTR11_LCSSA:%.*]] = phi i16* [ [[INCDEC_PTR11]], [[WHILE_END]] ] ; CHECK-T1-NEXT: [[INCDEC_PTR11_LCSSA:%.*]] = phi i16* [ [[INCDEC_PTR11]], [[WHILE_END]] ]
; CHECK-T1-NEXT: [[ADD_PTR_LCSSA:%.*]] = phi i16* [ [[ADD_PTR]], [[WHILE_END]] ] ; CHECK-T1-NEXT: [[ADD_PTR_LCSSA:%.*]] = phi i16* [ [[ADD_PTR]], [[WHILE_END]] ]
; CHECK-T1-NEXT: [[INC_LCSSA:%.*]] = phi i32 [ [[INC]], [[WHILE_END]] ]
; CHECK-T1-NEXT: [[DEC12_LCSSA:%.*]] = phi i32 [ [[DEC12]], [[WHILE_END]] ] ; CHECK-T1-NEXT: [[DEC12_LCSSA:%.*]] = phi i32 [ [[DEC12]], [[WHILE_END]] ]
; CHECK-T1-NEXT: [[TMP6:%.*]] = add nuw nsw i32 [[UMIN]], 2
; CHECK-T1-NEXT: br label [[WHILE_END13]] ; CHECK-T1-NEXT: br label [[WHILE_END13]]
; CHECK-T1: while.end13: ; CHECK-T1: while.end13:
; CHECK-T1-NEXT: [[POUT_0_LCSSA:%.*]] = phi i16* [ [[PDST]], [[ENTRY:%.*]] ], [ [[INCDEC_PTR11_LCSSA]], [[WHILE_END13_LOOPEXIT]] ] ; CHECK-T1-NEXT: [[POUT_0_LCSSA:%.*]] = phi i16* [ [[PDST]], [[ENTRY:%.*]] ], [ [[INCDEC_PTR11_LCSSA]], [[WHILE_END13_LOOPEXIT]] ]
; CHECK-T1-NEXT: [[PY_0_LCSSA:%.*]] = phi i16* [ [[PSRCA_PSRCB]], [[ENTRY]] ], [ [[ADD_PTR_LCSSA]], [[WHILE_END13_LOOPEXIT]] ] ; CHECK-T1-NEXT: [[PY_0_LCSSA:%.*]] = phi i16* [ [[PSRCA_PSRCB]], [[ENTRY]] ], [ [[ADD_PTR_LCSSA]], [[WHILE_END13_LOOPEXIT]] ]
; CHECK-T1-NEXT: [[BLOCKSIZE1_0_LCSSA:%.*]] = phi i32 [ [[SUB]], [[ENTRY]] ], [ [[DEC12_LCSSA]], [[WHILE_END13_LOOPEXIT]] ] ; CHECK-T1-NEXT: [[BLOCKSIZE1_0_LCSSA:%.*]] = phi i32 [ [[SUB]], [[ENTRY]] ], [ [[DEC12_LCSSA]], [[WHILE_END13_LOOPEXIT]] ]
; CHECK-T1-NEXT: [[COUNT_0_LCSSA:%.*]] = phi i32 [ 1, [[ENTRY]] ], [ [[TMP6]], [[WHILE_END13_LOOPEXIT]] ] ; CHECK-T1-NEXT: [[COUNT_0_LCSSA:%.*]] = phi i32 [ 1, [[ENTRY]] ], [ [[INC_LCSSA]], [[WHILE_END13_LOOPEXIT]] ]
; CHECK-T1-NEXT: [[CMP161068:%.*]] = icmp eq i32 [[BLOCKSIZE1_0_LCSSA]], 0 ; CHECK-T1-NEXT: [[CMP161068:%.*]] = icmp eq i32 [[BLOCKSIZE1_0_LCSSA]], 0
; CHECK-T1-NEXT: br i1 [[CMP161068]], label [[EXIT:%.*]], label [[WHILE_BODY18_PREHEADER:%.*]] ; CHECK-T1-NEXT: br i1 [[CMP161068]], label [[EXIT:%.*]], label [[WHILE_BODY18_PREHEADER:%.*]]
; CHECK-T1: while.body18.preheader: ; CHECK-T1: while.body18.preheader:
; CHECK-T1-NEXT: [[ADD_PTR14:%.*]] = getelementptr inbounds i16, i16* [[PY_0_LCSSA]], i32 -1 ; CHECK-T1-NEXT: [[ADD_PTR14:%.*]] = getelementptr inbounds i16, i16* [[PY_0_LCSSA]], i32 -1
; CHECK-T1-NEXT: br label [[WHILE_BODY18:%.*]] ; CHECK-T1-NEXT: br label [[WHILE_BODY18:%.*]]
; CHECK-T1: while.body18: ; CHECK-T1: while.body18:
; CHECK-T1-NEXT: [[COUNT_11072:%.*]] = phi i32 [ [[INC49:%.*]], [[WHILE_END43:%.*]] ], [ [[COUNT_0_LCSSA]], [[WHILE_BODY18_PREHEADER]] ] ; CHECK-T1-NEXT: [[COUNT_11072:%.*]] = phi i32 [ [[INC49:%.*]], [[WHILE_END43:%.*]] ], [ [[COUNT_0_LCSSA]], [[WHILE_BODY18_PREHEADER]] ]
; CHECK-T1-NEXT: [[BLOCKSIZE1_11071:%.*]] = phi i32 [ [[DEC50:%.*]], [[WHILE_END43]] ], [ [[BLOCKSIZE1_0_LCSSA]], [[WHILE_BODY18_PREHEADER]] ] ; CHECK-T1-NEXT: [[BLOCKSIZE1_11071:%.*]] = phi i32 [ [[DEC50:%.*]], [[WHILE_END43]] ], [ [[BLOCKSIZE1_0_LCSSA]], [[WHILE_BODY18_PREHEADER]] ]
; CHECK-T1-NEXT: [[PY_21070:%.*]] = phi i16* [ [[ADD_PTR48:%.*]], [[WHILE_END43]] ], [ [[ADD_PTR14]], [[WHILE_BODY18_PREHEADER]] ] ; CHECK-T1-NEXT: [[PY_21070:%.*]] = phi i16* [ [[ADD_PTR48:%.*]], [[WHILE_END43]] ], [ [[ADD_PTR14]], [[WHILE_BODY18_PREHEADER]] ]
; CHECK-T1-NEXT: [[POUT_11069:%.*]] = phi i16* [ [[INCDEC_PTR46:%.*]], [[WHILE_END43]] ], [ [[POUT_0_LCSSA]], [[WHILE_BODY18_PREHEADER]] ] ; CHECK-T1-NEXT: [[POUT_11069:%.*]] = phi i16* [ [[INCDEC_PTR46:%.*]], [[WHILE_END43]] ], [ [[POUT_0_LCSSA]], [[WHILE_BODY18_PREHEADER]] ]
; CHECK-T1-NEXT: [[SHR19:%.*]] = lshr i32 [[COUNT_11072]], 2 ; CHECK-T1-NEXT: [[SHR19:%.*]] = lshr i32 [[COUNT_11072]], 2
; CHECK-T1-NEXT: [[CMP211054:%.*]] = icmp eq i32 [[SHR19]], 0 ; CHECK-T1-NEXT: [[CMP211054:%.*]] = icmp eq i32 [[SHR19]], 0
; CHECK-T1-NEXT: br i1 [[CMP211054]], label [[WHILE_END31:%.*]], label [[WHILE_BODY23_PREHEADER:%.*]] ; CHECK-T1-NEXT: br i1 [[CMP211054]], label [[WHILE_END31:%.*]], label [[WHILE_BODY23_PREHEADER:%.*]]
; CHECK-T1: while.body23.preheader: ; CHECK-T1: while.body23.preheader:
; CHECK-T1-NEXT: br label [[WHILE_BODY23:%.*]] ; CHECK-T1-NEXT: br label [[WHILE_BODY23:%.*]]
; CHECK-T1: while.body23: ; CHECK-T1: while.body23:
; CHECK-T1-NEXT: [[K_11058:%.*]] = phi i32 [ [[DEC30:%.*]], [[WHILE_BODY23]] ], [ [[SHR19]], [[WHILE_BODY23_PREHEADER]] ] ; CHECK-T1-NEXT: [[K_11058:%.*]] = phi i32 [ [[DEC30:%.*]], [[WHILE_BODY23]] ], [ [[SHR19]], [[WHILE_BODY23_PREHEADER]] ]
; CHECK-T1-NEXT: [[SUM_11057:%.*]] = phi i32 [ [[ADD6_I878:%.*]], [[WHILE_BODY23]] ], [ 0, [[WHILE_BODY23_PREHEADER]] ] ; CHECK-T1-NEXT: [[SUM_11057:%.*]] = phi i32 [ [[ADD6_I878:%.*]], [[WHILE_BODY23]] ], [ 0, [[WHILE_BODY23_PREHEADER]] ]
; CHECK-T1-NEXT: [[PY_31056:%.*]] = phi i16* [ [[ADD_PTR_I884:%.*]], [[WHILE_BODY23]] ], [ [[PY_21070]], [[WHILE_BODY23_PREHEADER]] ] ; CHECK-T1-NEXT: [[PY_31056:%.*]] = phi i16* [ [[ADD_PTR_I884:%.*]], [[WHILE_BODY23]] ], [ [[PY_21070]], [[WHILE_BODY23_PREHEADER]] ]
; CHECK-T1-NEXT: [[PX_31055:%.*]] = phi i16* [ [[ADD_PTR_I890:%.*]], [[WHILE_BODY23]] ], [ [[PSRCB_PSRCA]], [[WHILE_BODY23_PREHEADER]] ] ; CHECK-T1-NEXT: [[PX_31055:%.*]] = phi i16* [ [[ADD_PTR_I890:%.*]], [[WHILE_BODY23]] ], [ [[PSRCB_PSRCA]], [[WHILE_BODY23_PREHEADER]] ]
; CHECK-T1-NEXT: [[ARRAYIDX_I907:%.*]] = getelementptr inbounds i16, i16* [[PX_31055]], i32 1 ; CHECK-T1-NEXT: [[ARRAYIDX_I907:%.*]] = getelementptr inbounds i16, i16* [[PX_31055]], i32 1
; CHECK-T1-NEXT: [[TMP7:%.*]] = load i16, i16* [[ARRAYIDX_I907]], align 2 ; CHECK-T1-NEXT: [[TMP4:%.*]] = load i16, i16* [[ARRAYIDX_I907]], align 2
; CHECK-T1-NEXT: [[TMP8:%.*]] = load i16, i16* [[PX_31055]], align 2 ; CHECK-T1-NEXT: [[TMP5:%.*]] = load i16, i16* [[PX_31055]], align 2
; CHECK-T1-NEXT: [[ADD_PTR_I912:%.*]] = getelementptr inbounds i16, i16* [[PX_31055]], i32 2 ; CHECK-T1-NEXT: [[ADD_PTR_I912:%.*]] = getelementptr inbounds i16, i16* [[PX_31055]], i32 2
; CHECK-T1-NEXT: [[ARRAYIDX_I901:%.*]] = getelementptr inbounds i16, i16* [[PY_31056]], i32 1 ; CHECK-T1-NEXT: [[ARRAYIDX_I901:%.*]] = getelementptr inbounds i16, i16* [[PY_31056]], i32 1
; CHECK-T1-NEXT: [[TMP9:%.*]] = load i16, i16* [[ARRAYIDX_I901]], align 2 ; CHECK-T1-NEXT: [[TMP6:%.*]] = load i16, i16* [[ARRAYIDX_I901]], align 2
; CHECK-T1-NEXT: [[TMP10:%.*]] = load i16, i16* [[PY_31056]], align 2 ; CHECK-T1-NEXT: [[TMP7:%.*]] = load i16, i16* [[PY_31056]], align 2
; CHECK-T1-NEXT: [[ADD_PTR_I906:%.*]] = getelementptr inbounds i16, i16* [[PY_31056]], i32 -2 ; CHECK-T1-NEXT: [[ADD_PTR_I906:%.*]] = getelementptr inbounds i16, i16* [[PY_31056]], i32 -2
; CHECK-T1-NEXT: [[SHR_I892:%.*]] = sext i16 [[TMP8]] to i32 ; CHECK-T1-NEXT: [[SHR_I892:%.*]] = sext i16 [[TMP5]] to i32
; CHECK-T1-NEXT: [[SHR1_I893:%.*]] = sext i16 [[TMP9]] to i32 ; CHECK-T1-NEXT: [[SHR1_I893:%.*]] = sext i16 [[TMP6]] to i32
; CHECK-T1-NEXT: [[MUL_I894:%.*]] = mul nsw i32 [[SHR1_I893]], [[SHR_I892]] ; CHECK-T1-NEXT: [[MUL_I894:%.*]] = mul nsw i32 [[SHR1_I893]], [[SHR_I892]]
; CHECK-T1-NEXT: [[SHR2_I895:%.*]] = sext i16 [[TMP7]] to i32 ; CHECK-T1-NEXT: [[SHR2_I895:%.*]] = sext i16 [[TMP4]] to i32
; CHECK-T1-NEXT: [[SHR4_I897:%.*]] = sext i16 [[TMP10]] to i32 ; CHECK-T1-NEXT: [[SHR4_I897:%.*]] = sext i16 [[TMP7]] to i32
; CHECK-T1-NEXT: [[MUL5_I898:%.*]] = mul nsw i32 [[SHR4_I897]], [[SHR2_I895]] ; CHECK-T1-NEXT: [[MUL5_I898:%.*]] = mul nsw i32 [[SHR4_I897]], [[SHR2_I895]]
; CHECK-T1-NEXT: [[ADD_I899:%.*]] = add i32 [[MUL_I894]], [[SUM_11057]] ; CHECK-T1-NEXT: [[ADD_I899:%.*]] = add i32 [[MUL_I894]], [[SUM_11057]]
; CHECK-T1-NEXT: [[ADD6_I900:%.*]] = add i32 [[ADD_I899]], [[MUL5_I898]] ; CHECK-T1-NEXT: [[ADD6_I900:%.*]] = add i32 [[ADD_I899]], [[MUL5_I898]]
; CHECK-T1-NEXT: [[ARRAYIDX_I885:%.*]] = getelementptr inbounds i16, i16* [[PX_31055]], i32 3 ; CHECK-T1-NEXT: [[ARRAYIDX_I885:%.*]] = getelementptr inbounds i16, i16* [[PX_31055]], i32 3
; CHECK-T1-NEXT: [[TMP11:%.*]] = load i16, i16* [[ARRAYIDX_I885]], align 2 ; CHECK-T1-NEXT: [[TMP8:%.*]] = load i16, i16* [[ARRAYIDX_I885]], align 2
; CHECK-T1-NEXT: [[TMP12:%.*]] = load i16, i16* [[ADD_PTR_I912]], align 2 ; CHECK-T1-NEXT: [[TMP9:%.*]] = load i16, i16* [[ADD_PTR_I912]], align 2
; CHECK-T1-NEXT: [[ADD_PTR_I890]] = getelementptr inbounds i16, i16* [[PX_31055]], i32 4 ; CHECK-T1-NEXT: [[ADD_PTR_I890]] = getelementptr inbounds i16, i16* [[PX_31055]], i32 4
; CHECK-T1-NEXT: [[ARRAYIDX_I879:%.*]] = getelementptr inbounds i16, i16* [[PY_31056]], i32 -1 ; CHECK-T1-NEXT: [[ARRAYIDX_I879:%.*]] = getelementptr inbounds i16, i16* [[PY_31056]], i32 -1
; CHECK-T1-NEXT: [[TMP13:%.*]] = load i16, i16* [[ARRAYIDX_I879]], align 2 ; CHECK-T1-NEXT: [[TMP10:%.*]] = load i16, i16* [[ARRAYIDX_I879]], align 2
; CHECK-T1-NEXT: [[TMP14:%.*]] = load i16, i16* [[ADD_PTR_I906]], align 2 ; CHECK-T1-NEXT: [[TMP11:%.*]] = load i16, i16* [[ADD_PTR_I906]], align 2
; CHECK-T1-NEXT: [[ADD_PTR_I884]] = getelementptr inbounds i16, i16* [[PY_31056]], i32 -4 ; CHECK-T1-NEXT: [[ADD_PTR_I884]] = getelementptr inbounds i16, i16* [[PY_31056]], i32 -4
; CHECK-T1-NEXT: [[SHR_I870:%.*]] = sext i16 [[TMP12]] to i32 ; CHECK-T1-NEXT: [[SHR_I870:%.*]] = sext i16 [[TMP9]] to i32
; CHECK-T1-NEXT: [[SHR1_I871:%.*]] = sext i16 [[TMP13]] to i32 ; CHECK-T1-NEXT: [[SHR1_I871:%.*]] = sext i16 [[TMP10]] to i32
; CHECK-T1-NEXT: [[MUL_I872:%.*]] = mul nsw i32 [[SHR1_I871]], [[SHR_I870]] ; CHECK-T1-NEXT: [[MUL_I872:%.*]] = mul nsw i32 [[SHR1_I871]], [[SHR_I870]]
; CHECK-T1-NEXT: [[SHR2_I873:%.*]] = sext i16 [[TMP11]] to i32 ; CHECK-T1-NEXT: [[SHR2_I873:%.*]] = sext i16 [[TMP8]] to i32
; CHECK-T1-NEXT: [[SHR4_I875:%.*]] = sext i16 [[TMP14]] to i32 ; CHECK-T1-NEXT: [[SHR4_I875:%.*]] = sext i16 [[TMP11]] to i32
; CHECK-T1-NEXT: [[MUL5_I876:%.*]] = mul nsw i32 [[SHR4_I875]], [[SHR2_I873]] ; CHECK-T1-NEXT: [[MUL5_I876:%.*]] = mul nsw i32 [[SHR4_I875]], [[SHR2_I873]]
; CHECK-T1-NEXT: [[ADD_I877:%.*]] = add i32 [[ADD6_I900]], [[MUL_I872]] ; CHECK-T1-NEXT: [[ADD_I877:%.*]] = add i32 [[ADD6_I900]], [[MUL_I872]]
; CHECK-T1-NEXT: [[ADD6_I878]] = add i32 [[ADD_I877]], [[MUL5_I876]] ; CHECK-T1-NEXT: [[ADD6_I878]] = add i32 [[ADD_I877]], [[MUL5_I876]]
; CHECK-T1-NEXT: [[DEC30]] = add nsw i32 [[K_11058]], -1 ; CHECK-T1-NEXT: [[DEC30]] = add nsw i32 [[K_11058]], -1
; CHECK-T1-NEXT: [[CMP21:%.*]] = icmp eq i32 [[DEC30]], 0 ; CHECK-T1-NEXT: [[CMP21:%.*]] = icmp eq i32 [[DEC30]], 0
; CHECK-T1-NEXT: br i1 [[CMP21]], label [[WHILE_END31_LOOPEXIT:%.*]], label [[WHILE_BODY23]] ; CHECK-T1-NEXT: br i1 [[CMP21]], label [[WHILE_END31_LOOPEXIT:%.*]], label [[WHILE_BODY23]]
; CHECK-T1: while.end31.loopexit: ; CHECK-T1: while.end31.loopexit:
; CHECK-T1-NEXT: [[ADD_PTR_I890_LCSSA:%.*]] = phi i16* [ [[ADD_PTR_I890]], [[WHILE_BODY23]] ] ; CHECK-T1-NEXT: [[ADD_PTR_I890_LCSSA:%.*]] = phi i16* [ [[ADD_PTR_I890]], [[WHILE_BODY23]] ]
Show All 11 Lines
; CHECK-T1-NEXT: [[ADD_PTR32:%.*]] = getelementptr inbounds i16, i16* [[PY_3_LCSSA]], i32 1 ; CHECK-T1-NEXT: [[ADD_PTR32:%.*]] = getelementptr inbounds i16, i16* [[PY_3_LCSSA]], i32 1
; CHECK-T1-NEXT: br label [[WHILE_BODY36:%.*]] ; CHECK-T1-NEXT: br label [[WHILE_BODY36:%.*]]
; CHECK-T1: while.body36: ; CHECK-T1: while.body36:
; CHECK-T1-NEXT: [[K_21066:%.*]] = phi i32 [ [[DEC42:%.*]], [[WHILE_BODY36]] ], [ [[REM]], [[WHILE_BODY36_PREHEADER]] ] ; CHECK-T1-NEXT: [[K_21066:%.*]] = phi i32 [ [[DEC42:%.*]], [[WHILE_BODY36]] ], [ [[REM]], [[WHILE_BODY36_PREHEADER]] ]
; CHECK-T1-NEXT: [[SUM_21065:%.*]] = phi i32 [ [[ADD6_I868:%.*]], [[WHILE_BODY36]] ], [ [[SUM_1_LCSSA]], [[WHILE_BODY36_PREHEADER]] ] ; CHECK-T1-NEXT: [[SUM_21065:%.*]] = phi i32 [ [[ADD6_I868:%.*]], [[WHILE_BODY36]] ], [ [[SUM_1_LCSSA]], [[WHILE_BODY36_PREHEADER]] ]
; CHECK-T1-NEXT: [[PY_41064:%.*]] = phi i16* [ [[INCDEC_PTR39:%.*]], [[WHILE_BODY36]] ], [ [[ADD_PTR32]], [[WHILE_BODY36_PREHEADER]] ] ; CHECK-T1-NEXT: [[PY_41064:%.*]] = phi i16* [ [[INCDEC_PTR39:%.*]], [[WHILE_BODY36]] ], [ [[ADD_PTR32]], [[WHILE_BODY36_PREHEADER]] ]
; CHECK-T1-NEXT: [[PX_41063:%.*]] = phi i16* [ [[INCDEC_PTR37:%.*]], [[WHILE_BODY36]] ], [ [[PX_3_LCSSA]], [[WHILE_BODY36_PREHEADER]] ] ; CHECK-T1-NEXT: [[PX_41063:%.*]] = phi i16* [ [[INCDEC_PTR37:%.*]], [[WHILE_BODY36]] ], [ [[PX_3_LCSSA]], [[WHILE_BODY36_PREHEADER]] ]
; CHECK-T1-NEXT: [[INCDEC_PTR37]] = getelementptr inbounds i16, i16* [[PX_41063]], i32 1 ; CHECK-T1-NEXT: [[INCDEC_PTR37]] = getelementptr inbounds i16, i16* [[PX_41063]], i32 1
; CHECK-T1-NEXT: [[TMP15:%.*]] = load i16, i16* [[PX_41063]], align 2 ; CHECK-T1-NEXT: [[TMP12:%.*]] = load i16, i16* [[PX_41063]], align 2
; CHECK-T1-NEXT: [[CONV38:%.*]] = sext i16 [[TMP15]] to i32 ; CHECK-T1-NEXT: [[CONV38:%.*]] = sext i16 [[TMP12]] to i32
; CHECK-T1-NEXT: [[INCDEC_PTR39]] = getelementptr inbounds i16, i16* [[PY_41064]], i32 -1 ; CHECK-T1-NEXT: [[INCDEC_PTR39]] = getelementptr inbounds i16, i16* [[PY_41064]], i32 -1
; CHECK-T1-NEXT: [[TMP16:%.*]] = load i16, i16* [[PY_41064]], align 2 ; CHECK-T1-NEXT: [[TMP13:%.*]] = load i16, i16* [[PY_41064]], align 2
; CHECK-T1-NEXT: [[CONV40:%.*]] = sext i16 [[TMP16]] to i32 ; CHECK-T1-NEXT: [[CONV40:%.*]] = sext i16 [[TMP13]] to i32
; CHECK-T1-NEXT: [[MUL_I863:%.*]] = mul nsw i32 [[CONV40]], [[CONV38]] ; CHECK-T1-NEXT: [[MUL_I863:%.*]] = mul nsw i32 [[CONV40]], [[CONV38]]
; CHECK-T1-NEXT: [[SHR3_I864:%.*]] = ashr i32 [[CONV38]], 16 ; CHECK-T1-NEXT: [[SHR3_I864:%.*]] = ashr i32 [[CONV38]], 16
; CHECK-T1-NEXT: [[SHR4_I865:%.*]] = ashr i32 [[CONV40]], 16 ; CHECK-T1-NEXT: [[SHR4_I865:%.*]] = ashr i32 [[CONV40]], 16
; CHECK-T1-NEXT: [[MUL5_I866:%.*]] = mul nsw i32 [[SHR4_I865]], [[SHR3_I864]] ; CHECK-T1-NEXT: [[MUL5_I866:%.*]] = mul nsw i32 [[SHR4_I865]], [[SHR3_I864]]
; CHECK-T1-NEXT: [[ADD_I867:%.*]] = add i32 [[MUL_I863]], [[SUM_21065]] ; CHECK-T1-NEXT: [[ADD_I867:%.*]] = add i32 [[MUL_I863]], [[SUM_21065]]
; CHECK-T1-NEXT: [[ADD6_I868]] = add i32 [[ADD_I867]], [[MUL5_I866]] ; CHECK-T1-NEXT: [[ADD6_I868]] = add i32 [[ADD_I867]], [[MUL5_I866]]
; CHECK-T1-NEXT: [[DEC42]] = add nsw i32 [[K_21066]], -1 ; CHECK-T1-NEXT: [[DEC42]] = add nsw i32 [[K_21066]], -1
; CHECK-T1-NEXT: [[CMP34:%.*]] = icmp eq i32 [[DEC42]], 0 ; CHECK-T1-NEXT: [[CMP34:%.*]] = icmp eq i32 [[DEC42]], 0
; CHECK-T1-NEXT: br i1 [[CMP34]], label [[WHILE_END43_LOOPEXIT:%.*]], label [[WHILE_BODY36]] ; CHECK-T1-NEXT: br i1 [[CMP34]], label [[WHILE_END43_LOOPEXIT:%.*]], label [[WHILE_BODY36]]
; CHECK-T1: while.end43.loopexit: ; CHECK-T1: while.end43.loopexit:
; CHECK-T1-NEXT: [[ADD6_I868_LCSSA:%.*]] = phi i32 [ [[ADD6_I868]], [[WHILE_BODY36]] ] ; CHECK-T1-NEXT: [[ADD6_I868_LCSSA:%.*]] = phi i32 [ [[ADD6_I868]], [[WHILE_BODY36]] ]
; CHECK-T1-NEXT: br label [[WHILE_END43]] ; CHECK-T1-NEXT: br label [[WHILE_END43]]
; CHECK-T1: while.end43: ; CHECK-T1: while.end43:
; CHECK-T1-NEXT: [[SUM_2_LCSSA:%.*]] = phi i32 [ [[SUM_1_LCSSA]], [[WHILE_END31]] ], [ [[ADD6_I868_LCSSA]], [[WHILE_END43_LOOPEXIT]] ] ; CHECK-T1-NEXT: [[SUM_2_LCSSA:%.*]] = phi i32 [ [[SUM_1_LCSSA]], [[WHILE_END31]] ], [ [[ADD6_I868_LCSSA]], [[WHILE_END43_LOOPEXIT]] ]
; CHECK-T1-NEXT: [[TMP17:%.*]] = lshr i32 [[SUM_2_LCSSA]], 15 ; CHECK-T1-NEXT: [[TMP14:%.*]] = lshr i32 [[SUM_2_LCSSA]], 15
; CHECK-T1-NEXT: [[CONV45:%.*]] = trunc i32 [[TMP17]] to i16 ; CHECK-T1-NEXT: [[CONV45:%.*]] = trunc i32 [[TMP14]] to i16
; CHECK-T1-NEXT: [[INCDEC_PTR46]] = getelementptr inbounds i16, i16* [[POUT_11069]], i32 1 ; CHECK-T1-NEXT: [[INCDEC_PTR46]] = getelementptr inbounds i16, i16* [[POUT_11069]], i32 1
; CHECK-T1-NEXT: store i16 [[CONV45]], i16* [[POUT_11069]], align 2 ; CHECK-T1-NEXT: store i16 [[CONV45]], i16* [[POUT_11069]], align 2
; CHECK-T1-NEXT: [[SUB47:%.*]] = add i32 [[COUNT_11072]], -1 ; CHECK-T1-NEXT: [[SUB47:%.*]] = add i32 [[COUNT_11072]], -1
; CHECK-T1-NEXT: [[ADD_PTR48]] = getelementptr inbounds i16, i16* [[PSRCA_PSRCB]], i32 [[SUB47]] ; CHECK-T1-NEXT: [[ADD_PTR48]] = getelementptr inbounds i16, i16* [[PSRCA_PSRCB]], i32 [[SUB47]]
; CHECK-T1-NEXT: [[INC49]] = add i32 [[COUNT_11072]], 1 ; CHECK-T1-NEXT: [[INC49]] = add i32 [[COUNT_11072]], 1
; CHECK-T1-NEXT: [[DEC50]] = add i32 [[BLOCKSIZE1_11071]], -1 ; CHECK-T1-NEXT: [[DEC50]] = add i32 [[BLOCKSIZE1_11071]], -1
; CHECK-T1-NEXT: [[CMP16:%.*]] = icmp eq i32 [[DEC50]], 0 ; CHECK-T1-NEXT: [[CMP16:%.*]] = icmp eq i32 [[DEC50]], 0
; CHECK-T1-NEXT: br i1 [[CMP16]], label [[EXIT_LOOPEXIT:%.*]], label [[WHILE_BODY18]] ; CHECK-T1-NEXT: br i1 [[CMP16]], label [[EXIT_LOOPEXIT:%.*]], label [[WHILE_BODY18]]
; CHECK-T1: exit.loopexit: ; CHECK-T1: exit.loopexit:
; CHECK-T1-NEXT: br label [[EXIT]] ; CHECK-T1-NEXT: br label [[EXIT]]
; CHECK-T1: exit: ; CHECK-T1: exit:
; CHECK-T1-NEXT: ret void ; CHECK-T1-NEXT: ret void
; ;
; CHECK-T2-LABEL: @arm_conv_fast_q15( ; CHECK-T2-LABEL: @arm_conv_fast_q15(
; CHECK-T2-NEXT: entry: ; CHECK-T2-NEXT: entry:
; CHECK-T2-NEXT: [[CMP:%.*]] = icmp ult i32 [[SRCALEN:%.*]], [[SRCBLEN:%.*]] ; CHECK-T2-NEXT: [[CMP:%.*]] = icmp ult i32 [[SRCALEN:%.*]], [[SRCBLEN:%.*]]
; CHECK-T2-NEXT: [[SRCALEN_SRCBLEN:%.*]] = select i1 [[CMP]], i32 [[SRCALEN]], i32 [[SRCBLEN]] ; CHECK-T2-NEXT: [[SRCALEN_SRCBLEN:%.*]] = select i1 [[CMP]], i32 [[SRCALEN]], i32 [[SRCBLEN]]
; CHECK-T2-NEXT: [[PSRCB_PSRCA:%.*]] = select i1 [[CMP]], i16* [[PSRCB:%.*]], i16* [[PSRCA:%.*]] ; CHECK-T2-NEXT: [[PSRCB_PSRCA:%.*]] = select i1 [[CMP]], i16* [[PSRCB:%.*]], i16* [[PSRCA:%.*]]
; CHECK-T2-NEXT: [[PSRCA_PSRCB:%.*]] = select i1 [[CMP]], i16* [[PSRCA]], i16* [[PSRCB]] ; CHECK-T2-NEXT: [[PSRCA_PSRCB:%.*]] = select i1 [[CMP]], i16* [[PSRCA]], i16* [[PSRCB]]
; CHECK-T2-NEXT: [[SUB:%.*]] = add i32 [[SRCALEN_SRCBLEN]], -1 ; CHECK-T2-NEXT: [[SUB:%.*]] = add i32 [[SRCALEN_SRCBLEN]], -1
; CHECK-T2-NEXT: [[CMP41080:%.*]] = icmp eq i32 [[SUB]], 0 ; CHECK-T2-NEXT: [[CMP41080:%.*]] = icmp eq i32 [[SUB]], 0
; CHECK-T2-NEXT: br i1 [[CMP41080]], label [[WHILE_END13:%.*]], label [[WHILE_COND5_PREHEADER_PREHEADER:%.*]] ; CHECK-T2-NEXT: br i1 [[CMP41080]], label [[WHILE_END13:%.*]], label [[WHILE_COND5_PREHEADER_PREHEADER:%.*]]
; CHECK-T2: while.cond5.preheader.preheader: ; CHECK-T2: while.cond5.preheader.preheader:
; CHECK-T2-NEXT: [[TMP0:%.*]] = add i32 [[SRCALEN_SRCBLEN]], -2
; CHECK-T2-NEXT: [[TMP1:%.*]] = icmp ult i32 [[TMP0]], 2
; CHECK-T2-NEXT: [[UMIN:%.*]] = select i1 [[TMP1]], i32 [[TMP0]], i32 2
; CHECK-T2-NEXT: br label [[WHILE_COND5_PREHEADER:%.*]] ; CHECK-T2-NEXT: br label [[WHILE_COND5_PREHEADER:%.*]]
; CHECK-T2: while.cond5.preheader: ; CHECK-T2: while.cond5.preheader:
; CHECK-T2-NEXT: [[COUNT_01084:%.*]] = phi i32 [ [[INC:%.*]], [[WHILE_END:%.*]] ], [ 1, [[WHILE_COND5_PREHEADER_PREHEADER]] ] ; CHECK-T2-NEXT: [[COUNT_01084:%.*]] = phi i32 [ [[INC:%.*]], [[WHILE_END:%.*]] ], [ 1, [[WHILE_COND5_PREHEADER_PREHEADER]] ]
; CHECK-T2-NEXT: [[BLOCKSIZE1_01083:%.*]] = phi i32 [ [[DEC12:%.*]], [[WHILE_END]] ], [ [[SUB]], [[WHILE_COND5_PREHEADER_PREHEADER]] ] ; CHECK-T2-NEXT: [[BLOCKSIZE1_01083:%.*]] = phi i32 [ [[DEC12:%.*]], [[WHILE_END]] ], [ [[SUB]], [[WHILE_COND5_PREHEADER_PREHEADER]] ]
; CHECK-T2-NEXT: [[PY_01082:%.*]] = phi i16* [ [[ADD_PTR:%.*]], [[WHILE_END]] ], [ [[PSRCA_PSRCB]], [[WHILE_COND5_PREHEADER_PREHEADER]] ] ; CHECK-T2-NEXT: [[PY_01082:%.*]] = phi i16* [ [[ADD_PTR:%.*]], [[WHILE_END]] ], [ [[PSRCA_PSRCB]], [[WHILE_COND5_PREHEADER_PREHEADER]] ]
; CHECK-T2-NEXT: [[POUT_01081:%.*]] = phi i16* [ [[INCDEC_PTR11:%.*]], [[WHILE_END]] ], [ [[PDST:%.*]], [[WHILE_COND5_PREHEADER_PREHEADER]] ] ; CHECK-T2-NEXT: [[POUT_01081:%.*]] = phi i16* [ [[INCDEC_PTR11:%.*]], [[WHILE_END]] ], [ [[PDST:%.*]], [[WHILE_COND5_PREHEADER_PREHEADER]] ]
; CHECK-T2-NEXT: br label [[WHILE_BODY7:%.*]] ; CHECK-T2-NEXT: br label [[WHILE_BODY7:%.*]]
; CHECK-T2: while.body7: ; CHECK-T2: while.body7:
; CHECK-T2-NEXT: [[K_01078:%.*]] = phi i32 [ [[DEC:%.*]], [[WHILE_BODY7]] ], [ [[COUNT_01084]], [[WHILE_COND5_PREHEADER]] ] ; CHECK-T2-NEXT: [[K_01078:%.*]] = phi i32 [ [[DEC:%.*]], [[WHILE_BODY7]] ], [ [[COUNT_01084]], [[WHILE_COND5_PREHEADER]] ]
; CHECK-T2-NEXT: [[SUM_01077:%.*]] = phi i32 [ [[ADD6_I:%.*]], [[WHILE_BODY7]] ], [ 0, [[WHILE_COND5_PREHEADER]] ] ; CHECK-T2-NEXT: [[SUM_01077:%.*]] = phi i32 [ [[ADD6_I:%.*]], [[WHILE_BODY7]] ], [ 0, [[WHILE_COND5_PREHEADER]] ]
; CHECK-T2-NEXT: [[PY_11076:%.*]] = phi i16* [ [[INCDEC_PTR8:%.*]], [[WHILE_BODY7]] ], [ [[PY_01082]], [[WHILE_COND5_PREHEADER]] ] ; CHECK-T2-NEXT: [[PY_11076:%.*]] = phi i16* [ [[INCDEC_PTR8:%.*]], [[WHILE_BODY7]] ], [ [[PY_01082]], [[WHILE_COND5_PREHEADER]] ]
; CHECK-T2-NEXT: [[PX_11075:%.*]] = phi i16* [ [[INCDEC_PTR:%.*]], [[WHILE_BODY7]] ], [ [[PSRCB_PSRCA]], [[WHILE_COND5_PREHEADER]] ] ; CHECK-T2-NEXT: [[PX_11075:%.*]] = phi i16* [ [[INCDEC_PTR:%.*]], [[WHILE_BODY7]] ], [ [[PSRCB_PSRCA]], [[WHILE_COND5_PREHEADER]] ]
; CHECK-T2-NEXT: [[INCDEC_PTR]] = getelementptr inbounds i16, i16* [[PX_11075]], i32 1 ; CHECK-T2-NEXT: [[INCDEC_PTR]] = getelementptr inbounds i16, i16* [[PX_11075]], i32 1
; CHECK-T2-NEXT: [[TMP2:%.*]] = load i16, i16* [[PX_11075]], align 2 ; CHECK-T2-NEXT: [[TMP0:%.*]] = load i16, i16* [[PX_11075]], align 2
; CHECK-T2-NEXT: [[CONV:%.*]] = sext i16 [[TMP2]] to i32 ; CHECK-T2-NEXT: [[CONV:%.*]] = sext i16 [[TMP0]] to i32
; CHECK-T2-NEXT: [[INCDEC_PTR8]] = getelementptr inbounds i16, i16* [[PY_11076]], i32 -1 ; CHECK-T2-NEXT: [[INCDEC_PTR8]] = getelementptr inbounds i16, i16* [[PY_11076]], i32 -1
; CHECK-T2-NEXT: [[TMP3:%.*]] = load i16, i16* [[PY_11076]], align 2 ; CHECK-T2-NEXT: [[TMP1:%.*]] = load i16, i16* [[PY_11076]], align 2
; CHECK-T2-NEXT: [[CONV9:%.*]] = sext i16 [[TMP3]] to i32 ; CHECK-T2-NEXT: [[CONV9:%.*]] = sext i16 [[TMP1]] to i32
; CHECK-T2-NEXT: [[MUL_I:%.*]] = mul nsw i32 [[CONV9]], [[CONV]] ; CHECK-T2-NEXT: [[MUL_I:%.*]] = mul nsw i32 [[CONV9]], [[CONV]]
; CHECK-T2-NEXT: [[SHR3_I:%.*]] = ashr i32 [[CONV]], 16 ; CHECK-T2-NEXT: [[SHR3_I:%.*]] = ashr i32 [[CONV]], 16
; CHECK-T2-NEXT: [[SHR4_I:%.*]] = ashr i32 [[CONV9]], 16 ; CHECK-T2-NEXT: [[SHR4_I:%.*]] = ashr i32 [[CONV9]], 16
; CHECK-T2-NEXT: [[MUL5_I:%.*]] = mul nsw i32 [[SHR4_I]], [[SHR3_I]] ; CHECK-T2-NEXT: [[MUL5_I:%.*]] = mul nsw i32 [[SHR4_I]], [[SHR3_I]]
; CHECK-T2-NEXT: [[ADD_I:%.*]] = add i32 [[MUL_I]], [[SUM_01077]] ; CHECK-T2-NEXT: [[ADD_I:%.*]] = add i32 [[MUL_I]], [[SUM_01077]]
; CHECK-T2-NEXT: [[ADD6_I]] = add i32 [[ADD_I]], [[MUL5_I]] ; CHECK-T2-NEXT: [[ADD6_I]] = add i32 [[ADD_I]], [[MUL5_I]]
; CHECK-T2-NEXT: [[DEC]] = add nsw i32 [[K_01078]], -1 ; CHECK-T2-NEXT: [[DEC]] = add nsw i32 [[K_01078]], -1
; CHECK-T2-NEXT: [[CMP6:%.*]] = icmp eq i32 [[DEC]], 0 ; CHECK-T2-NEXT: [[CMP6:%.*]] = icmp eq i32 [[DEC]], 0
; CHECK-T2-NEXT: br i1 [[CMP6]], label [[WHILE_END]], label [[WHILE_BODY7]] ; CHECK-T2-NEXT: br i1 [[CMP6]], label [[WHILE_END]], label [[WHILE_BODY7]]
; CHECK-T2: while.end: ; CHECK-T2: while.end:
; CHECK-T2-NEXT: [[ADD6_I_LCSSA:%.*]] = phi i32 [ [[ADD6_I]], [[WHILE_BODY7]] ] ; CHECK-T2-NEXT: [[ADD6_I_LCSSA:%.*]] = phi i32 [ [[ADD6_I]], [[WHILE_BODY7]] ]
; CHECK-T2-NEXT: [[TMP4:%.*]] = lshr i32 [[ADD6_I_LCSSA]], 15 ; CHECK-T2-NEXT: [[TMP2:%.*]] = lshr i32 [[ADD6_I_LCSSA]], 15
; CHECK-T2-NEXT: [[CONV10:%.*]] = trunc i32 [[TMP4]] to i16 ; CHECK-T2-NEXT: [[CONV10:%.*]] = trunc i32 [[TMP2]] to i16
; CHECK-T2-NEXT: [[INCDEC_PTR11]] = getelementptr inbounds i16, i16* [[POUT_01081]], i32 1 ; CHECK-T2-NEXT: [[INCDEC_PTR11]] = getelementptr inbounds i16, i16* [[POUT_01081]], i32 1
; CHECK-T2-NEXT: store i16 [[CONV10]], i16* [[POUT_01081]], align 2 ; CHECK-T2-NEXT: store i16 [[CONV10]], i16* [[POUT_01081]], align 2
; CHECK-T2-NEXT: [[ADD_PTR]] = getelementptr inbounds i16, i16* [[PSRCA_PSRCB]], i32 [[COUNT_01084]] ; CHECK-T2-NEXT: [[ADD_PTR]] = getelementptr inbounds i16, i16* [[PSRCA_PSRCB]], i32 [[COUNT_01084]]
; CHECK-T2-NEXT: [[INC]] = add nuw nsw i32 [[COUNT_01084]], 1 ; CHECK-T2-NEXT: [[INC]] = add nuw nsw i32 [[COUNT_01084]], 1
; CHECK-T2-NEXT: [[DEC12]] = add i32 [[BLOCKSIZE1_01083]], -1 ; CHECK-T2-NEXT: [[DEC12]] = add i32 [[BLOCKSIZE1_01083]], -1
; CHECK-T2-NEXT: [[CMP3:%.*]] = icmp ult i32 [[COUNT_01084]], 3 ; CHECK-T2-NEXT: [[CMP3:%.*]] = icmp ult i32 [[COUNT_01084]], 3
; CHECK-T2-NEXT: [[CMP4:%.*]] = icmp ne i32 [[DEC12]], 0 ; CHECK-T2-NEXT: [[CMP4:%.*]] = icmp ne i32 [[DEC12]], 0
; CHECK-T2-NEXT: [[TMP5:%.*]] = and i1 [[CMP4]], [[CMP3]] ; CHECK-T2-NEXT: [[TMP3:%.*]] = and i1 [[CMP4]], [[CMP3]]
; CHECK-T2-NEXT: br i1 [[TMP5]], label [[WHILE_COND5_PREHEADER]], label [[WHILE_END13_LOOPEXIT:%.*]] ; CHECK-T2-NEXT: br i1 [[TMP3]], label [[WHILE_COND5_PREHEADER]], label [[WHILE_END13_LOOPEXIT:%.*]]
; CHECK-T2: while.end13.loopexit: ; CHECK-T2: while.end13.loopexit:
; CHECK-T2-NEXT: [[INCDEC_PTR11_LCSSA:%.*]] = phi i16* [ [[INCDEC_PTR11]], [[WHILE_END]] ] ; CHECK-T2-NEXT: [[INCDEC_PTR11_LCSSA:%.*]] = phi i16* [ [[INCDEC_PTR11]], [[WHILE_END]] ]
; CHECK-T2-NEXT: [[ADD_PTR_LCSSA:%.*]] = phi i16* [ [[ADD_PTR]], [[WHILE_END]] ] ; CHECK-T2-NEXT: [[ADD_PTR_LCSSA:%.*]] = phi i16* [ [[ADD_PTR]], [[WHILE_END]] ]
; CHECK-T2-NEXT: [[INC_LCSSA:%.*]] = phi i32 [ [[INC]], [[WHILE_END]] ]
; CHECK-T2-NEXT: [[DEC12_LCSSA:%.*]] = phi i32 [ [[DEC12]], [[WHILE_END]] ] ; CHECK-T2-NEXT: [[DEC12_LCSSA:%.*]] = phi i32 [ [[DEC12]], [[WHILE_END]] ]
; CHECK-T2-NEXT: [[TMP6:%.*]] = add nuw nsw i32 [[UMIN]], 2
; CHECK-T2-NEXT: br label [[WHILE_END13]] ; CHECK-T2-NEXT: br label [[WHILE_END13]]
; CHECK-T2: while.end13: ; CHECK-T2: while.end13:
; CHECK-T2-NEXT: [[POUT_0_LCSSA:%.*]] = phi i16* [ [[PDST]], [[ENTRY:%.*]] ], [ [[INCDEC_PTR11_LCSSA]], [[WHILE_END13_LOOPEXIT]] ] ; CHECK-T2-NEXT: [[POUT_0_LCSSA:%.*]] = phi i16* [ [[PDST]], [[ENTRY:%.*]] ], [ [[INCDEC_PTR11_LCSSA]], [[WHILE_END13_LOOPEXIT]] ]
; CHECK-T2-NEXT: [[PY_0_LCSSA:%.*]] = phi i16* [ [[PSRCA_PSRCB]], [[ENTRY]] ], [ [[ADD_PTR_LCSSA]], [[WHILE_END13_LOOPEXIT]] ] ; CHECK-T2-NEXT: [[PY_0_LCSSA:%.*]] = phi i16* [ [[PSRCA_PSRCB]], [[ENTRY]] ], [ [[ADD_PTR_LCSSA]], [[WHILE_END13_LOOPEXIT]] ]
; CHECK-T2-NEXT: [[BLOCKSIZE1_0_LCSSA:%.*]] = phi i32 [ [[SUB]], [[ENTRY]] ], [ [[DEC12_LCSSA]], [[WHILE_END13_LOOPEXIT]] ] ; CHECK-T2-NEXT: [[BLOCKSIZE1_0_LCSSA:%.*]] = phi i32 [ [[SUB]], [[ENTRY]] ], [ [[DEC12_LCSSA]], [[WHILE_END13_LOOPEXIT]] ]
; CHECK-T2-NEXT: [[COUNT_0_LCSSA:%.*]] = phi i32 [ 1, [[ENTRY]] ], [ [[TMP6]], [[WHILE_END13_LOOPEXIT]] ] ; CHECK-T2-NEXT: [[COUNT_0_LCSSA:%.*]] = phi i32 [ 1, [[ENTRY]] ], [ [[INC_LCSSA]], [[WHILE_END13_LOOPEXIT]] ]
; CHECK-T2-NEXT: [[CMP161068:%.*]] = icmp eq i32 [[BLOCKSIZE1_0_LCSSA]], 0 ; CHECK-T2-NEXT: [[CMP161068:%.*]] = icmp eq i32 [[BLOCKSIZE1_0_LCSSA]], 0
; CHECK-T2-NEXT: br i1 [[CMP161068]], label [[EXIT:%.*]], label [[WHILE_BODY18_PREHEADER:%.*]] ; CHECK-T2-NEXT: br i1 [[CMP161068]], label [[EXIT:%.*]], label [[WHILE_BODY18_PREHEADER:%.*]]
; CHECK-T2: while.body18.preheader: ; CHECK-T2: while.body18.preheader:
; CHECK-T2-NEXT: [[ADD_PTR14:%.*]] = getelementptr inbounds i16, i16* [[PY_0_LCSSA]], i32 -1 ; CHECK-T2-NEXT: [[ADD_PTR14:%.*]] = getelementptr inbounds i16, i16* [[PY_0_LCSSA]], i32 -1
; CHECK-T2-NEXT: br label [[WHILE_BODY18:%.*]] ; CHECK-T2-NEXT: br label [[WHILE_BODY18:%.*]]
; CHECK-T2: while.body18: ; CHECK-T2: while.body18:
; CHECK-T2-NEXT: [[COUNT_11072:%.*]] = phi i32 [ [[INC49:%.*]], [[WHILE_END43:%.*]] ], [ [[COUNT_0_LCSSA]], [[WHILE_BODY18_PREHEADER]] ] ; CHECK-T2-NEXT: [[COUNT_11072:%.*]] = phi i32 [ [[INC49:%.*]], [[WHILE_END43:%.*]] ], [ [[COUNT_0_LCSSA]], [[WHILE_BODY18_PREHEADER]] ]
; CHECK-T2-NEXT: [[BLOCKSIZE1_11071:%.*]] = phi i32 [ [[DEC50:%.*]], [[WHILE_END43]] ], [ [[BLOCKSIZE1_0_LCSSA]], [[WHILE_BODY18_PREHEADER]] ] ; CHECK-T2-NEXT: [[BLOCKSIZE1_11071:%.*]] = phi i32 [ [[DEC50:%.*]], [[WHILE_END43]] ], [ [[BLOCKSIZE1_0_LCSSA]], [[WHILE_BODY18_PREHEADER]] ]
; CHECK-T2-NEXT: [[PY_21070:%.*]] = phi i16* [ [[ADD_PTR48:%.*]], [[WHILE_END43]] ], [ [[ADD_PTR14]], [[WHILE_BODY18_PREHEADER]] ] ; CHECK-T2-NEXT: [[PY_21070:%.*]] = phi i16* [ [[ADD_PTR48:%.*]], [[WHILE_END43]] ], [ [[ADD_PTR14]], [[WHILE_BODY18_PREHEADER]] ]
; CHECK-T2-NEXT: [[POUT_11069:%.*]] = phi i16* [ [[INCDEC_PTR46:%.*]], [[WHILE_END43]] ], [ [[POUT_0_LCSSA]], [[WHILE_BODY18_PREHEADER]] ] ; CHECK-T2-NEXT: [[POUT_11069:%.*]] = phi i16* [ [[INCDEC_PTR46:%.*]], [[WHILE_END43]] ], [ [[POUT_0_LCSSA]], [[WHILE_BODY18_PREHEADER]] ]
; CHECK-T2-NEXT: [[SHR19:%.*]] = lshr i32 [[COUNT_11072]], 2 ; CHECK-T2-NEXT: [[SHR19:%.*]] = lshr i32 [[COUNT_11072]], 2
; CHECK-T2-NEXT: [[CMP211054:%.*]] = icmp eq i32 [[SHR19]], 0 ; CHECK-T2-NEXT: [[CMP211054:%.*]] = icmp eq i32 [[SHR19]], 0
; CHECK-T2-NEXT: br i1 [[CMP211054]], label [[WHILE_END31:%.*]], label [[WHILE_BODY23_PREHEADER:%.*]] ; CHECK-T2-NEXT: br i1 [[CMP211054]], label [[WHILE_END31:%.*]], label [[WHILE_BODY23_PREHEADER:%.*]]
; CHECK-T2: while.body23.preheader: ; CHECK-T2: while.body23.preheader:
; CHECK-T2-NEXT: br label [[WHILE_BODY23:%.*]] ; CHECK-T2-NEXT: br label [[WHILE_BODY23:%.*]]
; CHECK-T2: while.body23: ; CHECK-T2: while.body23:
; CHECK-T2-NEXT: [[K_11058:%.*]] = phi i32 [ [[DEC30:%.*]], [[WHILE_BODY23]] ], [ [[SHR19]], [[WHILE_BODY23_PREHEADER]] ] ; CHECK-T2-NEXT: [[K_11058:%.*]] = phi i32 [ [[DEC30:%.*]], [[WHILE_BODY23]] ], [ [[SHR19]], [[WHILE_BODY23_PREHEADER]] ]
; CHECK-T2-NEXT: [[SUM_11057:%.*]] = phi i32 [ [[ADD6_I878:%.*]], [[WHILE_BODY23]] ], [ 0, [[WHILE_BODY23_PREHEADER]] ] ; CHECK-T2-NEXT: [[SUM_11057:%.*]] = phi i32 [ [[ADD6_I878:%.*]], [[WHILE_BODY23]] ], [ 0, [[WHILE_BODY23_PREHEADER]] ]
; CHECK-T2-NEXT: [[PY_31056:%.*]] = phi i16* [ [[ADD_PTR_I884:%.*]], [[WHILE_BODY23]] ], [ [[PY_21070]], [[WHILE_BODY23_PREHEADER]] ] ; CHECK-T2-NEXT: [[PY_31056:%.*]] = phi i16* [ [[ADD_PTR_I884:%.*]], [[WHILE_BODY23]] ], [ [[PY_21070]], [[WHILE_BODY23_PREHEADER]] ]
; CHECK-T2-NEXT: [[PX_31055:%.*]] = phi i16* [ [[ADD_PTR_I890:%.*]], [[WHILE_BODY23]] ], [ [[PSRCB_PSRCA]], [[WHILE_BODY23_PREHEADER]] ] ; CHECK-T2-NEXT: [[PX_31055:%.*]] = phi i16* [ [[ADD_PTR_I890:%.*]], [[WHILE_BODY23]] ], [ [[PSRCB_PSRCA]], [[WHILE_BODY23_PREHEADER]] ]
; CHECK-T2-NEXT: [[ARRAYIDX_I907:%.*]] = getelementptr inbounds i16, i16* [[PX_31055]], i32 1 ; CHECK-T2-NEXT: [[ARRAYIDX_I907:%.*]] = getelementptr inbounds i16, i16* [[PX_31055]], i32 1
; CHECK-T2-NEXT: [[TMP7:%.*]] = load i16, i16* [[ARRAYIDX_I907]], align 2 ; CHECK-T2-NEXT: [[TMP4:%.*]] = load i16, i16* [[ARRAYIDX_I907]], align 2
; CHECK-T2-NEXT: [[TMP8:%.*]] = load i16, i16* [[PX_31055]], align 2 ; CHECK-T2-NEXT: [[TMP5:%.*]] = load i16, i16* [[PX_31055]], align 2
; CHECK-T2-NEXT: [[ADD_PTR_I912:%.*]] = getelementptr inbounds i16, i16* [[PX_31055]], i32 2 ; CHECK-T2-NEXT: [[ADD_PTR_I912:%.*]] = getelementptr inbounds i16, i16* [[PX_31055]], i32 2
; CHECK-T2-NEXT: [[ARRAYIDX_I901:%.*]] = getelementptr inbounds i16, i16* [[PY_31056]], i32 1 ; CHECK-T2-NEXT: [[ARRAYIDX_I901:%.*]] = getelementptr inbounds i16, i16* [[PY_31056]], i32 1
; CHECK-T2-NEXT: [[TMP9:%.*]] = load i16, i16* [[ARRAYIDX_I901]], align 2 ; CHECK-T2-NEXT: [[TMP6:%.*]] = load i16, i16* [[ARRAYIDX_I901]], align 2
; CHECK-T2-NEXT: [[TMP10:%.*]] = load i16, i16* [[PY_31056]], align 2 ; CHECK-T2-NEXT: [[TMP7:%.*]] = load i16, i16* [[PY_31056]], align 2
; CHECK-T2-NEXT: [[ADD_PTR_I906:%.*]] = getelementptr inbounds i16, i16* [[PY_31056]], i32 -2 ; CHECK-T2-NEXT: [[ADD_PTR_I906:%.*]] = getelementptr inbounds i16, i16* [[PY_31056]], i32 -2
; CHECK-T2-NEXT: [[SHR_I892:%.*]] = sext i16 [[TMP8]] to i32 ; CHECK-T2-NEXT: [[SHR_I892:%.*]] = sext i16 [[TMP5]] to i32
; CHECK-T2-NEXT: [[SHR1_I893:%.*]] = sext i16 [[TMP9]] to i32 ; CHECK-T2-NEXT: [[SHR1_I893:%.*]] = sext i16 [[TMP6]] to i32
; CHECK-T2-NEXT: [[MUL_I894:%.*]] = mul nsw i32 [[SHR1_I893]], [[SHR_I892]] ; CHECK-T2-NEXT: [[MUL_I894:%.*]] = mul nsw i32 [[SHR1_I893]], [[SHR_I892]]
; CHECK-T2-NEXT: [[SHR2_I895:%.*]] = sext i16 [[TMP7]] to i32 ; CHECK-T2-NEXT: [[SHR2_I895:%.*]] = sext i16 [[TMP4]] to i32
; CHECK-T2-NEXT: [[SHR4_I897:%.*]] = sext i16 [[TMP10]] to i32 ; CHECK-T2-NEXT: [[SHR4_I897:%.*]] = sext i16 [[TMP7]] to i32
; CHECK-T2-NEXT: [[MUL5_I898:%.*]] = mul nsw i32 [[SHR4_I897]], [[SHR2_I895]] ; CHECK-T2-NEXT: [[MUL5_I898:%.*]] = mul nsw i32 [[SHR4_I897]], [[SHR2_I895]]
; CHECK-T2-NEXT: [[ADD_I899:%.*]] = add i32 [[MUL_I894]], [[SUM_11057]] ; CHECK-T2-NEXT: [[ADD_I899:%.*]] = add i32 [[MUL_I894]], [[SUM_11057]]
; CHECK-T2-NEXT: [[ADD6_I900:%.*]] = add i32 [[ADD_I899]], [[MUL5_I898]] ; CHECK-T2-NEXT: [[ADD6_I900:%.*]] = add i32 [[ADD_I899]], [[MUL5_I898]]
; CHECK-T2-NEXT: [[ARRAYIDX_I885:%.*]] = getelementptr inbounds i16, i16* [[PX_31055]], i32 3 ; CHECK-T2-NEXT: [[ARRAYIDX_I885:%.*]] = getelementptr inbounds i16, i16* [[PX_31055]], i32 3
; CHECK-T2-NEXT: [[TMP11:%.*]] = load i16, i16* [[ARRAYIDX_I885]], align 2 ; CHECK-T2-NEXT: [[TMP8:%.*]] = load i16, i16* [[ARRAYIDX_I885]], align 2
; CHECK-T2-NEXT: [[TMP12:%.*]] = load i16, i16* [[ADD_PTR_I912]], align 2 ; CHECK-T2-NEXT: [[TMP9:%.*]] = load i16, i16* [[ADD_PTR_I912]], align 2
; CHECK-T2-NEXT: [[ADD_PTR_I890]] = getelementptr inbounds i16, i16* [[PX_31055]], i32 4 ; CHECK-T2-NEXT: [[ADD_PTR_I890]] = getelementptr inbounds i16, i16* [[PX_31055]], i32 4
; CHECK-T2-NEXT: [[ARRAYIDX_I879:%.*]] = getelementptr inbounds i16, i16* [[PY_31056]], i32 -1 ; CHECK-T2-NEXT: [[ARRAYIDX_I879:%.*]] = getelementptr inbounds i16, i16* [[PY_31056]], i32 -1
; CHECK-T2-NEXT: [[TMP13:%.*]] = load i16, i16* [[ARRAYIDX_I879]], align 2 ; CHECK-T2-NEXT: [[TMP10:%.*]] = load i16, i16* [[ARRAYIDX_I879]], align 2
; CHECK-T2-NEXT: [[TMP14:%.*]] = load i16, i16* [[ADD_PTR_I906]], align 2 ; CHECK-T2-NEXT: [[TMP11:%.*]] = load i16, i16* [[ADD_PTR_I906]], align 2
; CHECK-T2-NEXT: [[ADD_PTR_I884]] = getelementptr inbounds i16, i16* [[PY_31056]], i32 -4 ; CHECK-T2-NEXT: [[ADD_PTR_I884]] = getelementptr inbounds i16, i16* [[PY_31056]], i32 -4
; CHECK-T2-NEXT: [[SHR_I870:%.*]] = sext i16 [[TMP12]] to i32 ; CHECK-T2-NEXT: [[SHR_I870:%.*]] = sext i16 [[TMP9]] to i32
; CHECK-T2-NEXT: [[SHR1_I871:%.*]] = sext i16 [[TMP13]] to i32 ; CHECK-T2-NEXT: [[SHR1_I871:%.*]] = sext i16 [[TMP10]] to i32
; CHECK-T2-NEXT: [[MUL_I872:%.*]] = mul nsw i32 [[SHR1_I871]], [[SHR_I870]] ; CHECK-T2-NEXT: [[MUL_I872:%.*]] = mul nsw i32 [[SHR1_I871]], [[SHR_I870]]
; CHECK-T2-NEXT: [[SHR2_I873:%.*]] = sext i16 [[TMP11]] to i32 ; CHECK-T2-NEXT: [[SHR2_I873:%.*]] = sext i16 [[TMP8]] to i32
; CHECK-T2-NEXT: [[SHR4_I875:%.*]] = sext i16 [[TMP14]] to i32 ; CHECK-T2-NEXT: [[SHR4_I875:%.*]] = sext i16 [[TMP11]] to i32
; CHECK-T2-NEXT: [[MUL5_I876:%.*]] = mul nsw i32 [[SHR4_I875]], [[SHR2_I873]] ; CHECK-T2-NEXT: [[MUL5_I876:%.*]] = mul nsw i32 [[SHR4_I875]], [[SHR2_I873]]
; CHECK-T2-NEXT: [[ADD_I877:%.*]] = add i32 [[ADD6_I900]], [[MUL_I872]] ; CHECK-T2-NEXT: [[ADD_I877:%.*]] = add i32 [[ADD6_I900]], [[MUL_I872]]
; CHECK-T2-NEXT: [[ADD6_I878]] = add i32 [[ADD_I877]], [[MUL5_I876]] ; CHECK-T2-NEXT: [[ADD6_I878]] = add i32 [[ADD_I877]], [[MUL5_I876]]
; CHECK-T2-NEXT: [[DEC30]] = add nsw i32 [[K_11058]], -1 ; CHECK-T2-NEXT: [[DEC30]] = add nsw i32 [[K_11058]], -1
; CHECK-T2-NEXT: [[CMP21:%.*]] = icmp eq i32 [[DEC30]], 0 ; CHECK-T2-NEXT: [[CMP21:%.*]] = icmp eq i32 [[DEC30]], 0
; CHECK-T2-NEXT: br i1 [[CMP21]], label [[WHILE_END31_LOOPEXIT:%.*]], label [[WHILE_BODY23]] ; CHECK-T2-NEXT: br i1 [[CMP21]], label [[WHILE_END31_LOOPEXIT:%.*]], label [[WHILE_BODY23]]
; CHECK-T2: while.end31.loopexit: ; CHECK-T2: while.end31.loopexit:
; CHECK-T2-NEXT: [[ADD_PTR_I890_LCSSA:%.*]] = phi i16* [ [[ADD_PTR_I890]], [[WHILE_BODY23]] ] ; CHECK-T2-NEXT: [[ADD_PTR_I890_LCSSA:%.*]] = phi i16* [ [[ADD_PTR_I890]], [[WHILE_BODY23]] ]
Show All 11 Lines
; CHECK-T2-NEXT: [[ADD_PTR32:%.*]] = getelementptr inbounds i16, i16* [[PY_3_LCSSA]], i32 1 ; CHECK-T2-NEXT: [[ADD_PTR32:%.*]] = getelementptr inbounds i16, i16* [[PY_3_LCSSA]], i32 1
; CHECK-T2-NEXT: br label [[WHILE_BODY36:%.*]] ; CHECK-T2-NEXT: br label [[WHILE_BODY36:%.*]]
; CHECK-T2: while.body36: ; CHECK-T2: while.body36:
; CHECK-T2-NEXT: [[K_21066:%.*]] = phi i32 [ [[DEC42:%.*]], [[WHILE_BODY36]] ], [ [[REM]], [[WHILE_BODY36_PREHEADER]] ] ; CHECK-T2-NEXT: [[K_21066:%.*]] = phi i32 [ [[DEC42:%.*]], [[WHILE_BODY36]] ], [ [[REM]], [[WHILE_BODY36_PREHEADER]] ]
; CHECK-T2-NEXT: [[SUM_21065:%.*]] = phi i32 [ [[ADD6_I868:%.*]], [[WHILE_BODY36]] ], [ [[SUM_1_LCSSA]], [[WHILE_BODY36_PREHEADER]] ] ; CHECK-T2-NEXT: [[SUM_21065:%.*]] = phi i32 [ [[ADD6_I868:%.*]], [[WHILE_BODY36]] ], [ [[SUM_1_LCSSA]], [[WHILE_BODY36_PREHEADER]] ]
; CHECK-T2-NEXT: [[PY_41064:%.*]] = phi i16* [ [[INCDEC_PTR39:%.*]], [[WHILE_BODY36]] ], [ [[ADD_PTR32]], [[WHILE_BODY36_PREHEADER]] ] ; CHECK-T2-NEXT: [[PY_41064:%.*]] = phi i16* [ [[INCDEC_PTR39:%.*]], [[WHILE_BODY36]] ], [ [[ADD_PTR32]], [[WHILE_BODY36_PREHEADER]] ]
; CHECK-T2-NEXT: [[PX_41063:%.*]] = phi i16* [ [[INCDEC_PTR37:%.*]], [[WHILE_BODY36]] ], [ [[PX_3_LCSSA]], [[WHILE_BODY36_PREHEADER]] ] ; CHECK-T2-NEXT: [[PX_41063:%.*]] = phi i16* [ [[INCDEC_PTR37:%.*]], [[WHILE_BODY36]] ], [ [[PX_3_LCSSA]], [[WHILE_BODY36_PREHEADER]] ]
; CHECK-T2-NEXT: [[INCDEC_PTR37]] = getelementptr inbounds i16, i16* [[PX_41063]], i32 1 ; CHECK-T2-NEXT: [[INCDEC_PTR37]] = getelementptr inbounds i16, i16* [[PX_41063]], i32 1
; CHECK-T2-NEXT: [[TMP15:%.*]] = load i16, i16* [[PX_41063]], align 2 ; CHECK-T2-NEXT: [[TMP12:%.*]] = load i16, i16* [[PX_41063]], align 2
; CHECK-T2-NEXT: [[CONV38:%.*]] = sext i16 [[TMP15]] to i32 ; CHECK-T2-NEXT: [[CONV38:%.*]] = sext i16 [[TMP12]] to i32
; CHECK-T2-NEXT: [[INCDEC_PTR39]] = getelementptr inbounds i16, i16* [[PY_41064]], i32 -1 ; CHECK-T2-NEXT: [[INCDEC_PTR39]] = getelementptr inbounds i16, i16* [[PY_41064]], i32 -1
; CHECK-T2-NEXT: [[TMP16:%.*]] = load i16, i16* [[PY_41064]], align 2 ; CHECK-T2-NEXT: [[TMP13:%.*]] = load i16, i16* [[PY_41064]], align 2
; CHECK-T2-NEXT: [[CONV40:%.*]] = sext i16 [[TMP16]] to i32 ; CHECK-T2-NEXT: [[CONV40:%.*]] = sext i16 [[TMP13]] to i32
; CHECK-T2-NEXT: [[MUL_I863:%.*]] = mul nsw i32 [[CONV40]], [[CONV38]] ; CHECK-T2-NEXT: [[MUL_I863:%.*]] = mul nsw i32 [[CONV40]], [[CONV38]]
; CHECK-T2-NEXT: [[SHR3_I864:%.*]] = ashr i32 [[CONV38]], 16 ; CHECK-T2-NEXT: [[SHR3_I864:%.*]] = ashr i32 [[CONV38]], 16
; CHECK-T2-NEXT: [[SHR4_I865:%.*]] = ashr i32 [[CONV40]], 16 ; CHECK-T2-NEXT: [[SHR4_I865:%.*]] = ashr i32 [[CONV40]], 16
; CHECK-T2-NEXT: [[MUL5_I866:%.*]] = mul nsw i32 [[SHR4_I865]], [[SHR3_I864]] ; CHECK-T2-NEXT: [[MUL5_I866:%.*]] = mul nsw i32 [[SHR4_I865]], [[SHR3_I864]]
; CHECK-T2-NEXT: [[ADD_I867:%.*]] = add i32 [[MUL_I863]], [[SUM_21065]] ; CHECK-T2-NEXT: [[ADD_I867:%.*]] = add i32 [[MUL_I863]], [[SUM_21065]]
; CHECK-T2-NEXT: [[ADD6_I868]] = add i32 [[ADD_I867]], [[MUL5_I866]] ; CHECK-T2-NEXT: [[ADD6_I868]] = add i32 [[ADD_I867]], [[MUL5_I866]]
; CHECK-T2-NEXT: [[DEC42]] = add nsw i32 [[K_21066]], -1 ; CHECK-T2-NEXT: [[DEC42]] = add nsw i32 [[K_21066]], -1
; CHECK-T2-NEXT: [[CMP34:%.*]] = icmp eq i32 [[DEC42]], 0 ; CHECK-T2-NEXT: [[CMP34:%.*]] = icmp eq i32 [[DEC42]], 0
; CHECK-T2-NEXT: br i1 [[CMP34]], label [[WHILE_END43_LOOPEXIT:%.*]], label [[WHILE_BODY36]] ; CHECK-T2-NEXT: br i1 [[CMP34]], label [[WHILE_END43_LOOPEXIT:%.*]], label [[WHILE_BODY36]]
; CHECK-T2: while.end43.loopexit: ; CHECK-T2: while.end43.loopexit:
; CHECK-T2-NEXT: [[ADD6_I868_LCSSA:%.*]] = phi i32 [ [[ADD6_I868]], [[WHILE_BODY36]] ] ; CHECK-T2-NEXT: [[ADD6_I868_LCSSA:%.*]] = phi i32 [ [[ADD6_I868]], [[WHILE_BODY36]] ]
; CHECK-T2-NEXT: br label [[WHILE_END43]] ; CHECK-T2-NEXT: br label [[WHILE_END43]]
; CHECK-T2: while.end43: ; CHECK-T2: while.end43:
; CHECK-T2-NEXT: [[SUM_2_LCSSA:%.*]] = phi i32 [ [[SUM_1_LCSSA]], [[WHILE_END31]] ], [ [[ADD6_I868_LCSSA]], [[WHILE_END43_LOOPEXIT]] ] ; CHECK-T2-NEXT: [[SUM_2_LCSSA:%.*]] = phi i32 [ [[SUM_1_LCSSA]], [[WHILE_END31]] ], [ [[ADD6_I868_LCSSA]], [[WHILE_END43_LOOPEXIT]] ]
; CHECK-T2-NEXT: [[TMP17:%.*]] = lshr i32 [[SUM_2_LCSSA]], 15 ; CHECK-T2-NEXT: [[TMP14:%.*]] = lshr i32 [[SUM_2_LCSSA]], 15
; CHECK-T2-NEXT: [[CONV45:%.*]] = trunc i32 [[TMP17]] to i16 ; CHECK-T2-NEXT: [[CONV45:%.*]] = trunc i32 [[TMP14]] to i16
; CHECK-T2-NEXT: [[INCDEC_PTR46]] = getelementptr inbounds i16, i16* [[POUT_11069]], i32 1 ; CHECK-T2-NEXT: [[INCDEC_PTR46]] = getelementptr inbounds i16, i16* [[POUT_11069]], i32 1
; CHECK-T2-NEXT: store i16 [[CONV45]], i16* [[POUT_11069]], align 2 ; CHECK-T2-NEXT: store i16 [[CONV45]], i16* [[POUT_11069]], align 2
; CHECK-T2-NEXT: [[SUB47:%.*]] = add i32 [[COUNT_11072]], -1 ; CHECK-T2-NEXT: [[SUB47:%.*]] = add i32 [[COUNT_11072]], -1
; CHECK-T2-NEXT: [[ADD_PTR48]] = getelementptr inbounds i16, i16* [[PSRCA_PSRCB]], i32 [[SUB47]] ; CHECK-T2-NEXT: [[ADD_PTR48]] = getelementptr inbounds i16, i16* [[PSRCA_PSRCB]], i32 [[SUB47]]
; CHECK-T2-NEXT: [[INC49]] = add i32 [[COUNT_11072]], 1 ; CHECK-T2-NEXT: [[INC49]] = add i32 [[COUNT_11072]], 1
; CHECK-T2-NEXT: [[DEC50]] = add i32 [[BLOCKSIZE1_11071]], -1 ; CHECK-T2-NEXT: [[DEC50]] = add i32 [[BLOCKSIZE1_11071]], -1
; CHECK-T2-NEXT: [[CMP16:%.*]] = icmp eq i32 [[DEC50]], 0 ; CHECK-T2-NEXT: [[CMP16:%.*]] = icmp eq i32 [[DEC50]], 0
; CHECK-T2-NEXT: br i1 [[CMP16]], label [[EXIT_LOOPEXIT:%.*]], label [[WHILE_BODY18]] ; CHECK-T2-NEXT: br i1 [[CMP16]], label [[EXIT_LOOPEXIT:%.*]], label [[WHILE_BODY18]]
▲ Show 20 Lines • Show All 166 Lines • Show Last 20 Lines

llvm/test/CodeGen/ARM/indvar-unroll-imm-cost.ll

Show All 12 Lines
; CHECK: for.cond.cleanup: ; CHECK: for.cond.cleanup:
; CHECK-NEXT: ret void ; CHECK-NEXT: ret void
; CHECK: for.body: ; CHECK: for.body:
; CHECK-NEXT: [[I_092:%.*]] = phi i32 [ [[INC42:%.*]], [[FOR_END40:%.*]] ], [ 0, [[FOR_BODY_PREHEADER]] ] ; CHECK-NEXT: [[I_092:%.*]] = phi i32 [ [[INC42:%.*]], [[FOR_END40:%.*]] ], [ 0, [[FOR_BODY_PREHEADER]] ]
; CHECK-NEXT: [[PDEST_ADDR_091:%.*]] = phi i32* [ [[PDEST_ADDR_2_LCSSA:%.*]], [[FOR_END40]] ], [ [[PDEST:%.*]], [[FOR_BODY_PREHEADER]] ] ; CHECK-NEXT: [[PDEST_ADDR_091:%.*]] = phi i32* [ [[PDEST_ADDR_2_LCSSA:%.*]], [[FOR_END40]] ], [ [[PDEST:%.*]], [[FOR_BODY_PREHEADER]] ]
; CHECK-NEXT: [[PSRCA_ADDR_090:%.*]] = phi i16* [ [[PSRCA_ADDR_2_LCSSA:%.*]], [[FOR_END40]] ], [ [[PSRCA:%.*]], [[FOR_BODY_PREHEADER]] ] ; CHECK-NEXT: [[PSRCA_ADDR_090:%.*]] = phi i16* [ [[PSRCA_ADDR_2_LCSSA:%.*]], [[FOR_END40]] ], [ [[PSRCA:%.*]], [[FOR_BODY_PREHEADER]] ]
; CHECK-NEXT: [[PSRCB_ADDR_089:%.*]] = phi i16* [ [[PSRCB_ADDR_2_LCSSA:%.*]], [[FOR_END40]] ], [ [[PSRCB:%.*]], [[FOR_BODY_PREHEADER]] ] ; CHECK-NEXT: [[PSRCB_ADDR_089:%.*]] = phi i16* [ [[PSRCB_ADDR_2_LCSSA:%.*]], [[FOR_END40]] ], [ [[PSRCB:%.*]], [[FOR_BODY_PREHEADER]] ]
; CHECK-NEXT: [[TMP0:%.*]] = lshr i32 [[I_092]], 2 ; CHECK-NEXT: [[TMP0:%.*]] = lshr i32 [[I_092]], 2
; CHECK-NEXT: [[TMP1:%.*]] = add i32 [[TMP0]], -1 ; CHECK-NEXT: [[TMP1:%.*]] = add nuw nsw i32 [[TMP0]], 3
; CHECK-NEXT: [[TMP2:%.*]] = lshr i32 [[TMP1]], 2 ; CHECK-NEXT: [[TMP2:%.*]] = and i32 [[TMP1]], 2147483644
; CHECK-NEXT: [[TMP3:%.*]] = add nuw nsw i32 [[TMP2]], 1 ; CHECK-NEXT: [[CMP272:%.*]] = icmp eq i32 [[TMP0]], 0
; CHECK-NEXT: [[TMP4:%.*]] = lshr i32 [[I_092]], 2
; CHECK-NEXT: [[TMP5:%.*]] = add nuw nsw i32 [[TMP4]], 3
; CHECK-NEXT: [[TMP6:%.*]] = and i32 [[TMP5]], 2147483644
; CHECK-NEXT: [[CMP272:%.*]] = icmp eq i32 [[TMP4]], 0
; CHECK-NEXT: br i1 [[CMP272]], label [[FOR_END:%.*]], label [[FOR_BODY3_PREHEADER:%.*]] ; CHECK-NEXT: br i1 [[CMP272]], label [[FOR_END:%.*]], label [[FOR_BODY3_PREHEADER:%.*]]
; CHECK: for.body3.preheader: ; CHECK: for.body3.preheader:
; CHECK-NEXT: [[XTRAITER:%.*]] = and i32 [[TMP3]], 3
; CHECK-NEXT: [[TMP7:%.*]] = icmp ult i32 [[TMP2]], 3
; CHECK-NEXT: br i1 [[TMP7]], label [[FOR_END_LOOPEXIT_UNR_LCSSA:%.*]], label [[FOR_BODY3_PREHEADER_NEW:%.*]]
; CHECK: for.body3.preheader.new:
; CHECK-NEXT: [[UNROLL_ITER:%.*]] = sub i32 [[TMP3]], [[XTRAITER]]
; CHECK-NEXT: br label [[FOR_BODY3:%.*]] ; CHECK-NEXT: br label [[FOR_BODY3:%.*]]
; CHECK: for.body3: ; CHECK: for.body3:
; CHECK-NEXT: [[J_076:%.*]] = phi i32 [ 0, [[FOR_BODY3_PREHEADER_NEW]] ], [ [[ADD24_3:%.*]], [[FOR_BODY3]] ] ; CHECK-NEXT: [[J_076:%.*]] = phi i32 [ [[ADD24:%.*]], [[FOR_BODY3]] ], [ 0, [[FOR_BODY3_PREHEADER]] ]
; CHECK-NEXT: [[PDEST_ADDR_175:%.*]] = phi i32* [ [[PDEST_ADDR_091]], [[FOR_BODY3_PREHEADER_NEW]] ], [ [[INCDEC_PTR_3:%.*]], [[FOR_BODY3]] ] ; CHECK-NEXT: [[PDEST_ADDR_175:%.*]] = phi i32* [ [[INCDEC_PTR:%.*]], [[FOR_BODY3]] ], [ [[PDEST_ADDR_091]], [[FOR_BODY3_PREHEADER]] ]
; CHECK-NEXT: [[PSRCA_ADDR_174:%.*]] = phi i16* [ [[PSRCA_ADDR_090]], [[FOR_BODY3_PREHEADER_NEW]] ], [ [[ADD_PTR_3:%.*]], [[FOR_BODY3]] ] ; CHECK-NEXT: [[PSRCA_ADDR_174:%.*]] = phi i16* [ [[ADD_PTR:%.*]], [[FOR_BODY3]] ], [ [[PSRCA_ADDR_090]], [[FOR_BODY3_PREHEADER]] ]
; CHECK-NEXT: [[PSRCB_ADDR_173:%.*]] = phi i16* [ [[PSRCB_ADDR_089]], [[FOR_BODY3_PREHEADER_NEW]] ], [ [[ADD_PTR23_3:%.*]], [[FOR_BODY3]] ] ; CHECK-NEXT: [[PSRCB_ADDR_173:%.*]] = phi i16* [ [[ADD_PTR23:%.*]], [[FOR_BODY3]] ], [ [[PSRCB_ADDR_089]], [[FOR_BODY3_PREHEADER]] ]
; CHECK-NEXT: [[NITER:%.*]] = phi i32 [ [[UNROLL_ITER]], [[FOR_BODY3_PREHEADER_NEW]] ], [ [[NITER_NSUB_3:%.*]], [[FOR_BODY3]] ] ; CHECK-NEXT: [[TMP3:%.*]] = load i16, i16* [[PSRCA_ADDR_174]], align 2
; CHECK-NEXT: [[TMP8:%.*]] = load i16, i16* [[PSRCA_ADDR_174]], align 2 ; CHECK-NEXT: [[CONV:%.*]] = sext i16 [[TMP3]] to i32
; CHECK-NEXT: [[CONV:%.*]] = sext i16 [[TMP8]] to i32 ; CHECK-NEXT: [[TMP4:%.*]] = load i16, i16* [[PSRCB_ADDR_173]], align 2
; CHECK-NEXT: [[TMP9:%.*]] = load i16, i16* [[PSRCB_ADDR_173]], align 2 ; CHECK-NEXT: [[CONV5:%.*]] = sext i16 [[TMP4]] to i32
; CHECK-NEXT: [[CONV5:%.*]] = sext i16 [[TMP9]] to i32
; CHECK-NEXT: [[MUL:%.*]] = mul nsw i32 [[CONV5]], [[CONV]] ; CHECK-NEXT: [[MUL:%.*]] = mul nsw i32 [[CONV5]], [[CONV]]
; CHECK-NEXT: [[ARRAYIDX6:%.*]] = getelementptr inbounds i16, i16* [[PSRCA_ADDR_174]], i32 1 ; CHECK-NEXT: [[ARRAYIDX6:%.*]] = getelementptr inbounds i16, i16* [[PSRCA_ADDR_174]], i32 1
; CHECK-NEXT: [[TMP10:%.*]] = load i16, i16* [[ARRAYIDX6]], align 2 ; CHECK-NEXT: [[TMP5:%.*]] = load i16, i16* [[ARRAYIDX6]], align 2
; CHECK-NEXT: [[CONV7:%.*]] = sext i16 [[TMP10]] to i32 ; CHECK-NEXT: [[CONV7:%.*]] = sext i16 [[TMP5]] to i32
; CHECK-NEXT: [[ARRAYIDX8:%.*]] = getelementptr inbounds i16, i16* [[PSRCB_ADDR_173]], i32 1 ; CHECK-NEXT: [[ARRAYIDX8:%.*]] = getelementptr inbounds i16, i16* [[PSRCB_ADDR_173]], i32 1
; CHECK-NEXT: [[TMP11:%.*]] = load i16, i16* [[ARRAYIDX8]], align 2 ; CHECK-NEXT: [[TMP6:%.*]] = load i16, i16* [[ARRAYIDX8]], align 2
; CHECK-NEXT: [[CONV9:%.*]] = sext i16 [[TMP11]] to i32 ; CHECK-NEXT: [[CONV9:%.*]] = sext i16 [[TMP6]] to i32
; CHECK-NEXT: [[MUL10:%.*]] = mul nsw i32 [[CONV9]], [[CONV7]] ; CHECK-NEXT: [[MUL10:%.*]] = mul nsw i32 [[CONV9]], [[CONV7]]
; CHECK-NEXT: [[ARRAYIDX11:%.*]] = getelementptr inbounds i16, i16* [[PSRCA_ADDR_174]], i32 2 ; CHECK-NEXT: [[ARRAYIDX11:%.*]] = getelementptr inbounds i16, i16* [[PSRCA_ADDR_174]], i32 2
; CHECK-NEXT: [[TMP12:%.*]] = load i16, i16* [[ARRAYIDX11]], align 2 ; CHECK-NEXT: [[TMP7:%.*]] = load i16, i16* [[ARRAYIDX11]], align 2
; CHECK-NEXT: [[CONV12:%.*]] = sext i16 [[TMP12]] to i32 ; CHECK-NEXT: [[CONV12:%.*]] = sext i16 [[TMP7]] to i32
; CHECK-NEXT: [[ARRAYIDX13:%.*]] = getelementptr inbounds i16, i16* [[PSRCB_ADDR_173]], i32 3 ; CHECK-NEXT: [[ARRAYIDX13:%.*]] = getelementptr inbounds i16, i16* [[PSRCB_ADDR_173]], i32 3
; CHECK-NEXT: [[TMP13:%.*]] = load i16, i16* [[ARRAYIDX13]], align 2 ; CHECK-NEXT: [[TMP8:%.*]] = load i16, i16* [[ARRAYIDX13]], align 2
; CHECK-NEXT: [[CONV14:%.*]] = sext i16 [[TMP13]] to i32 ; CHECK-NEXT: [[CONV14:%.*]] = sext i16 [[TMP8]] to i32
; CHECK-NEXT: [[MUL15:%.*]] = mul nsw i32 [[CONV14]], [[CONV12]] ; CHECK-NEXT: [[MUL15:%.*]] = mul nsw i32 [[CONV14]], [[CONV12]]
; CHECK-NEXT: [[ARRAYIDX17:%.*]] = getelementptr inbounds i16, i16* [[PSRCA_ADDR_174]], i32 3 ; CHECK-NEXT: [[ARRAYIDX17:%.*]] = getelementptr inbounds i16, i16* [[PSRCA_ADDR_174]], i32 3
; CHECK-NEXT: [[TMP14:%.*]] = load i16, i16* [[ARRAYIDX17]], align 2 ; CHECK-NEXT: [[TMP9:%.*]] = load i16, i16* [[ARRAYIDX17]], align 2
; CHECK-NEXT: [[CONV18:%.*]] = sext i16 [[TMP14]] to i32 ; CHECK-NEXT: [[CONV18:%.*]] = sext i16 [[TMP9]] to i32
; CHECK-NEXT: [[ADD21:%.*]] = add i32 [[MUL10]], [[MUL]] ; CHECK-NEXT: [[ADD21:%.*]] = add i32 [[MUL10]], [[MUL]]
; CHECK-NEXT: [[ADD:%.*]] = add i32 [[ADD21]], [[CONV14]] ; CHECK-NEXT: [[ADD:%.*]] = add i32 [[ADD21]], [[CONV14]]
; CHECK-NEXT: [[ADD16:%.*]] = add i32 [[ADD]], [[MUL15]] ; CHECK-NEXT: [[ADD16:%.*]] = add i32 [[ADD]], [[MUL15]]
; CHECK-NEXT: [[ADD22:%.*]] = add i32 [[ADD16]], [[CONV18]] ; CHECK-NEXT: [[ADD22:%.*]] = add i32 [[ADD16]], [[CONV18]]
; CHECK-NEXT: store i32 [[ADD22]], i32* [[PDEST_ADDR_175]], align 4 ; CHECK-NEXT: store i32 [[ADD22]], i32* [[PDEST_ADDR_175]], align 4
; CHECK-NEXT: [[ADD_PTR:%.*]] = getelementptr inbounds i16, i16* [[PSRCA_ADDR_174]], i32 4 ; CHECK-NEXT: [[ADD_PTR]] = getelementptr inbounds i16, i16* [[PSRCA_ADDR_174]], i32 4
; CHECK-NEXT: [[ADD_PTR23:%.*]] = getelementptr inbounds i16, i16* [[PSRCB_ADDR_173]], i32 4 ; CHECK-NEXT: [[ADD_PTR23]] = getelementptr inbounds i16, i16* [[PSRCB_ADDR_173]], i32 4
; CHECK-NEXT: [[INCDEC_PTR:%.*]] = getelementptr inbounds i32, i32* [[PDEST_ADDR_175]], i32 1 ; CHECK-NEXT: [[INCDEC_PTR]] = getelementptr inbounds i32, i32* [[PDEST_ADDR_175]], i32 1
; CHECK-NEXT: [[ADD24:%.*]] = add nuw nsw i32 [[J_076]], 4 ; CHECK-NEXT: [[ADD24]] = add nuw nsw i32 [[J_076]], 4
; CHECK-NEXT: [[NITER_NSUB:%.*]] = sub i32 [[NITER]], 1 ; CHECK-NEXT: [[CMP2:%.*]] = icmp ult i32 [[ADD24]], [[TMP0]]
; CHECK-NEXT: [[TMP15:%.*]] = load i16, i16* [[ADD_PTR]], align 2 ; CHECK-NEXT: br i1 [[CMP2]], label [[FOR_BODY3]], label [[FOR_END_LOOPEXIT:%.*]]
; CHECK-NEXT: [[CONV_1:%.*]] = sext i16 [[TMP15]] to i32
; CHECK-NEXT: [[TMP16:%.*]] = load i16, i16* [[ADD_PTR23]], align 2
; CHECK-NEXT: [[CONV5_1:%.*]] = sext i16 [[TMP16]] to i32
; CHECK-NEXT: [[MUL_1:%.*]] = mul nsw i32 [[CONV5_1]], [[CONV_1]]
; CHECK-NEXT: [[ARRAYIDX6_1:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR]], i32 1
; CHECK-NEXT: [[TMP17:%.*]] = load i16, i16* [[ARRAYIDX6_1]], align 2
; CHECK-NEXT: [[CONV7_1:%.*]] = sext i16 [[TMP17]] to i32
; CHECK-NEXT: [[ARRAYIDX8_1:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR23]], i32 1
; CHECK-NEXT: [[TMP18:%.*]] = load i16, i16* [[ARRAYIDX8_1]], align 2
; CHECK-NEXT: [[CONV9_1:%.*]] = sext i16 [[TMP18]] to i32
; CHECK-NEXT: [[MUL10_1:%.*]] = mul nsw i32 [[CONV9_1]], [[CONV7_1]]
; CHECK-NEXT: [[ARRAYIDX11_1:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR]], i32 2
; CHECK-NEXT: [[TMP19:%.*]] = load i16, i16* [[ARRAYIDX11_1]], align 2
; CHECK-NEXT: [[CONV12_1:%.*]] = sext i16 [[TMP19]] to i32
; CHECK-NEXT: [[ARRAYIDX13_1:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR23]], i32 3
; CHECK-NEXT: [[TMP20:%.*]] = load i16, i16* [[ARRAYIDX13_1]], align 2
; CHECK-NEXT: [[CONV14_1:%.*]] = sext i16 [[TMP20]] to i32
; CHECK-NEXT: [[MUL15_1:%.*]] = mul nsw i32 [[CONV14_1]], [[CONV12_1]]
; CHECK-NEXT: [[ARRAYIDX17_1:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR]], i32 3
; CHECK-NEXT: [[TMP21:%.*]] = load i16, i16* [[ARRAYIDX17_1]], align 2
; CHECK-NEXT: [[CONV18_1:%.*]] = sext i16 [[TMP21]] to i32
; CHECK-NEXT: [[ADD21_1:%.*]] = add i32 [[MUL10_1]], [[MUL_1]]
; CHECK-NEXT: [[ADD_1:%.*]] = add i32 [[ADD21_1]], [[CONV14_1]]
; CHECK-NEXT: [[ADD16_1:%.*]] = add i32 [[ADD_1]], [[MUL15_1]]
; CHECK-NEXT: [[ADD22_1:%.*]] = add i32 [[ADD16_1]], [[CONV18_1]]
; CHECK-NEXT: store i32 [[ADD22_1]], i32* [[INCDEC_PTR]], align 4
; CHECK-NEXT: [[ADD_PTR_1:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR]], i32 4
; CHECK-NEXT: [[ADD_PTR23_1:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR23]], i32 4
; CHECK-NEXT: [[INCDEC_PTR_1:%.*]] = getelementptr inbounds i32, i32* [[INCDEC_PTR]], i32 1
; CHECK-NEXT: [[ADD24_1:%.*]] = add nuw nsw i32 [[ADD24]], 4
; CHECK-NEXT: [[NITER_NSUB_1:%.*]] = sub i32 [[NITER_NSUB]], 1
; CHECK-NEXT: [[TMP22:%.*]] = load i16, i16* [[ADD_PTR_1]], align 2
; CHECK-NEXT: [[CONV_2:%.*]] = sext i16 [[TMP22]] to i32
; CHECK-NEXT: [[TMP23:%.*]] = load i16, i16* [[ADD_PTR23_1]], align 2
; CHECK-NEXT: [[CONV5_2:%.*]] = sext i16 [[TMP23]] to i32
; CHECK-NEXT: [[MUL_2:%.*]] = mul nsw i32 [[CONV5_2]], [[CONV_2]]
; CHECK-NEXT: [[ARRAYIDX6_2:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR_1]], i32 1
; CHECK-NEXT: [[TMP24:%.*]] = load i16, i16* [[ARRAYIDX6_2]], align 2
; CHECK-NEXT: [[CONV7_2:%.*]] = sext i16 [[TMP24]] to i32
; CHECK-NEXT: [[ARRAYIDX8_2:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR23_1]], i32 1
; CHECK-NEXT: [[TMP25:%.*]] = load i16, i16* [[ARRAYIDX8_2]], align 2
; CHECK-NEXT: [[CONV9_2:%.*]] = sext i16 [[TMP25]] to i32
; CHECK-NEXT: [[MUL10_2:%.*]] = mul nsw i32 [[CONV9_2]], [[CONV7_2]]
; CHECK-NEXT: [[ARRAYIDX11_2:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR_1]], i32 2
; CHECK-NEXT: [[TMP26:%.*]] = load i16, i16* [[ARRAYIDX11_2]], align 2
; CHECK-NEXT: [[CONV12_2:%.*]] = sext i16 [[TMP26]] to i32
; CHECK-NEXT: [[ARRAYIDX13_2:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR23_1]], i32 3
; CHECK-NEXT: [[TMP27:%.*]] = load i16, i16* [[ARRAYIDX13_2]], align 2
; CHECK-NEXT: [[CONV14_2:%.*]] = sext i16 [[TMP27]] to i32
; CHECK-NEXT: [[MUL15_2:%.*]] = mul nsw i32 [[CONV14_2]], [[CONV12_2]]
; CHECK-NEXT: [[ARRAYIDX17_2:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR_1]], i32 3
; CHECK-NEXT: [[TMP28:%.*]] = load i16, i16* [[ARRAYIDX17_2]], align 2
; CHECK-NEXT: [[CONV18_2:%.*]] = sext i16 [[TMP28]] to i32
; CHECK-NEXT: [[ADD21_2:%.*]] = add i32 [[MUL10_2]], [[MUL_2]]
; CHECK-NEXT: [[ADD_2:%.*]] = add i32 [[ADD21_2]], [[CONV14_2]]
; CHECK-NEXT: [[ADD16_2:%.*]] = add i32 [[ADD_2]], [[MUL15_2]]
; CHECK-NEXT: [[ADD22_2:%.*]] = add i32 [[ADD16_2]], [[CONV18_2]]
; CHECK-NEXT: store i32 [[ADD22_2]], i32* [[INCDEC_PTR_1]], align 4
; CHECK-NEXT: [[ADD_PTR_2:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR_1]], i32 4
; CHECK-NEXT: [[ADD_PTR23_2:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR23_1]], i32 4
; CHECK-NEXT: [[INCDEC_PTR_2:%.*]] = getelementptr inbounds i32, i32* [[INCDEC_PTR_1]], i32 1
; CHECK-NEXT: [[ADD24_2:%.*]] = add nuw nsw i32 [[ADD24_1]], 4
; CHECK-NEXT: [[NITER_NSUB_2:%.*]] = sub i32 [[NITER_NSUB_1]], 1
; CHECK-NEXT: [[TMP29:%.*]] = load i16, i16* [[ADD_PTR_2]], align 2
; CHECK-NEXT: [[CONV_3:%.*]] = sext i16 [[TMP29]] to i32
; CHECK-NEXT: [[TMP30:%.*]] = load i16, i16* [[ADD_PTR23_2]], align 2
; CHECK-NEXT: [[CONV5_3:%.*]] = sext i16 [[TMP30]] to i32
; CHECK-NEXT: [[MUL_3:%.*]] = mul nsw i32 [[CONV5_3]], [[CONV_3]]
; CHECK-NEXT: [[ARRAYIDX6_3:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR_2]], i32 1
; CHECK-NEXT: [[TMP31:%.*]] = load i16, i16* [[ARRAYIDX6_3]], align 2
; CHECK-NEXT: [[CONV7_3:%.*]] = sext i16 [[TMP31]] to i32
; CHECK-NEXT: [[ARRAYIDX8_3:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR23_2]], i32 1
; CHECK-NEXT: [[TMP32:%.*]] = load i16, i16* [[ARRAYIDX8_3]], align 2
; CHECK-NEXT: [[CONV9_3:%.*]] = sext i16 [[TMP32]] to i32
; CHECK-NEXT: [[MUL10_3:%.*]] = mul nsw i32 [[CONV9_3]], [[CONV7_3]]
; CHECK-NEXT: [[ARRAYIDX11_3:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR_2]], i32 2
; CHECK-NEXT: [[TMP33:%.*]] = load i16, i16* [[ARRAYIDX11_3]], align 2
; CHECK-NEXT: [[CONV12_3:%.*]] = sext i16 [[TMP33]] to i32
; CHECK-NEXT: [[ARRAYIDX13_3:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR23_2]], i32 3
; CHECK-NEXT: [[TMP34:%.*]] = load i16, i16* [[ARRAYIDX13_3]], align 2
; CHECK-NEXT: [[CONV14_3:%.*]] = sext i16 [[TMP34]] to i32
; CHECK-NEXT: [[MUL15_3:%.*]] = mul nsw i32 [[CONV14_3]], [[CONV12_3]]
; CHECK-NEXT: [[ARRAYIDX17_3:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR_2]], i32 3
; CHECK-NEXT: [[TMP35:%.*]] = load i16, i16* [[ARRAYIDX17_3]], align 2
; CHECK-NEXT: [[CONV18_3:%.*]] = sext i16 [[TMP35]] to i32
; CHECK-NEXT: [[ADD21_3:%.*]] = add i32 [[MUL10_3]], [[MUL_3]]
; CHECK-NEXT: [[ADD_3:%.*]] = add i32 [[ADD21_3]], [[CONV14_3]]
; CHECK-NEXT: [[ADD16_3:%.*]] = add i32 [[ADD_3]], [[MUL15_3]]
; CHECK-NEXT: [[ADD22_3:%.*]] = add i32 [[ADD16_3]], [[CONV18_3]]
; CHECK-NEXT: store i32 [[ADD22_3]], i32* [[INCDEC_PTR_2]], align 4
; CHECK-NEXT: [[ADD_PTR_3]] = getelementptr inbounds i16, i16* [[ADD_PTR_2]], i32 4
; CHECK-NEXT: [[ADD_PTR23_3]] = getelementptr inbounds i16, i16* [[ADD_PTR23_2]], i32 4
; CHECK-NEXT: [[INCDEC_PTR_3]] = getelementptr inbounds i32, i32* [[INCDEC_PTR_2]], i32 1
; CHECK-NEXT: [[ADD24_3]] = add nuw nsw i32 [[ADD24_2]], 4
; CHECK-NEXT: [[NITER_NSUB_3]] = sub i32 [[NITER_NSUB_2]], 1
; CHECK-NEXT: [[NITER_NCMP_3:%.*]] = icmp ne i32 [[NITER_NSUB_3]], 0
; CHECK-NEXT: br i1 [[NITER_NCMP_3]], label [[FOR_BODY3]], label [[FOR_END_LOOPEXIT_UNR_LCSSA_LOOPEXIT:%.*]]
; CHECK: for.end.loopexit.unr-lcssa.loopexit:
; CHECK-NEXT: [[ADD_PTR_LCSSA_PH_PH:%.*]] = phi i16* [ [[ADD_PTR_3]], [[FOR_BODY3]] ]
; CHECK-NEXT: [[ADD_PTR23_LCSSA_PH_PH:%.*]] = phi i16* [ [[ADD_PTR23_3]], [[FOR_BODY3]] ]
; CHECK-NEXT: [[INCDEC_PTR_LCSSA_PH_PH:%.*]] = phi i32* [ [[INCDEC_PTR_3]], [[FOR_BODY3]] ]
; CHECK-NEXT: [[J_076_UNR_PH:%.*]] = phi i32 [ [[ADD24_3]], [[FOR_BODY3]] ]
; CHECK-NEXT: [[PDEST_ADDR_175_UNR_PH:%.*]] = phi i32* [ [[INCDEC_PTR_3]], [[FOR_BODY3]] ]
; CHECK-NEXT: [[PSRCA_ADDR_174_UNR_PH:%.*]] = phi i16* [ [[ADD_PTR_3]], [[FOR_BODY3]] ]
; CHECK-NEXT: [[PSRCB_ADDR_173_UNR_PH:%.*]] = phi i16* [ [[ADD_PTR23_3]], [[FOR_BODY3]] ]
; CHECK-NEXT: br label [[FOR_END_LOOPEXIT_UNR_LCSSA]]
; CHECK: for.end.loopexit.unr-lcssa:
; CHECK-NEXT: [[ADD_PTR_LCSSA_PH:%.*]] = phi i16* [ undef, [[FOR_BODY3_PREHEADER]] ], [ [[ADD_PTR_LCSSA_PH_PH]], [[FOR_END_LOOPEXIT_UNR_LCSSA_LOOPEXIT]] ]
; CHECK-NEXT: [[ADD_PTR23_LCSSA_PH:%.*]] = phi i16* [ undef, [[FOR_BODY3_PREHEADER]] ], [ [[ADD_PTR23_LCSSA_PH_PH]], [[FOR_END_LOOPEXIT_UNR_LCSSA_LOOPEXIT]] ]
; CHECK-NEXT: [[INCDEC_PTR_LCSSA_PH:%.*]] = phi i32* [ undef, [[FOR_BODY3_PREHEADER]] ], [ [[INCDEC_PTR_LCSSA_PH_PH]], [[FOR_END_LOOPEXIT_UNR_LCSSA_LOOPEXIT]] ]
; CHECK-NEXT: [[J_076_UNR:%.*]] = phi i32 [ 0, [[FOR_BODY3_PREHEADER]] ], [ [[J_076_UNR_PH]], [[FOR_END_LOOPEXIT_UNR_LCSSA_LOOPEXIT]] ]
; CHECK-NEXT: [[PDEST_ADDR_175_UNR:%.*]] = phi i32* [ [[PDEST_ADDR_091]], [[FOR_BODY3_PREHEADER]] ], [ [[PDEST_ADDR_175_UNR_PH]], [[FOR_END_LOOPEXIT_UNR_LCSSA_LOOPEXIT]] ]
; CHECK-NEXT: [[PSRCA_ADDR_174_UNR:%.*]] = phi i16* [ [[PSRCA_ADDR_090]], [[FOR_BODY3_PREHEADER]] ], [ [[PSRCA_ADDR_174_UNR_PH]], [[FOR_END_LOOPEXIT_UNR_LCSSA_LOOPEXIT]] ]
; CHECK-NEXT: [[PSRCB_ADDR_173_UNR:%.*]] = phi i16* [ [[PSRCB_ADDR_089]], [[FOR_BODY3_PREHEADER]] ], [ [[PSRCB_ADDR_173_UNR_PH]], [[FOR_END_LOOPEXIT_UNR_LCSSA_LOOPEXIT]] ]
; CHECK-NEXT: [[LCMP_MOD:%.*]] = icmp ne i32 [[XTRAITER]], 0
; CHECK-NEXT: br i1 [[LCMP_MOD]], label [[FOR_BODY3_EPIL_PREHEADER:%.*]], label [[FOR_END_LOOPEXIT:%.*]]
; CHECK: for.body3.epil.preheader:
; CHECK-NEXT: br label [[FOR_BODY3_EPIL:%.*]]
; CHECK: for.body3.epil:
; CHECK-NEXT: [[TMP36:%.*]] = load i16, i16* [[PSRCA_ADDR_174_UNR]], align 2
; CHECK-NEXT: [[CONV_EPIL:%.*]] = sext i16 [[TMP36]] to i32
; CHECK-NEXT: [[TMP37:%.*]] = load i16, i16* [[PSRCB_ADDR_173_UNR]], align 2
; CHECK-NEXT: [[CONV5_EPIL:%.*]] = sext i16 [[TMP37]] to i32
; CHECK-NEXT: [[MUL_EPIL:%.*]] = mul nsw i32 [[CONV5_EPIL]], [[CONV_EPIL]]
; CHECK-NEXT: [[ARRAYIDX6_EPIL:%.*]] = getelementptr inbounds i16, i16* [[PSRCA_ADDR_174_UNR]], i32 1
; CHECK-NEXT: [[TMP38:%.*]] = load i16, i16* [[ARRAYIDX6_EPIL]], align 2
; CHECK-NEXT: [[CONV7_EPIL:%.*]] = sext i16 [[TMP38]] to i32
; CHECK-NEXT: [[ARRAYIDX8_EPIL:%.*]] = getelementptr inbounds i16, i16* [[PSRCB_ADDR_173_UNR]], i32 1
; CHECK-NEXT: [[TMP39:%.*]] = load i16, i16* [[ARRAYIDX8_EPIL]], align 2
; CHECK-NEXT: [[CONV9_EPIL:%.*]] = sext i16 [[TMP39]] to i32
; CHECK-NEXT: [[MUL10_EPIL:%.*]] = mul nsw i32 [[CONV9_EPIL]], [[CONV7_EPIL]]
; CHECK-NEXT: [[ARRAYIDX11_EPIL:%.*]] = getelementptr inbounds i16, i16* [[PSRCA_ADDR_174_UNR]], i32 2
; CHECK-NEXT: [[TMP40:%.*]] = load i16, i16* [[ARRAYIDX11_EPIL]], align 2
; CHECK-NEXT: [[CONV12_EPIL:%.*]] = sext i16 [[TMP40]] to i32
; CHECK-NEXT: [[ARRAYIDX13_EPIL:%.*]] = getelementptr inbounds i16, i16* [[PSRCB_ADDR_173_UNR]], i32 3
; CHECK-NEXT: [[TMP41:%.*]] = load i16, i16* [[ARRAYIDX13_EPIL]], align 2
; CHECK-NEXT: [[CONV14_EPIL:%.*]] = sext i16 [[TMP41]] to i32
; CHECK-NEXT: [[MUL15_EPIL:%.*]] = mul nsw i32 [[CONV14_EPIL]], [[CONV12_EPIL]]
; CHECK-NEXT: [[ARRAYIDX17_EPIL:%.*]] = getelementptr inbounds i16, i16* [[PSRCA_ADDR_174_UNR]], i32 3
; CHECK-NEXT: [[TMP42:%.*]] = load i16, i16* [[ARRAYIDX17_EPIL]], align 2
; CHECK-NEXT: [[CONV18_EPIL:%.*]] = sext i16 [[TMP42]] to i32
; CHECK-NEXT: [[ADD21_EPIL:%.*]] = add i32 [[MUL10_EPIL]], [[MUL_EPIL]]
; CHECK-NEXT: [[ADD_EPIL:%.*]] = add i32 [[ADD21_EPIL]], [[CONV14_EPIL]]
; CHECK-NEXT: [[ADD16_EPIL:%.*]] = add i32 [[ADD_EPIL]], [[MUL15_EPIL]]
; CHECK-NEXT: [[ADD22_EPIL:%.*]] = add i32 [[ADD16_EPIL]], [[CONV18_EPIL]]
; CHECK-NEXT: store i32 [[ADD22_EPIL]], i32* [[PDEST_ADDR_175_UNR]], align 4
; CHECK-NEXT: [[ADD_PTR_EPIL:%.*]] = getelementptr inbounds i16, i16* [[PSRCA_ADDR_174_UNR]], i32 4
; CHECK-NEXT: [[ADD_PTR23_EPIL:%.*]] = getelementptr inbounds i16, i16* [[PSRCB_ADDR_173_UNR]], i32 4
; CHECK-NEXT: [[INCDEC_PTR_EPIL:%.*]] = getelementptr inbounds i32, i32* [[PDEST_ADDR_175_UNR]], i32 1
; CHECK-NEXT: [[ADD24_EPIL:%.*]] = add nuw nsw i32 [[J_076_UNR]], 4
; CHECK-NEXT: [[EPIL_ITER_SUB:%.*]] = sub i32 [[XTRAITER]], 1
; CHECK-NEXT: [[EPIL_ITER_CMP:%.*]] = icmp ne i32 [[EPIL_ITER_SUB]], 0
; CHECK-NEXT: br i1 [[EPIL_ITER_CMP]], label [[FOR_BODY3_EPIL_1:%.*]], label [[FOR_END_LOOPEXIT_EPILOG_LCSSA:%.*]]
; CHECK: for.end.loopexit.epilog-lcssa:
; CHECK-NEXT: [[ADD_PTR_LCSSA_PH1:%.*]] = phi i16* [ [[ADD_PTR_EPIL]], [[FOR_BODY3_EPIL]] ], [ [[ADD_PTR_EPIL_1:%.*]], [[FOR_BODY3_EPIL_1]] ], [ [[ADD_PTR_EPIL_2:%.*]], [[FOR_BODY3_EPIL_2:%.*]] ]
; CHECK-NEXT: [[ADD_PTR23_LCSSA_PH2:%.*]] = phi i16* [ [[ADD_PTR23_EPIL]], [[FOR_BODY3_EPIL]] ], [ [[ADD_PTR23_EPIL_1:%.*]], [[FOR_BODY3_EPIL_1]] ], [ [[ADD_PTR23_EPIL_2:%.*]], [[FOR_BODY3_EPIL_2]] ]
; CHECK-NEXT: [[INCDEC_PTR_LCSSA_PH3:%.*]] = phi i32* [ [[INCDEC_PTR_EPIL]], [[FOR_BODY3_EPIL]] ], [ [[INCDEC_PTR_EPIL_1:%.*]], [[FOR_BODY3_EPIL_1]] ], [ [[INCDEC_PTR_EPIL_2:%.*]], [[FOR_BODY3_EPIL_2]] ]
; CHECK-NEXT: br label [[FOR_END_LOOPEXIT]]
; CHECK: for.end.loopexit: ; CHECK: for.end.loopexit:
; CHECK-NEXT: [[ADD_PTR_LCSSA:%.*]] = phi i16* [ [[ADD_PTR_LCSSA_PH]], [[FOR_END_LOOPEXIT_UNR_LCSSA]] ], [ [[ADD_PTR_LCSSA_PH1]], [[FOR_END_LOOPEXIT_EPILOG_LCSSA]] ] ; CHECK-NEXT: [[ADD_PTR_LCSSA:%.*]] = phi i16* [ [[ADD_PTR]], [[FOR_BODY3]] ]
; CHECK-NEXT: [[ADD_PTR23_LCSSA:%.*]] = phi i16* [ [[ADD_PTR23_LCSSA_PH]], [[FOR_END_LOOPEXIT_UNR_LCSSA]] ], [ [[ADD_PTR23_LCSSA_PH2]], [[FOR_END_LOOPEXIT_EPILOG_LCSSA]] ] ; CHECK-NEXT: [[ADD_PTR23_LCSSA:%.*]] = phi i16* [ [[ADD_PTR23]], [[FOR_BODY3]] ]
; CHECK-NEXT: [[INCDEC_PTR_LCSSA:%.*]] = phi i32* [ [[INCDEC_PTR_LCSSA_PH]], [[FOR_END_LOOPEXIT_UNR_LCSSA]] ], [ [[INCDEC_PTR_LCSSA_PH3]], [[FOR_END_LOOPEXIT_EPILOG_LCSSA]] ] ; CHECK-NEXT: [[INCDEC_PTR_LCSSA:%.*]] = phi i32* [ [[INCDEC_PTR]], [[FOR_BODY3]] ]
; CHECK-NEXT: br label [[FOR_END]] ; CHECK-NEXT: br label [[FOR_END]]
; CHECK: for.end: ; CHECK: for.end:
; CHECK-NEXT: [[PSRCB_ADDR_1_LCSSA:%.*]] = phi i16* [ [[PSRCB_ADDR_089]], [[FOR_BODY]] ], [ [[ADD_PTR23_LCSSA]], [[FOR_END_LOOPEXIT]] ] ; CHECK-NEXT: [[PSRCB_ADDR_1_LCSSA:%.*]] = phi i16* [ [[PSRCB_ADDR_089]], [[FOR_BODY]] ], [ [[ADD_PTR23_LCSSA]], [[FOR_END_LOOPEXIT]] ]
; CHECK-NEXT: [[PSRCA_ADDR_1_LCSSA:%.*]] = phi i16* [ [[PSRCA_ADDR_090]], [[FOR_BODY]] ], [ [[ADD_PTR_LCSSA]], [[FOR_END_LOOPEXIT]] ] ; CHECK-NEXT: [[PSRCA_ADDR_1_LCSSA:%.*]] = phi i16* [ [[PSRCA_ADDR_090]], [[FOR_BODY]] ], [ [[ADD_PTR_LCSSA]], [[FOR_END_LOOPEXIT]] ]
; CHECK-NEXT: [[PDEST_ADDR_1_LCSSA:%.*]] = phi i32* [ [[PDEST_ADDR_091]], [[FOR_BODY]] ], [ [[INCDEC_PTR_LCSSA]], [[FOR_END_LOOPEXIT]] ] ; CHECK-NEXT: [[PDEST_ADDR_1_LCSSA:%.*]] = phi i32* [ [[PDEST_ADDR_091]], [[FOR_BODY]] ], [ [[INCDEC_PTR_LCSSA]], [[FOR_END_LOOPEXIT]] ]
; CHECK-NEXT: [[J_0_LCSSA:%.*]] = phi i32 [ 0, [[FOR_BODY]] ], [ [[TMP6]], [[FOR_END_LOOPEXIT]] ] ; CHECK-NEXT: [[J_0_LCSSA:%.*]] = phi i32 [ 0, [[FOR_BODY]] ], [ [[TMP2]], [[FOR_END_LOOPEXIT]] ]
; CHECK-NEXT: [[REM:%.*]] = and i32 [[TMP4]], 3 ; CHECK-NEXT: [[REM:%.*]] = and i32 [[TMP0]], 3
; CHECK-NEXT: [[ADD25:%.*]] = or i32 [[J_0_LCSSA]], [[REM]] ; CHECK-NEXT: [[ADD25:%.*]] = or i32 [[J_0_LCSSA]], [[REM]]
; CHECK-NEXT: [[CMP2780:%.*]] = icmp ugt i32 [[ADD25]], [[J_0_LCSSA]] ; CHECK-NEXT: [[CMP2780:%.*]] = icmp ugt i32 [[ADD25]], [[J_0_LCSSA]]
; CHECK-NEXT: br i1 [[CMP2780]], label [[FOR_BODY29_PREHEADER:%.*]], label [[FOR_END40]] ; CHECK-NEXT: br i1 [[CMP2780]], label [[FOR_BODY29_PREHEADER:%.*]], label [[FOR_END40]]
; CHECK: for.body29.preheader: ; CHECK: for.body29.preheader:
; CHECK-NEXT: [[TMP43:%.*]] = sub nsw i32 [[ADD25]], [[J_0_LCSSA]] ; CHECK-NEXT: [[TMP10:%.*]] = sub nsw i32 [[ADD25]], [[J_0_LCSSA]]
; CHECK-NEXT: [[TMP44:%.*]] = sub i32 [[ADD25]], [[J_0_LCSSA]] ; CHECK-NEXT: [[TMP11:%.*]] = sub i32 [[ADD25]], [[J_0_LCSSA]]
; CHECK-NEXT: [[TMP45:%.*]] = add i32 [[ADD25]], -1 ; CHECK-NEXT: [[TMP12:%.*]] = add i32 [[ADD25]], -1
; CHECK-NEXT: [[TMP46:%.*]] = sub i32 [[TMP45]], [[J_0_LCSSA]] ; CHECK-NEXT: [[TMP13:%.*]] = sub i32 [[TMP12]], [[J_0_LCSSA]]
; CHECK-NEXT: [[XTRAITER4:%.*]] = and i32 [[TMP44]], 3 ; CHECK-NEXT: [[XTRAITER:%.*]] = and i32 [[TMP11]], 3
; CHECK-NEXT: [[LCMP_MOD5:%.*]] = icmp ne i32 [[XTRAITER4]], 0 ; CHECK-NEXT: [[LCMP_MOD:%.*]] = icmp ne i32 [[XTRAITER]], 0
; CHECK-NEXT: br i1 [[LCMP_MOD5]], label [[FOR_BODY29_PROL_PREHEADER:%.*]], label [[FOR_BODY29_PROL_LOOPEXIT:%.*]] ; CHECK-NEXT: br i1 [[LCMP_MOD]], label [[FOR_BODY29_PROL_PREHEADER:%.*]], label [[FOR_BODY29_PROL_LOOPEXIT:%.*]]
; CHECK: for.body29.prol.preheader: ; CHECK: for.body29.prol.preheader:
; CHECK-NEXT: br label [[FOR_BODY29_PROL:%.*]] ; CHECK-NEXT: br label [[FOR_BODY29_PROL:%.*]]
; CHECK: for.body29.prol: ; CHECK: for.body29.prol:
; CHECK-NEXT: [[ARRAYIDX30_PROL:%.*]] = getelementptr inbounds i16, i16* [[PSRCA_ADDR_1_LCSSA]], i32 [[J_0_LCSSA]] ; CHECK-NEXT: [[ARRAYIDX30_PROL:%.*]] = getelementptr inbounds i16, i16* [[PSRCA_ADDR_1_LCSSA]], i32 [[J_0_LCSSA]]
; CHECK-NEXT: [[TMP47:%.*]] = load i16, i16* [[ARRAYIDX30_PROL]], align 2 ; CHECK-NEXT: [[TMP14:%.*]] = load i16, i16* [[ARRAYIDX30_PROL]], align 2
; CHECK-NEXT: [[CONV31_PROL:%.*]] = sext i16 [[TMP47]] to i32 ; CHECK-NEXT: [[CONV31_PROL:%.*]] = sext i16 [[TMP14]] to i32
; CHECK-NEXT: [[ARRAYIDX32_PROL:%.*]] = getelementptr inbounds i16, i16* [[PSRCB_ADDR_1_LCSSA]], i32 [[J_0_LCSSA]] ; CHECK-NEXT: [[ARRAYIDX32_PROL:%.*]] = getelementptr inbounds i16, i16* [[PSRCB_ADDR_1_LCSSA]], i32 [[J_0_LCSSA]]
; CHECK-NEXT: [[TMP48:%.*]] = load i16, i16* [[ARRAYIDX32_PROL]], align 2 ; CHECK-NEXT: [[TMP15:%.*]] = load i16, i16* [[ARRAYIDX32_PROL]], align 2
; CHECK-NEXT: [[CONV33_PROL:%.*]] = sext i16 [[TMP48]] to i32 ; CHECK-NEXT: [[CONV33_PROL:%.*]] = sext i16 [[TMP15]] to i32
; CHECK-NEXT: [[MUL34_PROL:%.*]] = mul nsw i32 [[CONV33_PROL]], [[CONV31_PROL]] ; CHECK-NEXT: [[MUL34_PROL:%.*]] = mul nsw i32 [[CONV33_PROL]], [[CONV31_PROL]]
; CHECK-NEXT: [[TMP49:%.*]] = load i32, i32* [[PDEST_ADDR_1_LCSSA]], align 4 ; CHECK-NEXT: [[TMP16:%.*]] = load i32, i32* [[PDEST_ADDR_1_LCSSA]], align 4
; CHECK-NEXT: [[ADD35_PROL:%.*]] = add nsw i32 [[MUL34_PROL]], [[TMP49]] ; CHECK-NEXT: [[ADD35_PROL:%.*]] = add nsw i32 [[MUL34_PROL]], [[TMP16]]
; CHECK-NEXT: store i32 [[ADD35_PROL]], i32* [[PDEST_ADDR_1_LCSSA]], align 4 ; CHECK-NEXT: store i32 [[ADD35_PROL]], i32* [[PDEST_ADDR_1_LCSSA]], align 4
; CHECK-NEXT: [[INCDEC_PTR36_PROL:%.*]] = getelementptr inbounds i16, i16* [[PSRCA_ADDR_1_LCSSA]], i32 1 ; CHECK-NEXT: [[INCDEC_PTR36_PROL:%.*]] = getelementptr inbounds i16, i16* [[PSRCA_ADDR_1_LCSSA]], i32 1
; CHECK-NEXT: [[INCDEC_PTR37_PROL:%.*]] = getelementptr inbounds i16, i16* [[PSRCB_ADDR_1_LCSSA]], i32 1 ; CHECK-NEXT: [[INCDEC_PTR37_PROL:%.*]] = getelementptr inbounds i16, i16* [[PSRCB_ADDR_1_LCSSA]], i32 1
; CHECK-NEXT: [[INCDEC_PTR38_PROL:%.*]] = getelementptr inbounds i32, i32* [[PDEST_ADDR_1_LCSSA]], i32 1 ; CHECK-NEXT: [[INCDEC_PTR38_PROL:%.*]] = getelementptr inbounds i32, i32* [[PDEST_ADDR_1_LCSSA]], i32 1
; CHECK-NEXT: [[INC_PROL:%.*]] = add nuw i32 [[J_0_LCSSA]], 1 ; CHECK-NEXT: [[INC_PROL:%.*]] = add nuw i32 [[J_0_LCSSA]], 1
; CHECK-NEXT: [[PROL_ITER_SUB:%.*]] = sub i32 [[XTRAITER4]], 1 ; CHECK-NEXT: [[PROL_ITER_SUB:%.*]] = sub i32 [[XTRAITER]], 1
; CHECK-NEXT: [[PROL_ITER_CMP:%.*]] = icmp ne i32 [[PROL_ITER_SUB]], 0 ; CHECK-NEXT: [[PROL_ITER_CMP:%.*]] = icmp ne i32 [[PROL_ITER_SUB]], 0
; CHECK-NEXT: br i1 [[PROL_ITER_CMP]], label [[FOR_BODY29_PROL_1:%.*]], label [[FOR_BODY29_PROL_LOOPEXIT_UNR_LCSSA:%.*]] ; CHECK-NEXT: br i1 [[PROL_ITER_CMP]], label [[FOR_BODY29_PROL_1:%.*]], label [[FOR_BODY29_PROL_LOOPEXIT_UNR_LCSSA:%.*]]
; CHECK: for.body29.prol.loopexit.unr-lcssa: ; CHECK: for.body29.prol.loopexit.unr-lcssa:
; CHECK-NEXT: [[J_184_UNR_PH:%.*]] = phi i32 [ [[INC_PROL]], [[FOR_BODY29_PROL]] ], [ [[INC_PROL_1:%.*]], [[FOR_BODY29_PROL_1]] ], [ [[INC_PROL_2:%.*]], [[FOR_BODY29_PROL_2:%.*]] ] ; CHECK-NEXT: [[J_184_UNR_PH:%.*]] = phi i32 [ [[INC_PROL]], [[FOR_BODY29_PROL]] ], [ [[INC_PROL_1:%.*]], [[FOR_BODY29_PROL_1]] ], [ [[INC_PROL_2:%.*]], [[FOR_BODY29_PROL_2:%.*]] ]
; CHECK-NEXT: [[PDEST_ADDR_283_UNR_PH:%.*]] = phi i32* [ [[INCDEC_PTR38_PROL]], [[FOR_BODY29_PROL]] ], [ [[INCDEC_PTR38_PROL_1:%.*]], [[FOR_BODY29_PROL_1]] ], [ [[INCDEC_PTR38_PROL_2:%.*]], [[FOR_BODY29_PROL_2]] ] ; CHECK-NEXT: [[PDEST_ADDR_283_UNR_PH:%.*]] = phi i32* [ [[INCDEC_PTR38_PROL]], [[FOR_BODY29_PROL]] ], [ [[INCDEC_PTR38_PROL_1:%.*]], [[FOR_BODY29_PROL_1]] ], [ [[INCDEC_PTR38_PROL_2:%.*]], [[FOR_BODY29_PROL_2]] ]
; CHECK-NEXT: [[PSRCA_ADDR_282_UNR_PH:%.*]] = phi i16* [ [[INCDEC_PTR36_PROL]], [[FOR_BODY29_PROL]] ], [ [[INCDEC_PTR36_PROL_1:%.*]], [[FOR_BODY29_PROL_1]] ], [ [[INCDEC_PTR36_PROL_2:%.*]], [[FOR_BODY29_PROL_2]] ] ; CHECK-NEXT: [[PSRCA_ADDR_282_UNR_PH:%.*]] = phi i16* [ [[INCDEC_PTR36_PROL]], [[FOR_BODY29_PROL]] ], [ [[INCDEC_PTR36_PROL_1:%.*]], [[FOR_BODY29_PROL_1]] ], [ [[INCDEC_PTR36_PROL_2:%.*]], [[FOR_BODY29_PROL_2]] ]
; CHECK-NEXT: [[PSRCB_ADDR_281_UNR_PH:%.*]] = phi i16* [ [[INCDEC_PTR37_PROL]], [[FOR_BODY29_PROL]] ], [ [[INCDEC_PTR37_PROL_1:%.*]], [[FOR_BODY29_PROL_1]] ], [ [[INCDEC_PTR37_PROL_2:%.*]], [[FOR_BODY29_PROL_2]] ] ; CHECK-NEXT: [[PSRCB_ADDR_281_UNR_PH:%.*]] = phi i16* [ [[INCDEC_PTR37_PROL]], [[FOR_BODY29_PROL]] ], [ [[INCDEC_PTR37_PROL_1:%.*]], [[FOR_BODY29_PROL_1]] ], [ [[INCDEC_PTR37_PROL_2:%.*]], [[FOR_BODY29_PROL_2]] ]
; CHECK-NEXT: br label [[FOR_BODY29_PROL_LOOPEXIT]] ; CHECK-NEXT: br label [[FOR_BODY29_PROL_LOOPEXIT]]
; CHECK: for.body29.prol.loopexit: ; CHECK: for.body29.prol.loopexit:
; CHECK-NEXT: [[J_184_UNR:%.*]] = phi i32 [ [[J_0_LCSSA]], [[FOR_BODY29_PREHEADER]] ], [ [[J_184_UNR_PH]], [[FOR_BODY29_PROL_LOOPEXIT_UNR_LCSSA]] ] ; CHECK-NEXT: [[J_184_UNR:%.*]] = phi i32 [ [[J_0_LCSSA]], [[FOR_BODY29_PREHEADER]] ], [ [[J_184_UNR_PH]], [[FOR_BODY29_PROL_LOOPEXIT_UNR_LCSSA]] ]
; CHECK-NEXT: [[PDEST_ADDR_283_UNR:%.*]] = phi i32* [ [[PDEST_ADDR_1_LCSSA]], [[FOR_BODY29_PREHEADER]] ], [ [[PDEST_ADDR_283_UNR_PH]], [[FOR_BODY29_PROL_LOOPEXIT_UNR_LCSSA]] ] ; CHECK-NEXT: [[PDEST_ADDR_283_UNR:%.*]] = phi i32* [ [[PDEST_ADDR_1_LCSSA]], [[FOR_BODY29_PREHEADER]] ], [ [[PDEST_ADDR_283_UNR_PH]], [[FOR_BODY29_PROL_LOOPEXIT_UNR_LCSSA]] ]
; CHECK-NEXT: [[PSRCA_ADDR_282_UNR:%.*]] = phi i16* [ [[PSRCA_ADDR_1_LCSSA]], [[FOR_BODY29_PREHEADER]] ], [ [[PSRCA_ADDR_282_UNR_PH]], [[FOR_BODY29_PROL_LOOPEXIT_UNR_LCSSA]] ] ; CHECK-NEXT: [[PSRCA_ADDR_282_UNR:%.*]] = phi i16* [ [[PSRCA_ADDR_1_LCSSA]], [[FOR_BODY29_PREHEADER]] ], [ [[PSRCA_ADDR_282_UNR_PH]], [[FOR_BODY29_PROL_LOOPEXIT_UNR_LCSSA]] ]
; CHECK-NEXT: [[PSRCB_ADDR_281_UNR:%.*]] = phi i16* [ [[PSRCB_ADDR_1_LCSSA]], [[FOR_BODY29_PREHEADER]] ], [ [[PSRCB_ADDR_281_UNR_PH]], [[FOR_BODY29_PROL_LOOPEXIT_UNR_LCSSA]] ] ; CHECK-NEXT: [[PSRCB_ADDR_281_UNR:%.*]] = phi i16* [ [[PSRCB_ADDR_1_LCSSA]], [[FOR_BODY29_PREHEADER]] ], [ [[PSRCB_ADDR_281_UNR_PH]], [[FOR_BODY29_PROL_LOOPEXIT_UNR_LCSSA]] ]
; CHECK-NEXT: [[TMP50:%.*]] = icmp ult i32 [[TMP46]], 3 ; CHECK-NEXT: [[TMP17:%.*]] = icmp ult i32 [[TMP13]], 3
; CHECK-NEXT: br i1 [[TMP50]], label [[FOR_END40_LOOPEXIT:%.*]], label [[FOR_BODY29_PREHEADER_NEW:%.*]] ; CHECK-NEXT: br i1 [[TMP17]], label [[FOR_END40_LOOPEXIT:%.*]], label [[FOR_BODY29_PREHEADER_NEW:%.*]]
; CHECK: for.body29.preheader.new: ; CHECK: for.body29.preheader.new:
; CHECK-NEXT: br label [[FOR_BODY29:%.*]] ; CHECK-NEXT: br label [[FOR_BODY29:%.*]]
; CHECK: for.body29: ; CHECK: for.body29:
; CHECK-NEXT: [[J_184:%.*]] = phi i32 [ [[J_184_UNR]], [[FOR_BODY29_PREHEADER_NEW]] ], [ [[INC_3:%.*]], [[FOR_BODY29]] ] ; CHECK-NEXT: [[J_184:%.*]] = phi i32 [ [[J_184_UNR]], [[FOR_BODY29_PREHEADER_NEW]] ], [ [[INC_3:%.*]], [[FOR_BODY29]] ]
; CHECK-NEXT: [[PDEST_ADDR_283:%.*]] = phi i32* [ [[PDEST_ADDR_283_UNR]], [[FOR_BODY29_PREHEADER_NEW]] ], [ [[INCDEC_PTR38_3:%.*]], [[FOR_BODY29]] ] ; CHECK-NEXT: [[PDEST_ADDR_283:%.*]] = phi i32* [ [[PDEST_ADDR_283_UNR]], [[FOR_BODY29_PREHEADER_NEW]] ], [ [[INCDEC_PTR38_3:%.*]], [[FOR_BODY29]] ]
; CHECK-NEXT: [[PSRCA_ADDR_282:%.*]] = phi i16* [ [[PSRCA_ADDR_282_UNR]], [[FOR_BODY29_PREHEADER_NEW]] ], [ [[INCDEC_PTR36_3:%.*]], [[FOR_BODY29]] ] ; CHECK-NEXT: [[PSRCA_ADDR_282:%.*]] = phi i16* [ [[PSRCA_ADDR_282_UNR]], [[FOR_BODY29_PREHEADER_NEW]] ], [ [[INCDEC_PTR36_3:%.*]], [[FOR_BODY29]] ]
; CHECK-NEXT: [[PSRCB_ADDR_281:%.*]] = phi i16* [ [[PSRCB_ADDR_281_UNR]], [[FOR_BODY29_PREHEADER_NEW]] ], [ [[INCDEC_PTR37_3:%.*]], [[FOR_BODY29]] ] ; CHECK-NEXT: [[PSRCB_ADDR_281:%.*]] = phi i16* [ [[PSRCB_ADDR_281_UNR]], [[FOR_BODY29_PREHEADER_NEW]] ], [ [[INCDEC_PTR37_3:%.*]], [[FOR_BODY29]] ]
; CHECK-NEXT: [[ARRAYIDX30:%.*]] = getelementptr inbounds i16, i16* [[PSRCA_ADDR_282]], i32 [[J_184]] ; CHECK-NEXT: [[ARRAYIDX30:%.*]] = getelementptr inbounds i16, i16* [[PSRCA_ADDR_282]], i32 [[J_184]]
; CHECK-NEXT: [[TMP51:%.*]] = load i16, i16* [[ARRAYIDX30]], align 2 ; CHECK-NEXT: [[TMP18:%.*]] = load i16, i16* [[ARRAYIDX30]], align 2
; CHECK-NEXT: [[CONV31:%.*]] = sext i16 [[TMP51]] to i32 ; CHECK-NEXT: [[CONV31:%.*]] = sext i16 [[TMP18]] to i32
; CHECK-NEXT: [[ARRAYIDX32:%.*]] = getelementptr inbounds i16, i16* [[PSRCB_ADDR_281]], i32 [[J_184]] ; CHECK-NEXT: [[ARRAYIDX32:%.*]] = getelementptr inbounds i16, i16* [[PSRCB_ADDR_281]], i32 [[J_184]]
; CHECK-NEXT: [[TMP52:%.*]] = load i16, i16* [[ARRAYIDX32]], align 2 ; CHECK-NEXT: [[TMP19:%.*]] = load i16, i16* [[ARRAYIDX32]], align 2
; CHECK-NEXT: [[CONV33:%.*]] = sext i16 [[TMP52]] to i32 ; CHECK-NEXT: [[CONV33:%.*]] = sext i16 [[TMP19]] to i32
; CHECK-NEXT: [[MUL34:%.*]] = mul nsw i32 [[CONV33]], [[CONV31]] ; CHECK-NEXT: [[MUL34:%.*]] = mul nsw i32 [[CONV33]], [[CONV31]]
; CHECK-NEXT: [[TMP53:%.*]] = load i32, i32* [[PDEST_ADDR_283]], align 4 ; CHECK-NEXT: [[TMP20:%.*]] = load i32, i32* [[PDEST_ADDR_283]], align 4
; CHECK-NEXT: [[ADD35:%.*]] = add nsw i32 [[MUL34]], [[TMP53]] ; CHECK-NEXT: [[ADD35:%.*]] = add nsw i32 [[MUL34]], [[TMP20]]
; CHECK-NEXT: store i32 [[ADD35]], i32* [[PDEST_ADDR_283]], align 4 ; CHECK-NEXT: store i32 [[ADD35]], i32* [[PDEST_ADDR_283]], align 4
; CHECK-NEXT: [[INCDEC_PTR36:%.*]] = getelementptr inbounds i16, i16* [[PSRCA_ADDR_282]], i32 1 ; CHECK-NEXT: [[INCDEC_PTR36:%.*]] = getelementptr inbounds i16, i16* [[PSRCA_ADDR_282]], i32 1
; CHECK-NEXT: [[INCDEC_PTR37:%.*]] = getelementptr inbounds i16, i16* [[PSRCB_ADDR_281]], i32 1 ; CHECK-NEXT: [[INCDEC_PTR37:%.*]] = getelementptr inbounds i16, i16* [[PSRCB_ADDR_281]], i32 1
; CHECK-NEXT: [[INCDEC_PTR38:%.*]] = getelementptr inbounds i32, i32* [[PDEST_ADDR_283]], i32 1 ; CHECK-NEXT: [[INCDEC_PTR38:%.*]] = getelementptr inbounds i32, i32* [[PDEST_ADDR_283]], i32 1
; CHECK-NEXT: [[INC:%.*]] = add nuw i32 [[J_184]], 1 ; CHECK-NEXT: [[INC:%.*]] = add nuw i32 [[J_184]], 1
; CHECK-NEXT: [[ARRAYIDX30_1:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR36]], i32 [[INC]] ; CHECK-NEXT: [[ARRAYIDX30_1:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR36]], i32 [[INC]]
; CHECK-NEXT: [[TMP54:%.*]] = load i16, i16* [[ARRAYIDX30_1]], align 2 ; CHECK-NEXT: [[TMP21:%.*]] = load i16, i16* [[ARRAYIDX30_1]], align 2
; CHECK-NEXT: [[CONV31_1:%.*]] = sext i16 [[TMP54]] to i32 ; CHECK-NEXT: [[CONV31_1:%.*]] = sext i16 [[TMP21]] to i32
; CHECK-NEXT: [[ARRAYIDX32_1:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR37]], i32 [[INC]] ; CHECK-NEXT: [[ARRAYIDX32_1:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR37]], i32 [[INC]]
; CHECK-NEXT: [[TMP55:%.*]] = load i16, i16* [[ARRAYIDX32_1]], align 2 ; CHECK-NEXT: [[TMP22:%.*]] = load i16, i16* [[ARRAYIDX32_1]], align 2
; CHECK-NEXT: [[CONV33_1:%.*]] = sext i16 [[TMP55]] to i32 ; CHECK-NEXT: [[CONV33_1:%.*]] = sext i16 [[TMP22]] to i32
; CHECK-NEXT: [[MUL34_1:%.*]] = mul nsw i32 [[CONV33_1]], [[CONV31_1]] ; CHECK-NEXT: [[MUL34_1:%.*]] = mul nsw i32 [[CONV33_1]], [[CONV31_1]]
; CHECK-NEXT: [[TMP56:%.*]] = load i32, i32* [[INCDEC_PTR38]], align 4 ; CHECK-NEXT: [[TMP23:%.*]] = load i32, i32* [[INCDEC_PTR38]], align 4
; CHECK-NEXT: [[ADD35_1:%.*]] = add nsw i32 [[MUL34_1]], [[TMP56]] ; CHECK-NEXT: [[ADD35_1:%.*]] = add nsw i32 [[MUL34_1]], [[TMP23]]
; CHECK-NEXT: store i32 [[ADD35_1]], i32* [[INCDEC_PTR38]], align 4 ; CHECK-NEXT: store i32 [[ADD35_1]], i32* [[INCDEC_PTR38]], align 4
; CHECK-NEXT: [[INCDEC_PTR36_1:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR36]], i32 1 ; CHECK-NEXT: [[INCDEC_PTR36_1:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR36]], i32 1
; CHECK-NEXT: [[INCDEC_PTR37_1:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR37]], i32 1 ; CHECK-NEXT: [[INCDEC_PTR37_1:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR37]], i32 1
; CHECK-NEXT: [[INCDEC_PTR38_1:%.*]] = getelementptr inbounds i32, i32* [[INCDEC_PTR38]], i32 1 ; CHECK-NEXT: [[INCDEC_PTR38_1:%.*]] = getelementptr inbounds i32, i32* [[INCDEC_PTR38]], i32 1
; CHECK-NEXT: [[INC_1:%.*]] = add nuw i32 [[INC]], 1 ; CHECK-NEXT: [[INC_1:%.*]] = add nuw i32 [[INC]], 1
; CHECK-NEXT: [[ARRAYIDX30_2:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR36_1]], i32 [[INC_1]] ; CHECK-NEXT: [[ARRAYIDX30_2:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR36_1]], i32 [[INC_1]]
; CHECK-NEXT: [[TMP57:%.*]] = load i16, i16* [[ARRAYIDX30_2]], align 2 ; CHECK-NEXT: [[TMP24:%.*]] = load i16, i16* [[ARRAYIDX30_2]], align 2
; CHECK-NEXT: [[CONV31_2:%.*]] = sext i16 [[TMP57]] to i32 ; CHECK-NEXT: [[CONV31_2:%.*]] = sext i16 [[TMP24]] to i32
; CHECK-NEXT: [[ARRAYIDX32_2:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR37_1]], i32 [[INC_1]] ; CHECK-NEXT: [[ARRAYIDX32_2:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR37_1]], i32 [[INC_1]]
; CHECK-NEXT: [[TMP58:%.*]] = load i16, i16* [[ARRAYIDX32_2]], align 2 ; CHECK-NEXT: [[TMP25:%.*]] = load i16, i16* [[ARRAYIDX32_2]], align 2
; CHECK-NEXT: [[CONV33_2:%.*]] = sext i16 [[TMP58]] to i32 ; CHECK-NEXT: [[CONV33_2:%.*]] = sext i16 [[TMP25]] to i32
; CHECK-NEXT: [[MUL34_2:%.*]] = mul nsw i32 [[CONV33_2]], [[CONV31_2]] ; CHECK-NEXT: [[MUL34_2:%.*]] = mul nsw i32 [[CONV33_2]], [[CONV31_2]]
; CHECK-NEXT: [[TMP59:%.*]] = load i32, i32* [[INCDEC_PTR38_1]], align 4 ; CHECK-NEXT: [[TMP26:%.*]] = load i32, i32* [[INCDEC_PTR38_1]], align 4
; CHECK-NEXT: [[ADD35_2:%.*]] = add nsw i32 [[MUL34_2]], [[TMP59]] ; CHECK-NEXT: [[ADD35_2:%.*]] = add nsw i32 [[MUL34_2]], [[TMP26]]
; CHECK-NEXT: store i32 [[ADD35_2]], i32* [[INCDEC_PTR38_1]], align 4 ; CHECK-NEXT: store i32 [[ADD35_2]], i32* [[INCDEC_PTR38_1]], align 4
; CHECK-NEXT: [[INCDEC_PTR36_2:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR36_1]], i32 1 ; CHECK-NEXT: [[INCDEC_PTR36_2:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR36_1]], i32 1
; CHECK-NEXT: [[INCDEC_PTR37_2:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR37_1]], i32 1 ; CHECK-NEXT: [[INCDEC_PTR37_2:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR37_1]], i32 1
; CHECK-NEXT: [[INCDEC_PTR38_2:%.*]] = getelementptr inbounds i32, i32* [[INCDEC_PTR38_1]], i32 1 ; CHECK-NEXT: [[INCDEC_PTR38_2:%.*]] = getelementptr inbounds i32, i32* [[INCDEC_PTR38_1]], i32 1
; CHECK-NEXT: [[INC_2:%.*]] = add nuw i32 [[INC_1]], 1 ; CHECK-NEXT: [[INC_2:%.*]] = add nuw i32 [[INC_1]], 1
; CHECK-NEXT: [[ARRAYIDX30_3:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR36_2]], i32 [[INC_2]] ; CHECK-NEXT: [[ARRAYIDX30_3:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR36_2]], i32 [[INC_2]]
; CHECK-NEXT: [[TMP60:%.*]] = load i16, i16* [[ARRAYIDX30_3]], align 2 ; CHECK-NEXT: [[TMP27:%.*]] = load i16, i16* [[ARRAYIDX30_3]], align 2
; CHECK-NEXT: [[CONV31_3:%.*]] = sext i16 [[TMP60]] to i32 ; CHECK-NEXT: [[CONV31_3:%.*]] = sext i16 [[TMP27]] to i32
; CHECK-NEXT: [[ARRAYIDX32_3:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR37_2]], i32 [[INC_2]] ; CHECK-NEXT: [[ARRAYIDX32_3:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR37_2]], i32 [[INC_2]]
; CHECK-NEXT: [[TMP61:%.*]] = load i16, i16* [[ARRAYIDX32_3]], align 2 ; CHECK-NEXT: [[TMP28:%.*]] = load i16, i16* [[ARRAYIDX32_3]], align 2
; CHECK-NEXT: [[CONV33_3:%.*]] = sext i16 [[TMP61]] to i32 ; CHECK-NEXT: [[CONV33_3:%.*]] = sext i16 [[TMP28]] to i32
; CHECK-NEXT: [[MUL34_3:%.*]] = mul nsw i32 [[CONV33_3]], [[CONV31_3]] ; CHECK-NEXT: [[MUL34_3:%.*]] = mul nsw i32 [[CONV33_3]], [[CONV31_3]]
; CHECK-NEXT: [[TMP62:%.*]] = load i32, i32* [[INCDEC_PTR38_2]], align 4 ; CHECK-NEXT: [[TMP29:%.*]] = load i32, i32* [[INCDEC_PTR38_2]], align 4
; CHECK-NEXT: [[ADD35_3:%.*]] = add nsw i32 [[MUL34_3]], [[TMP62]] ; CHECK-NEXT: [[ADD35_3:%.*]] = add nsw i32 [[MUL34_3]], [[TMP29]]
; CHECK-NEXT: store i32 [[ADD35_3]], i32* [[INCDEC_PTR38_2]], align 4 ; CHECK-NEXT: store i32 [[ADD35_3]], i32* [[INCDEC_PTR38_2]], align 4
; CHECK-NEXT: [[INCDEC_PTR36_3]] = getelementptr inbounds i16, i16* [[INCDEC_PTR36_2]], i32 1 ; CHECK-NEXT: [[INCDEC_PTR36_3]] = getelementptr inbounds i16, i16* [[INCDEC_PTR36_2]], i32 1
; CHECK-NEXT: [[INCDEC_PTR37_3]] = getelementptr inbounds i16, i16* [[INCDEC_PTR37_2]], i32 1 ; CHECK-NEXT: [[INCDEC_PTR37_3]] = getelementptr inbounds i16, i16* [[INCDEC_PTR37_2]], i32 1
; CHECK-NEXT: [[INCDEC_PTR38_3]] = getelementptr inbounds i32, i32* [[INCDEC_PTR38_2]], i32 1 ; CHECK-NEXT: [[INCDEC_PTR38_3]] = getelementptr inbounds i32, i32* [[INCDEC_PTR38_2]], i32 1
; CHECK-NEXT: [[INC_3]] = add nuw i32 [[INC_2]], 1 ; CHECK-NEXT: [[INC_3]] = add nuw i32 [[INC_2]], 1
; CHECK-NEXT: [[EXITCOND_3:%.*]] = icmp eq i32 [[INC_3]], [[ADD25]] ; CHECK-NEXT: [[EXITCOND_3:%.*]] = icmp eq i32 [[INC_3]], [[ADD25]]
; CHECK-NEXT: br i1 [[EXITCOND_3]], label [[FOR_END40_LOOPEXIT_UNR_LCSSA:%.*]], label [[FOR_BODY29]] ; CHECK-NEXT: br i1 [[EXITCOND_3]], label [[FOR_END40_LOOPEXIT_UNR_LCSSA:%.*]], label [[FOR_BODY29]]
; CHECK: for.end40.loopexit.unr-lcssa: ; CHECK: for.end40.loopexit.unr-lcssa:
; CHECK-NEXT: br label [[FOR_END40_LOOPEXIT]] ; CHECK-NEXT: br label [[FOR_END40_LOOPEXIT]]
; CHECK: for.end40.loopexit: ; CHECK: for.end40.loopexit:
; CHECK-NEXT: [[SCEVGEP93:%.*]] = getelementptr i16, i16* [[PSRCB_ADDR_1_LCSSA]], i32 [[TMP43]] ; CHECK-NEXT: [[SCEVGEP93:%.*]] = getelementptr i16, i16* [[PSRCB_ADDR_1_LCSSA]], i32 [[TMP10]]
; CHECK-NEXT: [[SCEVGEP:%.*]] = getelementptr i16, i16* [[PSRCA_ADDR_1_LCSSA]], i32 [[TMP43]] ; CHECK-NEXT: [[SCEVGEP:%.*]] = getelementptr i16, i16* [[PSRCA_ADDR_1_LCSSA]], i32 [[TMP10]]
; CHECK-NEXT: [[SCEVGEP94:%.*]] = getelementptr i32, i32* [[PDEST_ADDR_1_LCSSA]], i32 [[TMP43]] ; CHECK-NEXT: [[SCEVGEP94:%.*]] = getelementptr i32, i32* [[PDEST_ADDR_1_LCSSA]], i32 [[TMP10]]
; CHECK-NEXT: br label [[FOR_END40]] ; CHECK-NEXT: br label [[FOR_END40]]
; CHECK: for.end40: ; CHECK: for.end40:
; CHECK-NEXT: [[PSRCB_ADDR_2_LCSSA]] = phi i16* [ [[PSRCB_ADDR_1_LCSSA]], [[FOR_END]] ], [ [[SCEVGEP93]], [[FOR_END40_LOOPEXIT]] ] ; CHECK-NEXT: [[PSRCB_ADDR_2_LCSSA]] = phi i16* [ [[PSRCB_ADDR_1_LCSSA]], [[FOR_END]] ], [ [[SCEVGEP93]], [[FOR_END40_LOOPEXIT]] ]
; CHECK-NEXT: [[PSRCA_ADDR_2_LCSSA]] = phi i16* [ [[PSRCA_ADDR_1_LCSSA]], [[FOR_END]] ], [ [[SCEVGEP]], [[FOR_END40_LOOPEXIT]] ] ; CHECK-NEXT: [[PSRCA_ADDR_2_LCSSA]] = phi i16* [ [[PSRCA_ADDR_1_LCSSA]], [[FOR_END]] ], [ [[SCEVGEP]], [[FOR_END40_LOOPEXIT]] ]
; CHECK-NEXT: [[PDEST_ADDR_2_LCSSA]] = phi i32* [ [[PDEST_ADDR_1_LCSSA]], [[FOR_END]] ], [ [[SCEVGEP94]], [[FOR_END40_LOOPEXIT]] ] ; CHECK-NEXT: [[PDEST_ADDR_2_LCSSA]] = phi i32* [ [[PDEST_ADDR_1_LCSSA]], [[FOR_END]] ], [ [[SCEVGEP94]], [[FOR_END40_LOOPEXIT]] ]
; CHECK-NEXT: [[INC42]] = add nuw i32 [[I_092]], 1 ; CHECK-NEXT: [[INC42]] = add nuw i32 [[I_092]], 1
; CHECK-NEXT: [[EXITCOND95:%.*]] = icmp eq i32 [[INC42]], [[BLKCNT]] ; CHECK-NEXT: [[EXITCOND95:%.*]] = icmp eq i32 [[INC42]], [[BLKCNT]]
; CHECK-NEXT: br i1 [[EXITCOND95]], label [[FOR_COND_CLEANUP_LOOPEXIT:%.*]], label [[FOR_BODY]] ; CHECK-NEXT: br i1 [[EXITCOND95]], label [[FOR_COND_CLEANUP_LOOPEXIT:%.*]], label [[FOR_BODY]]
; CHECK: for.body3.epil.1:
; CHECK-NEXT: [[TMP63:%.*]] = load i16, i16* [[ADD_PTR_EPIL]], align 2
; CHECK-NEXT: [[CONV_EPIL_1:%.*]] = sext i16 [[TMP63]] to i32
; CHECK-NEXT: [[TMP64:%.*]] = load i16, i16* [[ADD_PTR23_EPIL]], align 2
; CHECK-NEXT: [[CONV5_EPIL_1:%.*]] = sext i16 [[TMP64]] to i32
; CHECK-NEXT: [[MUL_EPIL_1:%.*]] = mul nsw i32 [[CONV5_EPIL_1]], [[CONV_EPIL_1]]
; CHECK-NEXT: [[ARRAYIDX6_EPIL_1:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR_EPIL]], i32 1
; CHECK-NEXT: [[TMP65:%.*]] = load i16, i16* [[ARRAYIDX6_EPIL_1]], align 2
; CHECK-NEXT: [[CONV7_EPIL_1:%.*]] = sext i16 [[TMP65]] to i32
; CHECK-NEXT: [[ARRAYIDX8_EPIL_1:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR23_EPIL]], i32 1
; CHECK-NEXT: [[TMP66:%.*]] = load i16, i16* [[ARRAYIDX8_EPIL_1]], align 2
; CHECK-NEXT: [[CONV9_EPIL_1:%.*]] = sext i16 [[TMP66]] to i32
; CHECK-NEXT: [[MUL10_EPIL_1:%.*]] = mul nsw i32 [[CONV9_EPIL_1]], [[CONV7_EPIL_1]]
; CHECK-NEXT: [[ARRAYIDX11_EPIL_1:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR_EPIL]], i32 2
; CHECK-NEXT: [[TMP67:%.*]] = load i16, i16* [[ARRAYIDX11_EPIL_1]], align 2
; CHECK-NEXT: [[CONV12_EPIL_1:%.*]] = sext i16 [[TMP67]] to i32
; CHECK-NEXT: [[ARRAYIDX13_EPIL_1:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR23_EPIL]], i32 3
; CHECK-NEXT: [[TMP68:%.*]] = load i16, i16* [[ARRAYIDX13_EPIL_1]], align 2
; CHECK-NEXT: [[CONV14_EPIL_1:%.*]] = sext i16 [[TMP68]] to i32
; CHECK-NEXT: [[MUL15_EPIL_1:%.*]] = mul nsw i32 [[CONV14_EPIL_1]], [[CONV12_EPIL_1]]
; CHECK-NEXT: [[ARRAYIDX17_EPIL_1:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR_EPIL]], i32 3
; CHECK-NEXT: [[TMP69:%.*]] = load i16, i16* [[ARRAYIDX17_EPIL_1]], align 2
; CHECK-NEXT: [[CONV18_EPIL_1:%.*]] = sext i16 [[TMP69]] to i32
; CHECK-NEXT: [[ADD21_EPIL_1:%.*]] = add i32 [[MUL10_EPIL_1]], [[MUL_EPIL_1]]
; CHECK-NEXT: [[ADD_EPIL_1:%.*]] = add i32 [[ADD21_EPIL_1]], [[CONV14_EPIL_1]]
; CHECK-NEXT: [[ADD16_EPIL_1:%.*]] = add i32 [[ADD_EPIL_1]], [[MUL15_EPIL_1]]
; CHECK-NEXT: [[ADD22_EPIL_1:%.*]] = add i32 [[ADD16_EPIL_1]], [[CONV18_EPIL_1]]
; CHECK-NEXT: store i32 [[ADD22_EPIL_1]], i32* [[INCDEC_PTR_EPIL]], align 4
; CHECK-NEXT: [[ADD_PTR_EPIL_1]] = getelementptr inbounds i16, i16* [[ADD_PTR_EPIL]], i32 4
; CHECK-NEXT: [[ADD_PTR23_EPIL_1]] = getelementptr inbounds i16, i16* [[ADD_PTR23_EPIL]], i32 4
; CHECK-NEXT: [[INCDEC_PTR_EPIL_1]] = getelementptr inbounds i32, i32* [[INCDEC_PTR_EPIL]], i32 1
; CHECK-NEXT: [[ADD24_EPIL_1:%.*]] = add nuw nsw i32 [[ADD24_EPIL]], 4
; CHECK-NEXT: [[EPIL_ITER_SUB_1:%.*]] = sub i32 [[EPIL_ITER_SUB]], 1
; CHECK-NEXT: [[EPIL_ITER_CMP_1:%.*]] = icmp ne i32 [[EPIL_ITER_SUB_1]], 0
; CHECK-NEXT: br i1 [[EPIL_ITER_CMP_1]], label [[FOR_BODY3_EPIL_2]], label [[FOR_END_LOOPEXIT_EPILOG_LCSSA]]
; CHECK: for.body3.epil.2:
; CHECK-NEXT: [[TMP70:%.*]] = load i16, i16* [[ADD_PTR_EPIL_1]], align 2
; CHECK-NEXT: [[CONV_EPIL_2:%.*]] = sext i16 [[TMP70]] to i32
; CHECK-NEXT: [[TMP71:%.*]] = load i16, i16* [[ADD_PTR23_EPIL_1]], align 2
; CHECK-NEXT: [[CONV5_EPIL_2:%.*]] = sext i16 [[TMP71]] to i32
; CHECK-NEXT: [[MUL_EPIL_2:%.*]] = mul nsw i32 [[CONV5_EPIL_2]], [[CONV_EPIL_2]]
; CHECK-NEXT: [[ARRAYIDX6_EPIL_2:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR_EPIL_1]], i32 1
; CHECK-NEXT: [[TMP72:%.*]] = load i16, i16* [[ARRAYIDX6_EPIL_2]], align 2
; CHECK-NEXT: [[CONV7_EPIL_2:%.*]] = sext i16 [[TMP72]] to i32
; CHECK-NEXT: [[ARRAYIDX8_EPIL_2:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR23_EPIL_1]], i32 1
; CHECK-NEXT: [[TMP73:%.*]] = load i16, i16* [[ARRAYIDX8_EPIL_2]], align 2
; CHECK-NEXT: [[CONV9_EPIL_2:%.*]] = sext i16 [[TMP73]] to i32
; CHECK-NEXT: [[MUL10_EPIL_2:%.*]] = mul nsw i32 [[CONV9_EPIL_2]], [[CONV7_EPIL_2]]
; CHECK-NEXT: [[ARRAYIDX11_EPIL_2:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR_EPIL_1]], i32 2
; CHECK-NEXT: [[TMP74:%.*]] = load i16, i16* [[ARRAYIDX11_EPIL_2]], align 2
; CHECK-NEXT: [[CONV12_EPIL_2:%.*]] = sext i16 [[TMP74]] to i32
; CHECK-NEXT: [[ARRAYIDX13_EPIL_2:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR23_EPIL_1]], i32 3
; CHECK-NEXT: [[TMP75:%.*]] = load i16, i16* [[ARRAYIDX13_EPIL_2]], align 2
; CHECK-NEXT: [[CONV14_EPIL_2:%.*]] = sext i16 [[TMP75]] to i32
; CHECK-NEXT: [[MUL15_EPIL_2:%.*]] = mul nsw i32 [[CONV14_EPIL_2]], [[CONV12_EPIL_2]]
; CHECK-NEXT: [[ARRAYIDX17_EPIL_2:%.*]] = getelementptr inbounds i16, i16* [[ADD_PTR_EPIL_1]], i32 3
; CHECK-NEXT: [[TMP76:%.*]] = load i16, i16* [[ARRAYIDX17_EPIL_2]], align 2
; CHECK-NEXT: [[CONV18_EPIL_2:%.*]] = sext i16 [[TMP76]] to i32
; CHECK-NEXT: [[ADD21_EPIL_2:%.*]] = add i32 [[MUL10_EPIL_2]], [[MUL_EPIL_2]]
; CHECK-NEXT: [[ADD_EPIL_2:%.*]] = add i32 [[ADD21_EPIL_2]], [[CONV14_EPIL_2]]
; CHECK-NEXT: [[ADD16_EPIL_2:%.*]] = add i32 [[ADD_EPIL_2]], [[MUL15_EPIL_2]]
; CHECK-NEXT: [[ADD22_EPIL_2:%.*]] = add i32 [[ADD16_EPIL_2]], [[CONV18_EPIL_2]]
; CHECK-NEXT: store i32 [[ADD22_EPIL_2]], i32* [[INCDEC_PTR_EPIL_1]], align 4
; CHECK-NEXT: [[ADD_PTR_EPIL_2]] = getelementptr inbounds i16, i16* [[ADD_PTR_EPIL_1]], i32 4
; CHECK-NEXT: [[ADD_PTR23_EPIL_2]] = getelementptr inbounds i16, i16* [[ADD_PTR23_EPIL_1]], i32 4
; CHECK-NEXT: [[INCDEC_PTR_EPIL_2]] = getelementptr inbounds i32, i32* [[INCDEC_PTR_EPIL_1]], i32 1
; CHECK-NEXT: [[ADD24_EPIL_2:%.*]] = add nuw nsw i32 [[ADD24_EPIL_1]], 4
; CHECK-NEXT: [[EPIL_ITER_SUB_2:%.*]] = sub i32 [[EPIL_ITER_SUB_1]], 1
; CHECK-NEXT: br label [[FOR_END_LOOPEXIT_EPILOG_LCSSA]]
; CHECK: for.body29.prol.1: ; CHECK: for.body29.prol.1:
; CHECK-NEXT: [[ARRAYIDX30_PROL_1:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR36_PROL]], i32 [[INC_PROL]] ; CHECK-NEXT: [[ARRAYIDX30_PROL_1:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR36_PROL]], i32 [[INC_PROL]]
; CHECK-NEXT: [[TMP77:%.*]] = load i16, i16* [[ARRAYIDX30_PROL_1]], align 2 ; CHECK-NEXT: [[TMP30:%.*]] = load i16, i16* [[ARRAYIDX30_PROL_1]], align 2
; CHECK-NEXT: [[CONV31_PROL_1:%.*]] = sext i16 [[TMP77]] to i32 ; CHECK-NEXT: [[CONV31_PROL_1:%.*]] = sext i16 [[TMP30]] to i32
; CHECK-NEXT: [[ARRAYIDX32_PROL_1:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR37_PROL]], i32 [[INC_PROL]] ; CHECK-NEXT: [[ARRAYIDX32_PROL_1:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR37_PROL]], i32 [[INC_PROL]]
; CHECK-NEXT: [[TMP78:%.*]] = load i16, i16* [[ARRAYIDX32_PROL_1]], align 2 ; CHECK-NEXT: [[TMP31:%.*]] = load i16, i16* [[ARRAYIDX32_PROL_1]], align 2
; CHECK-NEXT: [[CONV33_PROL_1:%.*]] = sext i16 [[TMP78]] to i32 ; CHECK-NEXT: [[CONV33_PROL_1:%.*]] = sext i16 [[TMP31]] to i32
; CHECK-NEXT: [[MUL34_PROL_1:%.*]] = mul nsw i32 [[CONV33_PROL_1]], [[CONV31_PROL_1]] ; CHECK-NEXT: [[MUL34_PROL_1:%.*]] = mul nsw i32 [[CONV33_PROL_1]], [[CONV31_PROL_1]]
; CHECK-NEXT: [[TMP79:%.*]] = load i32, i32* [[INCDEC_PTR38_PROL]], align 4 ; CHECK-NEXT: [[TMP32:%.*]] = load i32, i32* [[INCDEC_PTR38_PROL]], align 4
; CHECK-NEXT: [[ADD35_PROL_1:%.*]] = add nsw i32 [[MUL34_PROL_1]], [[TMP79]] ; CHECK-NEXT: [[ADD35_PROL_1:%.*]] = add nsw i32 [[MUL34_PROL_1]], [[TMP32]]
; CHECK-NEXT: store i32 [[ADD35_PROL_1]], i32* [[INCDEC_PTR38_PROL]], align 4 ; CHECK-NEXT: store i32 [[ADD35_PROL_1]], i32* [[INCDEC_PTR38_PROL]], align 4
; CHECK-NEXT: [[INCDEC_PTR36_PROL_1]] = getelementptr inbounds i16, i16* [[INCDEC_PTR36_PROL]], i32 1 ; CHECK-NEXT: [[INCDEC_PTR36_PROL_1]] = getelementptr inbounds i16, i16* [[INCDEC_PTR36_PROL]], i32 1
; CHECK-NEXT: [[INCDEC_PTR37_PROL_1]] = getelementptr inbounds i16, i16* [[INCDEC_PTR37_PROL]], i32 1 ; CHECK-NEXT: [[INCDEC_PTR37_PROL_1]] = getelementptr inbounds i16, i16* [[INCDEC_PTR37_PROL]], i32 1
; CHECK-NEXT: [[INCDEC_PTR38_PROL_1]] = getelementptr inbounds i32, i32* [[INCDEC_PTR38_PROL]], i32 1 ; CHECK-NEXT: [[INCDEC_PTR38_PROL_1]] = getelementptr inbounds i32, i32* [[INCDEC_PTR38_PROL]], i32 1
; CHECK-NEXT: [[INC_PROL_1]] = add nuw i32 [[INC_PROL]], 1 ; CHECK-NEXT: [[INC_PROL_1]] = add nuw i32 [[INC_PROL]], 1
; CHECK-NEXT: [[PROL_ITER_SUB_1:%.*]] = sub i32 [[PROL_ITER_SUB]], 1 ; CHECK-NEXT: [[PROL_ITER_SUB_1:%.*]] = sub i32 [[PROL_ITER_SUB]], 1
; CHECK-NEXT: [[PROL_ITER_CMP_1:%.*]] = icmp ne i32 [[PROL_ITER_SUB_1]], 0 ; CHECK-NEXT: [[PROL_ITER_CMP_1:%.*]] = icmp ne i32 [[PROL_ITER_SUB_1]], 0
; CHECK-NEXT: br i1 [[PROL_ITER_CMP_1]], label [[FOR_BODY29_PROL_2]], label [[FOR_BODY29_PROL_LOOPEXIT_UNR_LCSSA]] ; CHECK-NEXT: br i1 [[PROL_ITER_CMP_1]], label [[FOR_BODY29_PROL_2]], label [[FOR_BODY29_PROL_LOOPEXIT_UNR_LCSSA]]
; CHECK: for.body29.prol.2: ; CHECK: for.body29.prol.2:
; CHECK-NEXT: [[ARRAYIDX30_PROL_2:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR36_PROL_1]], i32 [[INC_PROL_1]] ; CHECK-NEXT: [[ARRAYIDX30_PROL_2:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR36_PROL_1]], i32 [[INC_PROL_1]]
; CHECK-NEXT: [[TMP80:%.*]] = load i16, i16* [[ARRAYIDX30_PROL_2]], align 2 ; CHECK-NEXT: [[TMP33:%.*]] = load i16, i16* [[ARRAYIDX30_PROL_2]], align 2
; CHECK-NEXT: [[CONV31_PROL_2:%.*]] = sext i16 [[TMP80]] to i32 ; CHECK-NEXT: [[CONV31_PROL_2:%.*]] = sext i16 [[TMP33]] to i32
; CHECK-NEXT: [[ARRAYIDX32_PROL_2:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR37_PROL_1]], i32 [[INC_PROL_1]] ; CHECK-NEXT: [[ARRAYIDX32_PROL_2:%.*]] = getelementptr inbounds i16, i16* [[INCDEC_PTR37_PROL_1]], i32 [[INC_PROL_1]]
; CHECK-NEXT: [[TMP81:%.*]] = load i16, i16* [[ARRAYIDX32_PROL_2]], align 2 ; CHECK-NEXT: [[TMP34:%.*]] = load i16, i16* [[ARRAYIDX32_PROL_2]], align 2
; CHECK-NEXT: [[CONV33_PROL_2:%.*]] = sext i16 [[TMP81]] to i32 ; CHECK-NEXT: [[CONV33_PROL_2:%.*]] = sext i16 [[TMP34]] to i32
; CHECK-NEXT: [[MUL34_PROL_2:%.*]] = mul nsw i32 [[CONV33_PROL_2]], [[CONV31_PROL_2]] ; CHECK-NEXT: [[MUL34_PROL_2:%.*]] = mul nsw i32 [[CONV33_PROL_2]], [[CONV31_PROL_2]]
; CHECK-NEXT: [[TMP82:%.*]] = load i32, i32* [[INCDEC_PTR38_PROL_1]], align 4 ; CHECK-NEXT: [[TMP35:%.*]] = load i32, i32* [[INCDEC_PTR38_PROL_1]], align 4
; CHECK-NEXT: [[ADD35_PROL_2:%.*]] = add nsw i32 [[MUL34_PROL_2]], [[TMP82]] ; CHECK-NEXT: [[ADD35_PROL_2:%.*]] = add nsw i32 [[MUL34_PROL_2]], [[TMP35]]
; CHECK-NEXT: store i32 [[ADD35_PROL_2]], i32* [[INCDEC_PTR38_PROL_1]], align 4 ; CHECK-NEXT: store i32 [[ADD35_PROL_2]], i32* [[INCDEC_PTR38_PROL_1]], align 4
; CHECK-NEXT: [[INCDEC_PTR36_PROL_2]] = getelementptr inbounds i16, i16* [[INCDEC_PTR36_PROL_1]], i32 1 ; CHECK-NEXT: [[INCDEC_PTR36_PROL_2]] = getelementptr inbounds i16, i16* [[INCDEC_PTR36_PROL_1]], i32 1
; CHECK-NEXT: [[INCDEC_PTR37_PROL_2]] = getelementptr inbounds i16, i16* [[INCDEC_PTR37_PROL_1]], i32 1 ; CHECK-NEXT: [[INCDEC_PTR37_PROL_2]] = getelementptr inbounds i16, i16* [[INCDEC_PTR37_PROL_1]], i32 1
; CHECK-NEXT: [[INCDEC_PTR38_PROL_2]] = getelementptr inbounds i32, i32* [[INCDEC_PTR38_PROL_1]], i32 1 ; CHECK-NEXT: [[INCDEC_PTR38_PROL_2]] = getelementptr inbounds i32, i32* [[INCDEC_PTR38_PROL_1]], i32 1
; CHECK-NEXT: [[INC_PROL_2]] = add nuw i32 [[INC_PROL_1]], 1 ; CHECK-NEXT: [[INC_PROL_2]] = add nuw i32 [[INC_PROL_1]], 1
; CHECK-NEXT: [[PROL_ITER_SUB_2:%.*]] = sub i32 [[PROL_ITER_SUB_1]], 1 ; CHECK-NEXT: [[PROL_ITER_SUB_2:%.*]] = sub i32 [[PROL_ITER_SUB_1]], 1
; CHECK-NEXT: br label [[FOR_BODY29_PROL_LOOPEXIT_UNR_LCSSA]] ; CHECK-NEXT: br label [[FOR_BODY29_PROL_LOOPEXIT_UNR_LCSSA]]
; ;
▲ Show 20 Lines • Show All 107 Lines • Show Last 20 Lines

llvm/test/Transforms/IndVarSimplify/eliminate-trunc.ll

Show First 20 Lines • Show All 60 Lines • ▼ Show 20 Linesexit:
ret void ret void
} }
; Max value at which we can eliminate trunc: SINT_MAX - 1. ; Max value at which we can eliminate trunc: SINT_MAX - 1.
define void @test_02(i32 %n) { define void @test_02(i32 %n) {
; ;
; CHECK-LABEL: @test_02( ; CHECK-LABEL: @test_02(
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: [[TMP0:%.*]] = icmp sgt i32 [[N:%.*]], 2147483646 ; CHECK-NEXT: [[SEXT:%.*]] = sext i32 [[N:%.*]] to i64
; CHECK-NEXT: [[SMAX:%.*]] = select i1 [[TMP0]], i32 [[N]], i32 2147483646
; CHECK-NEXT: [[TMP1:%.*]] = add nuw i32 [[SMAX]], 1
; CHECK-NEXT: [[WIDE_TRIP_COUNT:%.*]] = zext i32 [[TMP1]] to i64
; CHECK-NEXT: br label [[LOOP:%.*]] ; CHECK-NEXT: br label [[LOOP:%.*]]
; CHECK: loop: ; CHECK: loop:
; CHECK-NEXT: [[IV:%.*]] = phi i64 [ 2147483646, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LOOP]] ] ; CHECK-NEXT: [[IV:%.*]] = phi i64 [ 2147483646, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LOOP]] ]
; CHECK-NEXT: [[IV_NEXT]] = add nuw nsw i64 [[IV]], 1 ; CHECK-NEXT: [[IV_NEXT]] = add nuw nsw i64 [[IV]], 1
; CHECK-NEXT: [[EXITCOND:%.*]] = icmp ne i64 [[IV_NEXT]], [[WIDE_TRIP_COUNT]] ; CHECK-NEXT: [[TMP0:%.*]] = icmp slt i64 [[IV]], [[SEXT]]
; CHECK-NEXT: br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]] ; CHECK-NEXT: br i1 [[TMP0]], label [[LOOP]], label [[EXIT:%.*]]
; CHECK: exit: ; CHECK: exit:
; CHECK-NEXT: ret void ; CHECK-NEXT: ret void
; ;
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 2147483646, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 2147483646, %entry ], [ %iv.next, %loop ]
%iv.next = add i64 %iv, 1 %iv.next = add i64 %iv, 1
Show All 27 Linesexit:
ret void ret void
} }
; Minimum value at which we can apply the transform: SINT_MIN + 1. ; Minimum value at which we can apply the transform: SINT_MIN + 1.
define void @test_04(i32 %n) { define void @test_04(i32 %n) {
; ;
; CHECK-LABEL: @test_04( ; CHECK-LABEL: @test_04(
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: [[TMP0:%.*]] = icmp sgt i32 [[N:%.*]], -2147483647 ; CHECK-NEXT: [[SEXT:%.*]] = sext i32 [[N:%.*]] to i64
; CHECK-NEXT: [[SMAX:%.*]] = select i1 [[TMP0]], i32 [[N]], i32 -2147483647
; CHECK-NEXT: [[TMP1:%.*]] = add i32 [[SMAX]], 1
; CHECK-NEXT: br label [[LOOP:%.*]] ; CHECK-NEXT: br label [[LOOP:%.*]]
; CHECK: loop: ; CHECK: loop:
; CHECK-NEXT: [[IV:%.*]] = phi i64 [ -2147483647, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LOOP]] ] ; CHECK-NEXT: [[IV:%.*]] = phi i64 [ -2147483647, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LOOP]] ]
; CHECK-NEXT: [[IV_NEXT]] = add nsw i64 [[IV]], 1 ; CHECK-NEXT: [[IV_NEXT]] = add nsw i64 [[IV]], 1
; CHECK-NEXT: [[LFTR_WIDEIV:%.*]] = trunc i64 [[IV_NEXT]] to i32 ; CHECK-NEXT: [[TMP0:%.*]] = icmp slt i64 [[IV]], [[SEXT]]
; CHECK-NEXT: [[EXITCOND:%.*]] = icmp ne i32 [[LFTR_WIDEIV]], [[TMP1]] ; CHECK-NEXT: br i1 [[TMP0]], label [[LOOP]], label [[EXIT:%.*]]
; CHECK-NEXT: br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
; CHECK: exit: ; CHECK: exit:
; CHECK-NEXT: ret void ; CHECK-NEXT: ret void
; ;
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ -2147483647, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ -2147483647, %entry ], [ %iv.next, %loop ]
%iv.next = add i64 %iv, 1 %iv.next = add i64 %iv, 1
▲ Show 20 Lines • Show All 107 Lines • ▼ Show 20 Lines
exit: exit:
ret void ret void
} }
; Max value at which we can eliminate trunc: UINT_MAX - 1. ; Max value at which we can eliminate trunc: UINT_MAX - 1.
define void @test_02_unsigned(i32 %n) { define void @test_02_unsigned(i32 %n) {
; CHECK-LABEL: @test_02_unsigned( ; CHECK-LABEL: @test_02_unsigned(
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: [[TMP0:%.*]] = icmp ugt i32 [[N:%.*]], -2 ; CHECK-NEXT: [[ZEXT:%.*]] = zext i32 [[N:%.*]] to i64
; CHECK-NEXT: [[UMAX:%.*]] = select i1 [[TMP0]], i32 [[N]], i32 -2
; CHECK-NEXT: [[TMP1:%.*]] = add nsw i32 [[UMAX]], 1
; CHECK-NEXT: br label [[LOOP:%.*]] ; CHECK-NEXT: br label [[LOOP:%.*]]
; CHECK: loop: ; CHECK: loop:
; CHECK-NEXT: [[IV:%.*]] = phi i64 [ 4294967294, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LOOP]] ] ; CHECK-NEXT: [[IV:%.*]] = phi i64 [ 4294967294, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LOOP]] ]
; CHECK-NEXT: [[IV_NEXT]] = add nuw nsw i64 [[IV]], 1 ; CHECK-NEXT: [[IV_NEXT]] = add nuw nsw i64 [[IV]], 1
; CHECK-NEXT: [[LFTR_WIDEIV:%.*]] = trunc i64 [[IV_NEXT]] to i32 ; CHECK-NEXT: [[TMP0:%.*]] = icmp ult i64 [[IV]], [[ZEXT]]
; CHECK-NEXT: [[EXITCOND:%.*]] = icmp ne i32 [[LFTR_WIDEIV]], [[TMP1]] ; CHECK-NEXT: br i1 [[TMP0]], label [[LOOP]], label [[EXIT:%.*]]
; CHECK-NEXT: br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
; CHECK: exit: ; CHECK: exit:
; CHECK-NEXT: ret void ; CHECK-NEXT: ret void
; ;
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 4294967294, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 4294967294, %entry ], [ %iv.next, %loop ]
%iv.next = add i64 %iv, 1 %iv.next = add i64 %iv, 1
▲ Show 20 Lines • Show All 52 Lines • ▼ Show 20 Lines
exit: exit:
ret void ret void
} }
; Start from 1. ; Start from 1.
define void @test_05_unsigned(i32 %n) { define void @test_05_unsigned(i32 %n) {
; CHECK-LABEL: @test_05_unsigned( ; CHECK-LABEL: @test_05_unsigned(
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: [[TMP0:%.*]] = icmp ugt i32 [[N:%.*]], 1 ; CHECK-NEXT: [[ZEXT:%.*]] = zext i32 [[N:%.*]] to i64
; CHECK-NEXT: [[UMAX:%.*]] = select i1 [[TMP0]], i32 [[N]], i32 1
; CHECK-NEXT: [[TMP1:%.*]] = add i32 [[UMAX]], 1
; CHECK-NEXT: br label [[LOOP:%.*]] ; CHECK-NEXT: br label [[LOOP:%.*]]
; CHECK: loop: ; CHECK: loop:
; CHECK-NEXT: [[IV:%.*]] = phi i64 [ 1, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LOOP]] ] ; CHECK-NEXT: [[IV:%.*]] = phi i64 [ 1, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LOOP]] ]
; CHECK-NEXT: [[IV_NEXT]] = add nuw nsw i64 [[IV]], 1 ; CHECK-NEXT: [[IV_NEXT]] = add nuw nsw i64 [[IV]], 1
; CHECK-NEXT: [[LFTR_WIDEIV:%.*]] = trunc i64 [[IV_NEXT]] to i32 ; CHECK-NEXT: [[TMP0:%.*]] = icmp ult i64 [[IV]], [[ZEXT]]
; CHECK-NEXT: [[EXITCOND:%.*]] = icmp ne i32 [[LFTR_WIDEIV]], [[TMP1]] ; CHECK-NEXT: br i1 [[TMP0]], label [[LOOP]], label [[EXIT:%.*]]
; CHECK-NEXT: br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]]
; CHECK: exit: ; CHECK: exit:
; CHECK-NEXT: ret void ; CHECK-NEXT: ret void
; ;
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 1, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 1, %entry ], [ %iv.next, %loop ]
%iv.next = add i64 %iv, 1 %iv.next = add i64 %iv, 1
▲ Show 20 Lines • Show All 114 Lines • ▼ Show 20 Lines
exit: exit:
ret void ret void
} }
; Widen NE as signed. ; Widen NE as signed.
define void @test_10(i32 %n) { define void @test_10(i32 %n) {
; CHECK-LABEL: @test_10( ; CHECK-LABEL: @test_10(
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: [[TMP0:%.*]] = add i32 [[N:%.*]], 100 ; CHECK-NEXT: [[SEXT:%.*]] = sext i32 [[N:%.*]] to i64
; CHECK-NEXT: [[TMP1:%.*]] = zext i32 [[TMP0]] to i64
; CHECK-NEXT: [[TMP2:%.*]] = icmp ult i64 [[TMP1]], 90
; CHECK-NEXT: [[UMIN:%.*]] = select i1 [[TMP2]], i64 [[TMP1]], i64 90
; CHECK-NEXT: [[TMP3:%.*]] = add i64 [[UMIN]], -99
; CHECK-NEXT: br label [[LOOP:%.*]] ; CHECK-NEXT: br label [[LOOP:%.*]]
; CHECK: loop: ; CHECK: loop:
; CHECK-NEXT: [[IV:%.*]] = phi i64 [ -100, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LOOP]] ] ; CHECK-NEXT: [[IV:%.*]] = phi i64 [ -100, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LOOP]] ]
; CHECK-NEXT: [[IV_NEXT]] = add nuw nsw i64 [[IV]], 1 ; CHECK-NEXT: [[IV_NEXT]] = add nuw nsw i64 [[IV]], 1
; CHECK-NEXT: [[EXITCOND:%.*]] = icmp ne i64 [[IV_NEXT]], [[TMP3]] ; CHECK-NEXT: [[TMP0:%.*]] = icmp ne i64 [[IV]], [[SEXT]]
; CHECK-NEXT: br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]] ; CHECK-NEXT: [[NEGCMP:%.*]] = icmp slt i64 [[IV]], -10
; CHECK-NEXT: [[CMP:%.*]] = and i1 [[TMP0]], [[NEGCMP]]
; CHECK-NEXT: br i1 [[CMP]], label [[LOOP]], label [[EXIT:%.*]]
; CHECK: exit: ; CHECK: exit:
; CHECK-NEXT: ret void ; CHECK-NEXT: ret void
; ;
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ -100, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ -100, %entry ], [ %iv.next, %loop ]
%iv.next = add i64 %iv, 1 %iv.next = add i64 %iv, 1
▲ Show 20 Lines • Show All 54 Lines • ▼ Show 20 Lines
} }
; Show that we can turn signed comparison to unsigned and use zext while ; Show that we can turn signed comparison to unsigned and use zext while
; comparing non-negative values. ; comparing non-negative values.
define void @test_12(i32* %p) { define void @test_12(i32* %p) {
; CHECK-LABEL: @test_12( ; CHECK-LABEL: @test_12(
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: [[N:%.*]] = load i32, i32* [[P:%.*]], !range !0 ; CHECK-NEXT: [[N:%.*]] = load i32, i32* [[P:%.*]], !range !0
; CHECK-NEXT: [[TMP0:%.*]] = icmp sgt i32 [[N]], 1 ; CHECK-NEXT: [[ZEXT:%.*]] = zext i32 [[N]] to i64
; CHECK-NEXT: [[SMAX:%.*]] = select i1 [[TMP0]], i32 [[N]], i32 1
; CHECK-NEXT: [[WIDE_TRIP_COUNT:%.*]] = zext i32 [[SMAX]] to i64
; CHECK-NEXT: br label [[LOOP:%.*]] ; CHECK-NEXT: br label [[LOOP:%.*]]
; CHECK: loop: ; CHECK: loop:
; CHECK-NEXT: [[IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LOOP]] ] ; CHECK-NEXT: [[IV:%.*]] = phi i64 [ 0, [[ENTRY:%.*]] ], [ [[IV_NEXT:%.*]], [[LOOP]] ]
; CHECK-NEXT: [[IV_NEXT]] = add nuw nsw i64 [[IV]], 1 ; CHECK-NEXT: [[IV_NEXT]] = add nuw nsw i64 [[IV]], 1
; CHECK-NEXT: [[EXITCOND:%.*]] = icmp ne i64 [[IV_NEXT]], [[WIDE_TRIP_COUNT]] ; CHECK-NEXT: [[TMP0:%.*]] = icmp ult i64 [[IV_NEXT]], [[ZEXT]]
; CHECK-NEXT: br i1 [[EXITCOND]], label [[LOOP]], label [[EXIT:%.*]] ; CHECK-NEXT: br i1 [[TMP0]], label [[LOOP]], label [[EXIT:%.*]]
; CHECK: exit: ; CHECK: exit:
; CHECK-NEXT: ret void ; CHECK-NEXT: ret void
; ;
entry: entry:
%n = load i32, i32* %p, !range !0 %n = load i32, i32* %p, !range !0
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
▲ Show 20 Lines • Show All 113 Lines • Show Last 20 Lines

llvm/test/Transforms/IndVarSimplify/iv-widen.ll

Show First 20 Lines • Show All 116 Lines • ▼ Show 20 Lines
} }
define void @loop_1(i32 %lim) { define void @loop_1(i32 %lim) {
; CHECK-LABEL: @loop_1( ; CHECK-LABEL: @loop_1(
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: [[ENTRY_COND:%.*]] = icmp ne i32 [[LIM:%.*]], 0 ; CHECK-NEXT: [[ENTRY_COND:%.*]] = icmp ne i32 [[LIM:%.*]], 0
; CHECK-NEXT: br i1 [[ENTRY_COND]], label [[LOOP_PREHEADER:%.*]], label [[LEAVE:%.*]] ; CHECK-NEXT: br i1 [[ENTRY_COND]], label [[LOOP_PREHEADER:%.*]], label [[LEAVE:%.*]]
; CHECK: loop.preheader: ; CHECK: loop.preheader:
; CHECK-NEXT: [[TMP0:%.*]] = icmp ugt i32 [[LIM]], 2 ; CHECK-NEXT: [[TMP0:%.*]] = zext i32 [[LIM]] to i64
; CHECK-NEXT: [[UMAX:%.*]] = select i1 [[TMP0]], i32 [[LIM]], i32 2
; CHECK-NEXT: [[WIDE_TRIP_COUNT:%.*]] = zext i32 [[UMAX]] to i64
; CHECK-NEXT: br label [[LOOP:%.*]] ; CHECK-NEXT: br label [[LOOP:%.*]]
; CHECK: loop: ; CHECK: loop:
; CHECK-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ 1, [[LOOP_PREHEADER]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ] ; CHECK-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ 1, [[LOOP_PREHEADER]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
; CHECK-NEXT: [[INDVARS_IV_NEXT]] = add nuw nsw i64 [[INDVARS_IV]], 1 ; CHECK-NEXT: [[INDVARS_IV_NEXT]] = add nuw nsw i64 [[INDVARS_IV]], 1
; CHECK-NEXT: [[TMP1:%.*]] = add nsw i64 [[INDVARS_IV]], -1 ; CHECK-NEXT: [[TMP1:%.*]] = add nsw i64 [[INDVARS_IV]], -1
; CHECK-NEXT: call void @dummy.i64(i64 [[TMP1]]) ; CHECK-NEXT: call void @dummy.i64(i64 [[TMP1]])
; CHECK-NEXT: [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[WIDE_TRIP_COUNT]] ; CHECK-NEXT: [[BE_COND:%.*]] = icmp ult i64 [[INDVARS_IV_NEXT]], [[TMP0]]
; CHECK-NEXT: br i1 [[EXITCOND]], label [[LOOP]], label [[LEAVE_LOOPEXIT:%.*]] ; CHECK-NEXT: br i1 [[BE_COND]], label [[LOOP]], label [[LEAVE_LOOPEXIT:%.*]]
; CHECK: leave.loopexit: ; CHECK: leave.loopexit:
; CHECK-NEXT: br label [[LEAVE]] ; CHECK-NEXT: br label [[LEAVE]]
; CHECK: leave: ; CHECK: leave:
; CHECK-NEXT: ret void ; CHECK-NEXT: ret void
; ;
entry: entry:
%entry.cond = icmp ne i32 %lim, 0 %entry.cond = icmp ne i32 %lim, 0
br i1 %entry.cond, label %loop, label %leave br i1 %entry.cond, label %loop, label %leave
Show All 18 Lines
define void @loop_2(i32 %size, i32 %nsteps, i32 %hsize, i32* %lined, i8 %tmp1) { define void @loop_2(i32 %size, i32 %nsteps, i32 %hsize, i32* %lined, i8 %tmp1) {
; CHECK-LABEL: @loop_2( ; CHECK-LABEL: @loop_2(
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: [[CMP215:%.*]] = icmp sgt i32 [[SIZE:%.*]], 1 ; CHECK-NEXT: [[CMP215:%.*]] = icmp sgt i32 [[SIZE:%.*]], 1
; CHECK-NEXT: [[BC0:%.*]] = bitcast i32* [[LINED:%.*]] to i8* ; CHECK-NEXT: [[BC0:%.*]] = bitcast i32* [[LINED:%.*]] to i8*
; CHECK-NEXT: [[TMP0:%.*]] = sext i32 [[SIZE]] to i64 ; CHECK-NEXT: [[TMP0:%.*]] = sext i32 [[SIZE]] to i64
; CHECK-NEXT: [[TMP1:%.*]] = sext i32 [[HSIZE:%.*]] to i64 ; CHECK-NEXT: [[TMP1:%.*]] = sext i32 [[HSIZE:%.*]] to i64
; CHECK-NEXT: [[TMP2:%.*]] = icmp sgt i32 [[NSTEPS:%.*]], 1 ; CHECK-NEXT: [[TMP2:%.*]] = sext i32 [[NSTEPS:%.*]] to i64
; CHECK-NEXT: [[SMAX:%.*]] = select i1 [[TMP2]], i32 [[NSTEPS]], i32 1
; CHECK-NEXT: [[WIDE_TRIP_COUNT11:%.*]] = zext i32 [[SMAX]] to i64
; CHECK-NEXT: br label [[FOR_BODY:%.*]] ; CHECK-NEXT: br label [[FOR_BODY:%.*]]
; CHECK: for.body: ; CHECK: for.body:
; CHECK-NEXT: [[INDVARS_IV7:%.*]] = phi i64 [ [[INDVARS_IV_NEXT8:%.*]], [[FOR_INC:%.*]] ], [ 0, [[ENTRY:%.*]] ] ; CHECK-NEXT: [[INDVARS_IV7:%.*]] = phi i64 [ [[INDVARS_IV_NEXT8:%.*]], [[FOR_INC:%.*]] ], [ 0, [[ENTRY:%.*]] ]
; CHECK-NEXT: [[TMP3:%.*]] = mul nsw i64 [[INDVARS_IV7]], [[TMP0]] ; CHECK-NEXT: [[TMP3:%.*]] = mul nsw i64 [[INDVARS_IV7]], [[TMP0]]
; CHECK-NEXT: [[TMP4:%.*]] = add nsw i64 [[TMP3]], [[TMP1]] ; CHECK-NEXT: [[TMP4:%.*]] = add nsw i64 [[TMP3]], [[TMP1]]
; CHECK-NEXT: br i1 [[CMP215]], label [[FOR_BODY2_PREHEADER:%.*]], label [[FOR_INC]] ; CHECK-NEXT: br i1 [[CMP215]], label [[FOR_BODY2_PREHEADER:%.*]], label [[FOR_INC]]
; CHECK: for.body2.preheader: ; CHECK: for.body2.preheader:
; CHECK-NEXT: [[WIDE_TRIP_COUNT:%.*]] = zext i32 [[SIZE]] to i64 ; CHECK-NEXT: [[WIDE_TRIP_COUNT:%.*]] = zext i32 [[SIZE]] to i64
Show All 18 Lines
; CHECK-NEXT: store i8 [[TMP1]], i8* [[ADD_PTR2]], align 1 ; CHECK-NEXT: store i8 [[TMP1]], i8* [[ADD_PTR2]], align 1
; CHECK-NEXT: [[INDVARS_IV_NEXT3]] = add nuw nsw i64 [[INDVARS_IV2]], 1 ; CHECK-NEXT: [[INDVARS_IV_NEXT3]] = add nuw nsw i64 [[INDVARS_IV2]], 1
; CHECK-NEXT: [[EXITCOND6:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT3]], [[WIDE_TRIP_COUNT5]] ; CHECK-NEXT: [[EXITCOND6:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT3]], [[WIDE_TRIP_COUNT5]]
; CHECK-NEXT: br i1 [[EXITCOND6]], label [[FOR_BODY3]], label [[FOR_INC_LOOPEXIT:%.*]] ; CHECK-NEXT: br i1 [[EXITCOND6]], label [[FOR_BODY3]], label [[FOR_INC_LOOPEXIT:%.*]]
; CHECK: for.inc.loopexit: ; CHECK: for.inc.loopexit:
; CHECK-NEXT: br label [[FOR_INC]] ; CHECK-NEXT: br label [[FOR_INC]]
; CHECK: for.inc: ; CHECK: for.inc:
; CHECK-NEXT: [[INDVARS_IV_NEXT8]] = add nuw nsw i64 [[INDVARS_IV7]], 1 ; CHECK-NEXT: [[INDVARS_IV_NEXT8]] = add nuw nsw i64 [[INDVARS_IV7]], 1
; CHECK-NEXT: [[EXITCOND12:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT8]], [[WIDE_TRIP_COUNT11]] ; CHECK-NEXT: [[CMP:%.*]] = icmp slt i64 [[INDVARS_IV_NEXT8]], [[TMP2]]
; CHECK-NEXT: br i1 [[EXITCOND12]], label [[FOR_BODY]], label [[FOR_END_LOOPEXIT:%.*]] ; CHECK-NEXT: br i1 [[CMP]], label [[FOR_BODY]], label [[FOR_END_LOOPEXIT:%.*]]
; CHECK: for.end.loopexit: ; CHECK: for.end.loopexit:
; CHECK-NEXT: ret void ; CHECK-NEXT: ret void
; ;
entry: entry:
%cmp215 = icmp sgt i32 %size, 1 %cmp215 = icmp sgt i32 %size, 1
%bc0 = bitcast i32* %lined to i8* %bc0 = bitcast i32* %lined to i8*
br label %for.body br label %for.body
Show All 36 Lines

llvm/test/Transforms/IndVarSimplify/loop-invariant-conditions.ll

Show First 20 Lines • Show All 305 Lines • ▼ Show 20 Lines
for.end: ; preds = %if.end, %entry for.end: ; preds = %if.end, %entry
ret void ret void
} }
; The branch has to exit the loop if the condition is true ; The branch has to exit the loop if the condition is true
define void @test3_neg(i64 %start) { define void @test3_neg(i64 %start) {
; CHECK-LABEL: @test3_neg( ; CHECK-LABEL: @test3_neg(
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: [[TMP0:%.*]] = icmp sgt i64 [[START:%.*]], -1
; CHECK-NEXT: [[SMAX:%.*]] = select i1 [[TMP0]], i64 [[START]], i64 -1
; CHECK-NEXT: [[TMP1:%.*]] = add i64 [[SMAX]], 1
; CHECK-NEXT: br label [[LOOP:%.*]] ; CHECK-NEXT: br label [[LOOP:%.*]]
; CHECK: loop: ; CHECK: loop:
; CHECK-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[START]], [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ] ; CHECK-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[START:%.*]], [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[LOOP]] ]
; CHECK-NEXT: [[INDVARS_IV_NEXT]] = add i64 [[INDVARS_IV]], 1 ; CHECK-NEXT: [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
; CHECK-NEXT: [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[TMP1]] ; CHECK-NEXT: [[CMP1:%.*]] = icmp slt i64 [[INDVARS_IV]], -1
; CHECK-NEXT: br i1 [[EXITCOND]], label [[LOOP]], label [[FOR_END:%.*]] ; CHECK-NEXT: br i1 [[CMP1]], label [[LOOP]], label [[FOR_END:%.*]]
; CHECK: for.end: ; CHECK: for.end:
; CHECK-NEXT: ret void ; CHECK-NEXT: ret void
; ;
entry: entry:
br label %loop br label %loop
loop: loop:
%indvars.iv = phi i64 [ %start, %entry ], [ %indvars.iv.next, %loop ] %indvars.iv = phi i64 [ %start, %entry ], [ %indvars.iv.next, %loop ]
%indvars.iv.next = add nsw i64 %indvars.iv, 1 %indvars.iv.next = add nsw i64 %indvars.iv, 1
%cmp1 = icmp slt i64 %indvars.iv, -1 %cmp1 = icmp slt i64 %indvars.iv, -1
br i1 %cmp1, label %loop, label %for.end br i1 %cmp1, label %loop, label %for.end
for.end: ; preds = %if.end, %entry for.end: ; preds = %if.end, %entry
ret void ret void
} }
define void @test4_neg(i64 %start) { define void @test4_neg(i64 %start) {
; CHECK-LABEL: @test4_neg( ; CHECK-LABEL: @test4_neg(
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: [[TMP0:%.*]] = icmp sgt i64 [[START:%.*]], 0
; CHECK-NEXT: [[SMAX:%.*]] = select i1 [[TMP0]], i64 [[START]], i64 0
; CHECK-NEXT: [[TMP1:%.*]] = add nuw i64 [[SMAX]], 1
; CHECK-NEXT: br label [[LOOP:%.*]] ; CHECK-NEXT: br label [[LOOP:%.*]]
; CHECK: loop: ; CHECK: loop:
; CHECK-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[START]], [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[BACKEDGE:%.*]] ] ; CHECK-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[START:%.*]], [[ENTRY:%.*]] ], [ [[INDVARS_IV_NEXT:%.*]], [[BACKEDGE:%.*]] ]
; CHECK-NEXT: [[INDVARS_IV_NEXT]] = add i64 [[INDVARS_IV]], 1 ; CHECK-NEXT: [[INDVARS_IV_NEXT]] = add nsw i64 [[INDVARS_IV]], 1
; CHECK-NEXT: [[CMP:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT]], 25 ; CHECK-NEXT: [[CMP:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT]], 25
; CHECK-NEXT: br i1 [[CMP]], label [[BACKEDGE]], label [[FOR_END:%.*]] ; CHECK-NEXT: br i1 [[CMP]], label [[BACKEDGE]], label [[FOR_END:%.*]]
; CHECK: backedge: ; CHECK: backedge:
; CHECK-NEXT: call void @foo() ; CHECK-NEXT: call void @foo()
; CHECK-NEXT: [[EXITCOND:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT]], [[TMP1]] ; CHECK-NEXT: [[CMP1:%.*]] = icmp sgt i64 [[INDVARS_IV]], -1
; CHECK-NEXT: br i1 [[EXITCOND]], label [[FOR_END]], label [[LOOP]] ; CHECK-NEXT: br i1 [[CMP1]], label [[FOR_END]], label [[LOOP]]
; CHECK: for.end: ; CHECK: for.end:
; CHECK-NEXT: ret void ; CHECK-NEXT: ret void
; ;
entry: entry:
br label %loop br label %loop
loop: loop:
%indvars.iv = phi i64 [ %start, %entry ], [ %indvars.iv.next, %backedge ] %indvars.iv = phi i64 [ %start, %entry ], [ %indvars.iv.next, %backedge ]
▲ Show 20 Lines • Show All 263 Lines • Show Last 20 Lines

llvm/test/Transforms/IndVarSimplify/widen-loop-comp.ll

Show First 20 Lines • Show All 93 Lines • ▼ Show 20 Lines
define void @test2([8 x i8]* %a, i8* %b, i8 %limit) { define void @test2([8 x i8]* %a, i8* %b, i8 %limit) {
; CHECK-LABEL: @test2( ; CHECK-LABEL: @test2(
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: [[CONV:%.*]] = zext i8 [[LIMIT:%.*]] to i32 ; CHECK-NEXT: [[CONV:%.*]] = zext i8 [[LIMIT:%.*]] to i32
; CHECK-NEXT: br i1 undef, label [[FOR_COND1_PREHEADER_PREHEADER:%.*]], label [[FOR_COND1_PREHEADER_US_PREHEADER:%.*]] ; CHECK-NEXT: br i1 undef, label [[FOR_COND1_PREHEADER_PREHEADER:%.*]], label [[FOR_COND1_PREHEADER_US_PREHEADER:%.*]]
; CHECK: for.cond1.preheader.us.preheader: ; CHECK: for.cond1.preheader.us.preheader:
; CHECK-NEXT: [[TMP0:%.*]] = icmp sgt i32 [[CONV]], 1 ; CHECK-NEXT: [[TMP0:%.*]] = zext i32 [[CONV]] to i64
; CHECK-NEXT: [[SMAX:%.*]] = select i1 [[TMP0]], i32 [[CONV]], i32 1
; CHECK-NEXT: br label [[FOR_COND1_PREHEADER_US:%.*]] ; CHECK-NEXT: br label [[FOR_COND1_PREHEADER_US:%.*]]
; CHECK: for.cond1.preheader.preheader: ; CHECK: for.cond1.preheader.preheader:
; CHECK-NEXT: br label [[FOR_COND1_PREHEADER:%.*]] ; CHECK-NEXT: br label [[FOR_COND1_PREHEADER:%.*]]
; CHECK: for.cond1.preheader.us: ; CHECK: for.cond1.preheader.us:
; CHECK-NEXT: [[INDVARS_IV2:%.*]] = phi i64 [ 0, [[FOR_COND1_PREHEADER_US_PREHEADER]] ], [ [[INDVARS_IV_NEXT3:%.*]], [[FOR_INC13_US:%.*]] ] ; CHECK-NEXT: [[INDVARS_IV2:%.*]] = phi i64 [ 0, [[FOR_COND1_PREHEADER_US_PREHEADER]] ], [ [[INDVARS_IV_NEXT3:%.*]], [[FOR_INC13_US:%.*]] ]
; CHECK-NEXT: br i1 true, label [[FOR_BODY4_LR_PH_US:%.*]], label [[FOR_INC13_US]] ; CHECK-NEXT: br i1 true, label [[FOR_BODY4_LR_PH_US:%.*]], label [[FOR_INC13_US]]
; CHECK: for.inc13.us.loopexit: ; CHECK: for.inc13.us.loopexit:
; CHECK-NEXT: br label [[FOR_INC13_US]] ; CHECK-NEXT: br label [[FOR_INC13_US]]
; CHECK: for.inc13.us: ; CHECK: for.inc13.us:
; CHECK-NEXT: [[INDVARS_IV_NEXT3]] = add nuw nsw i64 [[INDVARS_IV2]], 1 ; CHECK-NEXT: [[INDVARS_IV_NEXT3]] = add nuw nsw i64 [[INDVARS_IV2]], 1
; CHECK-NEXT: [[EXITCOND4:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT3]], 4 ; CHECK-NEXT: [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT3]], 4
; CHECK-NEXT: br i1 [[EXITCOND4]], label [[FOR_COND1_PREHEADER_US]], label [[FOR_END_LOOPEXIT1:%.*]] ; CHECK-NEXT: br i1 [[EXITCOND]], label [[FOR_COND1_PREHEADER_US]], label [[FOR_END_LOOPEXIT1:%.*]]
; CHECK: for.body4.us: ; CHECK: for.body4.us:
; CHECK-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ 0, [[FOR_BODY4_LR_PH_US]] ], [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY4_US:%.*]] ] ; CHECK-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ 0, [[FOR_BODY4_LR_PH_US]] ], [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY4_US:%.*]] ]
; CHECK-NEXT: [[ARRAYIDX6_US:%.*]] = getelementptr inbounds [8 x i8], [8 x i8]* [[A:%.*]], i64 [[INDVARS_IV2]], i64 [[INDVARS_IV]] ; CHECK-NEXT: [[ARRAYIDX6_US:%.*]] = getelementptr inbounds [8 x i8], [8 x i8]* [[A:%.*]], i64 [[INDVARS_IV2]], i64 [[INDVARS_IV]]
; CHECK-NEXT: [[TMP1:%.*]] = load i8, i8* [[ARRAYIDX6_US]], align 1 ; CHECK-NEXT: [[TMP1:%.*]] = load i8, i8* [[ARRAYIDX6_US]], align 1
; CHECK-NEXT: [[IDXPROM7_US:%.*]] = zext i8 [[TMP1]] to i64 ; CHECK-NEXT: [[IDXPROM7_US:%.*]] = zext i8 [[TMP1]] to i64
; CHECK-NEXT: [[ARRAYIDX8_US:%.*]] = getelementptr inbounds i8, i8* [[B:%.*]], i64 [[IDXPROM7_US]] ; CHECK-NEXT: [[ARRAYIDX8_US:%.*]] = getelementptr inbounds i8, i8* [[B:%.*]], i64 [[IDXPROM7_US]]
; CHECK-NEXT: [[TMP2:%.*]] = load i8, i8* [[ARRAYIDX8_US]], align 1 ; CHECK-NEXT: [[TMP2:%.*]] = load i8, i8* [[ARRAYIDX8_US]], align 1
; CHECK-NEXT: store i8 [[TMP2]], i8* [[ARRAYIDX6_US]], align 1 ; CHECK-NEXT: store i8 [[TMP2]], i8* [[ARRAYIDX6_US]], align 1
; CHECK-NEXT: [[INDVARS_IV_NEXT]] = add nuw nsw i64 [[INDVARS_IV]], 1 ; CHECK-NEXT: [[INDVARS_IV_NEXT]] = add nuw nsw i64 [[INDVARS_IV]], 1
; CHECK-NEXT: [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[WIDE_TRIP_COUNT:%.*]] ; CHECK-NEXT: [[CMP2_US:%.*]] = icmp ult i64 [[INDVARS_IV_NEXT]], [[TMP0]]
; CHECK-NEXT: br i1 [[EXITCOND]], label [[FOR_BODY4_US]], label [[FOR_INC13_US_LOOPEXIT:%.*]] ; CHECK-NEXT: br i1 [[CMP2_US]], label [[FOR_BODY4_US]], label [[FOR_INC13_US_LOOPEXIT:%.*]]
; CHECK: for.body4.lr.ph.us: ; CHECK: for.body4.lr.ph.us:
; CHECK-NEXT: [[WIDE_TRIP_COUNT]] = zext i32 [[SMAX]] to i64
; CHECK-NEXT: br label [[FOR_BODY4_US]] ; CHECK-NEXT: br label [[FOR_BODY4_US]]
; CHECK: for.cond1.preheader: ; CHECK: for.cond1.preheader:
; CHECK-NEXT: br i1 false, label [[FOR_INC13:%.*]], label [[FOR_INC13]] ; CHECK-NEXT: br i1 false, label [[FOR_INC13:%.*]], label [[FOR_INC13]]
; CHECK: for.inc13: ; CHECK: for.inc13:
; CHECK-NEXT: br i1 false, label [[FOR_COND1_PREHEADER]], label [[FOR_END_LOOPEXIT:%.*]] ; CHECK-NEXT: br i1 false, label [[FOR_COND1_PREHEADER]], label [[FOR_END_LOOPEXIT:%.*]]
; CHECK: for.end.loopexit: ; CHECK: for.end.loopexit:
; CHECK-NEXT: br label [[FOR_END:%.*]] ; CHECK-NEXT: br label [[FOR_END:%.*]]
; CHECK: for.end.loopexit1: ; CHECK: for.end.loopexit1:
▲ Show 20 Lines • Show All 165 Lines • ▼ Show 20 Lines
for.end: for.end:
ret i32 %sum.0 ret i32 %sum.0
} }
define i32 @test6(i32* %a, i32 %b) { define i32 @test6(i32* %a, i32 %b) {
; CHECK-LABEL: @test6( ; CHECK-LABEL: @test6(
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: [[TMP0:%.*]] = icmp sgt i32 [[B:%.*]], -1 ; CHECK-NEXT: [[TMP0:%.*]] = sext i32 [[B:%.*]] to i64
; CHECK-NEXT: [[SMAX:%.*]] = select i1 [[TMP0]], i32 [[B]], i32 -1
; CHECK-NEXT: [[TMP1:%.*]] = add i32 [[SMAX]], 1
; CHECK-NEXT: [[WIDE_TRIP_COUNT:%.*]] = zext i32 [[TMP1]] to i64
; CHECK-NEXT: br label [[FOR_COND:%.*]] ; CHECK-NEXT: br label [[FOR_COND:%.*]]
; CHECK: for.cond: ; CHECK: for.cond:
; CHECK-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY:%.*]] ], [ 0, [[ENTRY:%.*]] ] ; CHECK-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY:%.*]] ], [ 0, [[ENTRY:%.*]] ]
; CHECK-NEXT: [[SUM_0:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[ADD:%.*]], [[FOR_BODY]] ] ; CHECK-NEXT: [[SUM_0:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[ADD:%.*]], [[FOR_BODY]] ]
; CHECK-NEXT: [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV]], [[WIDE_TRIP_COUNT]] ; CHECK-NEXT: [[CMP:%.*]] = icmp sle i64 [[INDVARS_IV]], [[TMP0]]
; CHECK-NEXT: br i1 [[EXITCOND]], label [[FOR_BODY]], label [[FOR_END:%.*]] ; CHECK-NEXT: br i1 [[CMP]], label [[FOR_BODY]], label [[FOR_END:%.*]]
; CHECK: for.body: ; CHECK: for.body:
; CHECK-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[A:%.*]], i64 [[INDVARS_IV]] ; CHECK-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[A:%.*]], i64 [[INDVARS_IV]]
; CHECK-NEXT: [[TMP2:%.*]] = load i32, i32* [[ARRAYIDX]], align 4 ; CHECK-NEXT: [[TMP1:%.*]] = load i32, i32* [[ARRAYIDX]], align 4
; CHECK-NEXT: [[ADD]] = add nsw i32 [[SUM_0]], [[TMP2]] ; CHECK-NEXT: [[ADD]] = add nsw i32 [[SUM_0]], [[TMP1]]
; CHECK-NEXT: [[INDVARS_IV_NEXT]] = add nuw nsw i64 [[INDVARS_IV]], 1 ; CHECK-NEXT: [[INDVARS_IV_NEXT]] = add nuw nsw i64 [[INDVARS_IV]], 1
; CHECK-NEXT: br label [[FOR_COND]] ; CHECK-NEXT: br label [[FOR_COND]]
; CHECK: for.end: ; CHECK: for.end:
; CHECK-NEXT: [[SUM_0_LCSSA:%.*]] = phi i32 [ [[SUM_0]], [[FOR_COND]] ] ; CHECK-NEXT: [[SUM_0_LCSSA:%.*]] = phi i32 [ [[SUM_0]], [[FOR_COND]] ]
; CHECK-NEXT: ret i32 [[SUM_0_LCSSA]] ; CHECK-NEXT: ret i32 [[SUM_0_LCSSA]]
; ;
entry: entry:
br label %for.cond br label %for.cond
Show All 15 Lines
for.end: for.end:
ret i32 %sum.0 ret i32 %sum.0
} }
define i32 @test7(i32* %a, i32 %b) { define i32 @test7(i32* %a, i32 %b) {
; CHECK-LABEL: @test7( ; CHECK-LABEL: @test7(
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: [[TMP0:%.*]] = zext i32 [[B:%.*]] to i64 ; CHECK-NEXT: [[TMP0:%.*]] = zext i32 [[B:%.*]] to i64
; CHECK-NEXT: [[TMP1:%.*]] = icmp sgt i32 [[B]], -1 ; CHECK-NEXT: [[TMP1:%.*]] = sext i32 [[B]] to i64
; CHECK-NEXT: [[SMAX:%.*]] = select i1 [[TMP1]], i32 [[B]], i32 -1
; CHECK-NEXT: [[TMP2:%.*]] = add i32 [[SMAX]], 2
; CHECK-NEXT: [[WIDE_TRIP_COUNT:%.*]] = zext i32 [[TMP2]] to i64
; CHECK-NEXT: br label [[FOR_COND:%.*]] ; CHECK-NEXT: br label [[FOR_COND:%.*]]
; CHECK: for.cond: ; CHECK: for.cond:
; CHECK-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY:%.*]] ], [ 0, [[ENTRY:%.*]] ] ; CHECK-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY:%.*]] ], [ 0, [[ENTRY:%.*]] ]
; CHECK-NEXT: [[SUM_0:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[ADD:%.*]], [[FOR_BODY]] ] ; CHECK-NEXT: [[SUM_0:%.*]] = phi i32 [ 0, [[ENTRY]] ], [ [[ADD:%.*]], [[FOR_BODY]] ]
; CHECK-NEXT: [[CMP:%.*]] = icmp ule i64 [[INDVARS_IV]], [[TMP0]] ; CHECK-NEXT: [[CMP:%.*]] = icmp ule i64 [[INDVARS_IV]], [[TMP0]]
; CHECK-NEXT: br i1 [[CMP]], label [[FOR_BODY]], label [[FOR_END:%.*]] ; CHECK-NEXT: br i1 [[CMP]], label [[FOR_BODY]], label [[FOR_END:%.*]]
; CHECK: for.body: ; CHECK: for.body:
; CHECK-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[A:%.*]], i64 [[INDVARS_IV]] ; CHECK-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[A:%.*]], i64 [[INDVARS_IV]]
; CHECK-NEXT: [[TMP3:%.*]] = load i32, i32* [[ARRAYIDX]], align 4 ; CHECK-NEXT: [[TMP2:%.*]] = load i32, i32* [[ARRAYIDX]], align 4
; CHECK-NEXT: [[ADD]] = add nsw i32 [[SUM_0]], [[TMP3]] ; CHECK-NEXT: [[ADD]] = add nsw i32 [[SUM_0]], [[TMP2]]
; CHECK-NEXT: [[INDVARS_IV_NEXT]] = add nuw nsw i64 [[INDVARS_IV]], 1 ; CHECK-NEXT: [[INDVARS_IV_NEXT]] = add nuw nsw i64 [[INDVARS_IV]], 1
; CHECK-NEXT: [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV_NEXT]], [[WIDE_TRIP_COUNT]] ; CHECK-NEXT: [[CMP2:%.*]] = icmp sle i64 [[INDVARS_IV]], [[TMP1]]
; CHECK-NEXT: br i1 [[EXITCOND]], label [[FOR_COND]], label [[FOR_END]] ; CHECK-NEXT: br i1 [[CMP2]], label [[FOR_COND]], label [[FOR_END]]
; CHECK: for.end: ; CHECK: for.end:
; CHECK-NEXT: [[SUM_0_LCSSA:%.*]] = phi i32 [ [[SUM_0]], [[FOR_BODY]] ], [ [[SUM_0]], [[FOR_COND]] ] ; CHECK-NEXT: [[SUM_0_LCSSA:%.*]] = phi i32 [ [[SUM_0]], [[FOR_BODY]] ], [ [[SUM_0]], [[FOR_COND]] ]
; CHECK-NEXT: ret i32 [[SUM_0_LCSSA]] ; CHECK-NEXT: ret i32 [[SUM_0_LCSSA]]
; ;
entry: entry:
br label %for.cond br label %for.cond
for.cond: for.cond:
▲ Show 20 Lines • Show All 72 Lines • ▼ Show 20 Lines
define i32 @test9(i32* %a, i32 %b, i32 %init) { define i32 @test9(i32* %a, i32 %b, i32 %init) {
; Note: %indvars.iv is the zero extension of %i.0 ; Note: %indvars.iv is the zero extension of %i.0
; CHECK-LABEL: @test9( ; CHECK-LABEL: @test9(
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: [[E:%.*]] = icmp sgt i32 [[INIT:%.*]], 0 ; CHECK-NEXT: [[E:%.*]] = icmp sgt i32 [[INIT:%.*]], 0
; CHECK-NEXT: br i1 [[E]], label [[FOR_COND_PREHEADER:%.*]], label [[LEAVE:%.*]] ; CHECK-NEXT: br i1 [[E]], label [[FOR_COND_PREHEADER:%.*]], label [[LEAVE:%.*]]
; CHECK: for.cond.preheader: ; CHECK: for.cond.preheader:
; CHECK-NEXT: [[TMP0:%.*]] = zext i32 [[INIT]] to i64 ; CHECK-NEXT: [[TMP0:%.*]] = zext i32 [[INIT]] to i64
; CHECK-NEXT: [[TMP1:%.*]] = icmp sgt i32 [[INIT]], [[B:%.*]] ; CHECK-NEXT: [[TMP1:%.*]] = sext i32 [[B:%.*]] to i64
; CHECK-NEXT: [[SMAX:%.*]] = select i1 [[TMP1]], i32 [[INIT]], i32 [[B]]
; CHECK-NEXT: [[WIDE_TRIP_COUNT:%.*]] = zext i32 [[SMAX]] to i64
; CHECK-NEXT: br label [[FOR_COND:%.*]] ; CHECK-NEXT: br label [[FOR_COND:%.*]]
; CHECK: for.cond: ; CHECK: for.cond:
; CHECK-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[TMP0]], [[FOR_COND_PREHEADER]] ], [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY:%.*]] ] ; CHECK-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[TMP0]], [[FOR_COND_PREHEADER]] ], [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY:%.*]] ]
; CHECK-NEXT: [[SUM_0:%.*]] = phi i32 [ [[ADD:%.*]], [[FOR_BODY]] ], [ 0, [[FOR_COND_PREHEADER]] ] ; CHECK-NEXT: [[SUM_0:%.*]] = phi i32 [ [[ADD:%.*]], [[FOR_BODY]] ], [ 0, [[FOR_COND_PREHEADER]] ]
; CHECK-NEXT: [[EXITCOND:%.*]] = icmp ne i64 [[INDVARS_IV]], [[WIDE_TRIP_COUNT]] ; CHECK-NEXT: [[CMP:%.*]] = icmp slt i64 [[INDVARS_IV]], [[TMP1]]
; CHECK-NEXT: br i1 [[EXITCOND]], label [[FOR_BODY]], label [[FOR_END:%.*]] ; CHECK-NEXT: br i1 [[CMP]], label [[FOR_BODY]], label [[FOR_END:%.*]]
; CHECK: for.body: ; CHECK: for.body:
; CHECK-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[A:%.*]], i64 [[INDVARS_IV]] ; CHECK-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds i32, i32* [[A:%.*]], i64 [[INDVARS_IV]]
; CHECK-NEXT: [[TMP2:%.*]] = load i32, i32* [[ARRAYIDX]], align 4 ; CHECK-NEXT: [[TMP2:%.*]] = load i32, i32* [[ARRAYIDX]], align 4
; CHECK-NEXT: [[ADD]] = add nsw i32 [[SUM_0]], [[TMP2]] ; CHECK-NEXT: [[ADD]] = add nsw i32 [[SUM_0]], [[TMP2]]
; CHECK-NEXT: [[INDVARS_IV_NEXT]] = add nuw nsw i64 [[INDVARS_IV]], 1 ; CHECK-NEXT: [[INDVARS_IV_NEXT]] = add nuw nsw i64 [[INDVARS_IV]], 1
; CHECK-NEXT: [[TMP3:%.*]] = trunc i64 [[INDVARS_IV_NEXT]] to i32 ; CHECK-NEXT: [[TMP3:%.*]] = trunc i64 [[INDVARS_IV_NEXT]] to i32
; CHECK-NEXT: [[CMP2:%.*]] = icmp slt i32 0, [[TMP3]] ; CHECK-NEXT: [[CMP2:%.*]] = icmp slt i32 0, [[TMP3]]
; CHECK-NEXT: br i1 [[CMP2]], label [[FOR_COND]], label [[FOR_END]] ; CHECK-NEXT: br i1 [[CMP2]], label [[FOR_COND]], label [[FOR_END]]
▲ Show 20 Lines • Show All 70 Lines • Show Last 20 Lines

llvm/test/Transforms/LoopUnroll/PowerPC/p8-unrolling-legalize-vectors.ll

Show All 13 Lines
; CHECK: for.body.lr.ph: ; CHECK: for.body.lr.ph:
; CHECK-NEXT: [[WIDE_TRIP_COUNT:%.*]] = zext i32 [[K]] to i64 ; CHECK-NEXT: [[WIDE_TRIP_COUNT:%.*]] = zext i32 [[K]] to i64
; CHECK-NEXT: [[MIN_ITERS_CHECK:%.*]] = icmp ult i32 [[K]], 16 ; CHECK-NEXT: [[MIN_ITERS_CHECK:%.*]] = icmp ult i32 [[K]], 16
; CHECK-NEXT: br i1 [[MIN_ITERS_CHECK]], label [[FOR_BODY_PREHEADER:%.*]], label [[VECTOR_PH:%.*]] ; CHECK-NEXT: br i1 [[MIN_ITERS_CHECK]], label [[FOR_BODY_PREHEADER:%.*]], label [[VECTOR_PH:%.*]]
; CHECK: vector.ph: ; CHECK: vector.ph:
; CHECK-NEXT: [[N_VEC:%.*]] = and i64 [[WIDE_TRIP_COUNT]], 4294967280 ; CHECK-NEXT: [[N_VEC:%.*]] = and i64 [[WIDE_TRIP_COUNT]], 4294967280
; CHECK-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <16 x i32> undef, i32 [[X:%.*]], i32 0 ; CHECK-NEXT: [[BROADCAST_SPLATINSERT:%.*]] = insertelement <16 x i32> undef, i32 [[X:%.*]], i32 0
; CHECK-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <16 x i32> [[BROADCAST_SPLATINSERT]], <16 x i32> undef, <16 x i32> zeroinitializer ; CHECK-NEXT: [[BROADCAST_SPLAT:%.*]] = shufflevector <16 x i32> [[BROADCAST_SPLATINSERT]], <16 x i32> undef, <16 x i32> zeroinitializer
; CHECK-NEXT: [[TMP0:%.*]] = add nsw i64 [[N_VEC]], -16
; CHECK-NEXT: [[TMP1:%.*]] = lshr i64 [[TMP0]], 4
; CHECK-NEXT: [[TMP2:%.*]] = add nuw nsw i64 [[TMP1]], 1
; CHECK-NEXT: [[XTRAITER1:%.*]] = and i64 [[TMP2]], 1
; CHECK-NEXT: [[TMP3:%.*]] = icmp ult i64 [[TMP1]], 1
; CHECK-NEXT: br i1 [[TMP3]], label [[MIDDLE_BLOCK_UNR_LCSSA:%.*]], label [[VECTOR_PH_NEW:%.*]]
; CHECK: vector.ph.new:
; CHECK-NEXT: [[UNROLL_ITER:%.*]] = sub i64 [[TMP2]], [[XTRAITER1]]
; CHECK-NEXT: br label [[VECTOR_BODY:%.*]] ; CHECK-NEXT: br label [[VECTOR_BODY:%.*]]
; CHECK: vector.body: ; CHECK: vector.body:
; CHECK-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH_NEW]] ], [ [[INDEX_NEXT_1:%.*]], [[VECTOR_BODY]] ] ; CHECK-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[VEC_IND12:%.*]] = phi <16 x i32> [ <i32 0, i32 1, i32 2, i32 3, i32 4, i32 5, i32 6, i32 7, i32 8, i32 9, i32 10, i32 11, i32 12, i32 13, i32 14, i32 15>, [[VECTOR_PH_NEW]] ], [ [[VEC_IND_NEXT13_1:%.*]], [[VECTOR_BODY]] ] ; CHECK-NEXT: [[VEC_IND12:%.*]] = phi <16 x i32> [ <i32 0, i32 1, i32 2, i32 3, i32 4, i32 5, i32 6, i32 7, i32 8, i32 9, i32 10, i32 11, i32 12, i32 13, i32 14, i32 15>, [[VECTOR_PH]] ], [ [[VEC_IND_NEXT13:%.*]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[NITER:%.*]] = phi i64 [ [[UNROLL_ITER]], [[VECTOR_PH_NEW]] ], [ [[NITER_NSUB_1:%.*]], [[VECTOR_BODY]] ] ; CHECK-NEXT: [[TMP0:%.*]] = shl <16 x i32> <i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1>, [[VEC_IND12]]
; CHECK-NEXT: [[TMP4:%.*]] = shl <16 x i32> <i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1>, [[VEC_IND12]] ; CHECK-NEXT: [[TMP1:%.*]] = and <16 x i32> [[TMP0]], [[BROADCAST_SPLAT]]
; CHECK-NEXT: [[TMP5:%.*]] = and <16 x i32> [[TMP4]], [[BROADCAST_SPLAT]] ; CHECK-NEXT: [[TMP2:%.*]] = icmp eq <16 x i32> [[TMP1]], zeroinitializer
; CHECK-NEXT: [[TMP6:%.*]] = icmp eq <16 x i32> [[TMP5]], zeroinitializer ; CHECK-NEXT: [[TMP3:%.*]] = select <16 x i1> [[TMP2]], <16 x i8> <i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48>, <16 x i8> <i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49>
; CHECK-NEXT: [[TMP7:%.*]] = select <16 x i1> [[TMP6]], <16 x i8> <i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48>, <16 x i8> <i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49> ; CHECK-NEXT: [[TMP4:%.*]] = getelementptr inbounds i8, i8* [[S:%.*]], i64 [[INDEX]]
; CHECK-NEXT: [[TMP8:%.*]] = getelementptr inbounds i8, i8* [[S:%.*]], i64 [[INDEX]] ; CHECK-NEXT: [[TMP5:%.*]] = bitcast i8* [[TMP4]] to <16 x i8>*
; CHECK-NEXT: [[TMP9:%.*]] = bitcast i8* [[TMP8]] to <16 x i8>* ; CHECK-NEXT: store <16 x i8> [[TMP3]], <16 x i8>* [[TMP5]], align 1
; CHECK-NEXT: store <16 x i8> [[TMP7]], <16 x i8>* [[TMP9]], align 1 ; CHECK-NEXT: [[INDEX_NEXT]] = add i64 [[INDEX]], 16
; CHECK-NEXT: [[INDEX_NEXT:%.*]] = add nuw nsw i64 [[INDEX]], 16 ; CHECK-NEXT: [[VEC_IND_NEXT13]] = add <16 x i32> [[VEC_IND12]], <i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16>
; CHECK-NEXT: [[VEC_IND_NEXT13:%.*]] = add <16 x i32> [[VEC_IND12]], <i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16> ; CHECK-NEXT: [[TMP6:%.*]] = icmp eq i64 [[INDEX_NEXT]], [[N_VEC]]
; CHECK-NEXT: [[NITER_NSUB:%.*]] = sub i64 [[NITER]], 1 ; CHECK-NEXT: br i1 [[TMP6]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]]
; CHECK-NEXT: [[TMP10:%.*]] = shl <16 x i32> <i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1>, [[VEC_IND_NEXT13]]
; CHECK-NEXT: [[TMP11:%.*]] = and <16 x i32> [[TMP10]], [[BROADCAST_SPLAT]]
; CHECK-NEXT: [[TMP12:%.*]] = icmp eq <16 x i32> [[TMP11]], zeroinitializer
; CHECK-NEXT: [[TMP13:%.*]] = select <16 x i1> [[TMP12]], <16 x i8> <i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48>, <16 x i8> <i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49>
; CHECK-NEXT: [[TMP14:%.*]] = getelementptr inbounds i8, i8* [[S]], i64 [[INDEX_NEXT]]
; CHECK-NEXT: [[TMP15:%.*]] = bitcast i8* [[TMP14]] to <16 x i8>*
; CHECK-NEXT: store <16 x i8> [[TMP13]], <16 x i8>* [[TMP15]], align 1
; CHECK-NEXT: [[INDEX_NEXT_1]] = add i64 [[INDEX_NEXT]], 16
; CHECK-NEXT: [[VEC_IND_NEXT13_1]] = add <16 x i32> [[VEC_IND_NEXT13]], <i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16>
; CHECK-NEXT: [[NITER_NSUB_1]] = sub i64 [[NITER_NSUB]], 1
; CHECK-NEXT: [[NITER_NCMP_1:%.*]] = icmp eq i64 [[NITER_NSUB_1]], 0
; CHECK-NEXT: br i1 [[NITER_NCMP_1]], label [[MIDDLE_BLOCK_UNR_LCSSA_LOOPEXIT:%.*]], label [[VECTOR_BODY]]
; CHECK: middle.block.unr-lcssa.loopexit:
; CHECK-NEXT: [[INDEX_UNR_PH:%.*]] = phi i64 [ [[INDEX_NEXT_1]], [[VECTOR_BODY]] ]
; CHECK-NEXT: [[VEC_IND12_UNR_PH:%.*]] = phi <16 x i32> [ [[VEC_IND_NEXT13_1]], [[VECTOR_BODY]] ]
; CHECK-NEXT: br label [[MIDDLE_BLOCK_UNR_LCSSA]]
; CHECK: middle.block.unr-lcssa:
; CHECK-NEXT: [[INDEX_UNR:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_UNR_PH]], [[MIDDLE_BLOCK_UNR_LCSSA_LOOPEXIT]] ]
; CHECK-NEXT: [[VEC_IND12_UNR:%.*]] = phi <16 x i32> [ <i32 0, i32 1, i32 2, i32 3, i32 4, i32 5, i32 6, i32 7, i32 8, i32 9, i32 10, i32 11, i32 12, i32 13, i32 14, i32 15>, [[VECTOR_PH]] ], [ [[VEC_IND12_UNR_PH]], [[MIDDLE_BLOCK_UNR_LCSSA_LOOPEXIT]] ]
; CHECK-NEXT: [[LCMP_MOD2:%.*]] = icmp ne i64 [[XTRAITER1]], 0
; CHECK-NEXT: br i1 [[LCMP_MOD2]], label [[VECTOR_BODY_EPIL_PREHEADER:%.*]], label [[MIDDLE_BLOCK:%.*]]
; CHECK: vector.body.epil.preheader:
; CHECK-NEXT: br label [[VECTOR_BODY_EPIL:%.*]]
; CHECK: vector.body.epil:
; CHECK-NEXT: [[INDEX_EPIL:%.*]] = phi i64 [ [[INDEX_UNR]], [[VECTOR_BODY_EPIL_PREHEADER]] ]
; CHECK-NEXT: [[VEC_IND12_EPIL:%.*]] = phi <16 x i32> [ [[VEC_IND12_UNR]], [[VECTOR_BODY_EPIL_PREHEADER]] ]
; CHECK-NEXT: [[TMP16:%.*]] = shl <16 x i32> <i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1>, [[VEC_IND12_EPIL]]
; CHECK-NEXT: [[TMP17:%.*]] = and <16 x i32> [[TMP16]], [[BROADCAST_SPLAT]]
; CHECK-NEXT: [[TMP18:%.*]] = icmp eq <16 x i32> [[TMP17]], zeroinitializer
; CHECK-NEXT: [[TMP19:%.*]] = select <16 x i1> [[TMP18]], <16 x i8> <i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48, i8 48>, <16 x i8> <i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49, i8 49>
; CHECK-NEXT: [[TMP20:%.*]] = getelementptr inbounds i8, i8* [[S]], i64 [[INDEX_EPIL]]
; CHECK-NEXT: [[TMP21:%.*]] = bitcast i8* [[TMP20]] to <16 x i8>*
; CHECK-NEXT: store <16 x i8> [[TMP19]], <16 x i8>* [[TMP21]], align 1
; CHECK-NEXT: [[INDEX_NEXT_EPIL:%.*]] = add i64 [[INDEX_EPIL]], 16
; CHECK-NEXT: [[VEC_IND_NEXT13_EPIL:%.*]] = add <16 x i32> [[VEC_IND12_EPIL]], <i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16>
; CHECK-NEXT: [[TMP22:%.*]] = icmp eq i64 [[INDEX_NEXT_EPIL]], [[N_VEC]]
; CHECK-NEXT: br label [[MIDDLE_BLOCK_EPILOG_LCSSA:%.*]]
; CHECK: middle.block.epilog-lcssa:
; CHECK-NEXT: br label [[MIDDLE_BLOCK]]
; CHECK: middle.block: ; CHECK: middle.block:
; CHECK-NEXT: [[CMP_N:%.*]] = icmp eq i64 [[N_VEC]], [[WIDE_TRIP_COUNT]] ; CHECK-NEXT: [[CMP_N:%.*]] = icmp eq i64 [[N_VEC]], [[WIDE_TRIP_COUNT]]
; CHECK-NEXT: br i1 [[CMP_N]], label [[FOR_END]], label [[FOR_BODY_PREHEADER]] ; CHECK-NEXT: br i1 [[CMP_N]], label [[FOR_END]], label [[FOR_BODY_PREHEADER]]
; CHECK: for.body.preheader: ; CHECK: for.body.preheader:
; CHECK-NEXT: [[INDVARS_IV_PH:%.*]] = phi i64 [ 0, [[FOR_BODY_LR_PH]] ], [ [[N_VEC]], [[MIDDLE_BLOCK]] ] ; CHECK-NEXT: [[INDVARS_IV_PH:%.*]] = phi i64 [ 0, [[FOR_BODY_LR_PH]] ], [ [[N_VEC]], [[MIDDLE_BLOCK]] ]
; CHECK-NEXT: [[TMP23:%.*]] = sub i64 [[WIDE_TRIP_COUNT]], [[INDVARS_IV_PH]] ; CHECK-NEXT: [[TMP7:%.*]] = sub i64 [[WIDE_TRIP_COUNT]], [[INDVARS_IV_PH]]
; CHECK-NEXT: [[TMP24:%.*]] = add i64 [[WIDE_TRIP_COUNT]], -1 ; CHECK-NEXT: [[TMP8:%.*]] = add i64 [[WIDE_TRIP_COUNT]], -1
; CHECK-NEXT: [[TMP25:%.*]] = sub i64 [[TMP24]], [[INDVARS_IV_PH]] ; CHECK-NEXT: [[TMP9:%.*]] = sub i64 [[TMP8]], [[INDVARS_IV_PH]]
; CHECK-NEXT: [[XTRAITER:%.*]] = and i64 [[TMP23]], 7 ; CHECK-NEXT: [[XTRAITER:%.*]] = and i64 [[TMP7]], 7
; CHECK-NEXT: [[LCMP_MOD:%.*]] = icmp ne i64 [[XTRAITER]], 0 ; CHECK-NEXT: [[LCMP_MOD:%.*]] = icmp ne i64 [[XTRAITER]], 0
; CHECK-NEXT: br i1 [[LCMP_MOD]], label [[FOR_BODY_PROL_PREHEADER:%.*]], label [[FOR_BODY_PROL_LOOPEXIT:%.*]] ; CHECK-NEXT: br i1 [[LCMP_MOD]], label [[FOR_BODY_PROL_PREHEADER:%.*]], label [[FOR_BODY_PROL_LOOPEXIT:%.*]]
; CHECK: for.body.prol.preheader: ; CHECK: for.body.prol.preheader:
; CHECK-NEXT: br label [[FOR_BODY_PROL:%.*]] ; CHECK-NEXT: br label [[FOR_BODY_PROL:%.*]]
; CHECK: for.body.prol: ; CHECK: for.body.prol:
; CHECK-NEXT: [[INDVARS_IV_PROL:%.*]] = phi i64 [ [[INDVARS_IV_NEXT_PROL:%.*]], [[FOR_BODY_PROL]] ], [ [[INDVARS_IV_PH]], [[FOR_BODY_PROL_PREHEADER]] ] ; CHECK-NEXT: [[INDVARS_IV_PROL:%.*]] = phi i64 [ [[INDVARS_IV_NEXT_PROL:%.*]], [[FOR_BODY_PROL]] ], [ [[INDVARS_IV_PH]], [[FOR_BODY_PROL_PREHEADER]] ]
; CHECK-NEXT: [[PROL_ITER:%.*]] = phi i64 [ [[XTRAITER]], [[FOR_BODY_PROL_PREHEADER]] ], [ [[PROL_ITER_SUB:%.*]], [[FOR_BODY_PROL]] ] ; CHECK-NEXT: [[PROL_ITER:%.*]] = phi i64 [ [[XTRAITER]], [[FOR_BODY_PROL_PREHEADER]] ], [ [[PROL_ITER_SUB:%.*]], [[FOR_BODY_PROL]] ]
; CHECK-NEXT: [[TMP26:%.*]] = trunc i64 [[INDVARS_IV_PROL]] to i32 ; CHECK-NEXT: [[TMP10:%.*]] = trunc i64 [[INDVARS_IV_PROL]] to i32
; CHECK-NEXT: [[SHL_PROL:%.*]] = shl i32 1, [[TMP26]] ; CHECK-NEXT: [[SHL_PROL:%.*]] = shl i32 1, [[TMP10]]
; CHECK-NEXT: [[AND_PROL:%.*]] = and i32 [[SHL_PROL]], [[X]] ; CHECK-NEXT: [[AND_PROL:%.*]] = and i32 [[SHL_PROL]], [[X]]
; CHECK-NEXT: [[TOBOOL_PROL:%.*]] = icmp eq i32 [[AND_PROL]], 0 ; CHECK-NEXT: [[TOBOOL_PROL:%.*]] = icmp eq i32 [[AND_PROL]], 0
; CHECK-NEXT: [[CONV_PROL:%.*]] = select i1 [[TOBOOL_PROL]], i8 48, i8 49 ; CHECK-NEXT: [[CONV_PROL:%.*]] = select i1 [[TOBOOL_PROL]], i8 48, i8 49
; CHECK-NEXT: [[ARRAYIDX_PROL:%.*]] = getelementptr inbounds i8, i8* [[S]], i64 [[INDVARS_IV_PROL]] ; CHECK-NEXT: [[ARRAYIDX_PROL:%.*]] = getelementptr inbounds i8, i8* [[S]], i64 [[INDVARS_IV_PROL]]
; CHECK-NEXT: store i8 [[CONV_PROL]], i8* [[ARRAYIDX_PROL]], align 1 ; CHECK-NEXT: store i8 [[CONV_PROL]], i8* [[ARRAYIDX_PROL]], align 1
; CHECK-NEXT: [[INDVARS_IV_NEXT_PROL]] = add nuw nsw i64 [[INDVARS_IV_PROL]], 1 ; CHECK-NEXT: [[INDVARS_IV_NEXT_PROL]] = add nuw nsw i64 [[INDVARS_IV_PROL]], 1
; CHECK-NEXT: [[EXITCOND_PROL:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT_PROL]], [[WIDE_TRIP_COUNT]] ; CHECK-NEXT: [[EXITCOND_PROL:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT_PROL]], [[WIDE_TRIP_COUNT]]
; CHECK-NEXT: [[PROL_ITER_SUB]] = sub i64 [[PROL_ITER]], 1 ; CHECK-NEXT: [[PROL_ITER_SUB]] = sub i64 [[PROL_ITER]], 1
; CHECK-NEXT: [[PROL_ITER_CMP:%.*]] = icmp ne i64 [[PROL_ITER_SUB]], 0 ; CHECK-NEXT: [[PROL_ITER_CMP:%.*]] = icmp ne i64 [[PROL_ITER_SUB]], 0
; CHECK-NEXT: br i1 [[PROL_ITER_CMP]], label [[FOR_BODY_PROL]], label [[FOR_BODY_PROL_LOOPEXIT_UNR_LCSSA:%.*]], !llvm.loop !0 ; CHECK-NEXT: br i1 [[PROL_ITER_CMP]], label [[FOR_BODY_PROL]], label [[FOR_BODY_PROL_LOOPEXIT_UNR_LCSSA:%.*]], !llvm.loop !0
; CHECK: for.body.prol.loopexit.unr-lcssa: ; CHECK: for.body.prol.loopexit.unr-lcssa:
; CHECK-NEXT: [[INDVARS_IV_UNR_PH:%.*]] = phi i64 [ [[INDVARS_IV_NEXT_PROL]], [[FOR_BODY_PROL]] ] ; CHECK-NEXT: [[INDVARS_IV_UNR_PH:%.*]] = phi i64 [ [[INDVARS_IV_NEXT_PROL]], [[FOR_BODY_PROL]] ]
; CHECK-NEXT: br label [[FOR_BODY_PROL_LOOPEXIT]] ; CHECK-NEXT: br label [[FOR_BODY_PROL_LOOPEXIT]]
; CHECK: for.body.prol.loopexit: ; CHECK: for.body.prol.loopexit:
; CHECK-NEXT: [[INDVARS_IV_UNR:%.*]] = phi i64 [ [[INDVARS_IV_PH]], [[FOR_BODY_PREHEADER]] ], [ [[INDVARS_IV_UNR_PH]], [[FOR_BODY_PROL_LOOPEXIT_UNR_LCSSA]] ] ; CHECK-NEXT: [[INDVARS_IV_UNR:%.*]] = phi i64 [ [[INDVARS_IV_PH]], [[FOR_BODY_PREHEADER]] ], [ [[INDVARS_IV_UNR_PH]], [[FOR_BODY_PROL_LOOPEXIT_UNR_LCSSA]] ]
; CHECK-NEXT: [[TMP27:%.*]] = icmp ult i64 [[TMP25]], 7 ; CHECK-NEXT: [[TMP11:%.*]] = icmp ult i64 [[TMP9]], 7
; CHECK-NEXT: br i1 [[TMP27]], label [[FOR_END_LOOPEXIT:%.*]], label [[FOR_BODY_PREHEADER_NEW:%.*]] ; CHECK-NEXT: br i1 [[TMP11]], label [[FOR_END_LOOPEXIT:%.*]], label [[FOR_BODY_PREHEADER_NEW:%.*]]
; CHECK: for.body.preheader.new: ; CHECK: for.body.preheader.new:
; CHECK-NEXT: br label [[FOR_BODY:%.*]] ; CHECK-NEXT: br label [[FOR_BODY:%.*]]
; CHECK: for.body: ; CHECK: for.body:
; CHECK-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[INDVARS_IV_UNR]], [[FOR_BODY_PREHEADER_NEW]] ], [ [[INDVARS_IV_NEXT_7:%.*]], [[FOR_BODY]] ] ; CHECK-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[INDVARS_IV_UNR]], [[FOR_BODY_PREHEADER_NEW]] ], [ [[INDVARS_IV_NEXT_7:%.*]], [[FOR_BODY]] ]
; CHECK-NEXT: [[TMP28:%.*]] = trunc i64 [[INDVARS_IV]] to i32 ; CHECK-NEXT: [[TMP12:%.*]] = trunc i64 [[INDVARS_IV]] to i32
; CHECK-NEXT: [[SHL:%.*]] = shl i32 1, [[TMP28]] ; CHECK-NEXT: [[SHL:%.*]] = shl i32 1, [[TMP12]]
; CHECK-NEXT: [[AND:%.*]] = and i32 [[SHL]], [[X]] ; CHECK-NEXT: [[AND:%.*]] = and i32 [[SHL]], [[X]]
; CHECK-NEXT: [[TOBOOL:%.*]] = icmp eq i32 [[AND]], 0 ; CHECK-NEXT: [[TOBOOL:%.*]] = icmp eq i32 [[AND]], 0
; CHECK-NEXT: [[CONV:%.*]] = select i1 [[TOBOOL]], i8 48, i8 49 ; CHECK-NEXT: [[CONV:%.*]] = select i1 [[TOBOOL]], i8 48, i8 49
; CHECK-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds i8, i8* [[S]], i64 [[INDVARS_IV]] ; CHECK-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds i8, i8* [[S]], i64 [[INDVARS_IV]]
; CHECK-NEXT: store i8 [[CONV]], i8* [[ARRAYIDX]], align 1 ; CHECK-NEXT: store i8 [[CONV]], i8* [[ARRAYIDX]], align 1
; CHECK-NEXT: [[INDVARS_IV_NEXT:%.*]] = add nuw nsw i64 [[INDVARS_IV]], 1 ; CHECK-NEXT: [[INDVARS_IV_NEXT:%.*]] = add nuw nsw i64 [[INDVARS_IV]], 1
; CHECK-NEXT: [[TMP29:%.*]] = trunc i64 [[INDVARS_IV_NEXT]] to i32 ; CHECK-NEXT: [[TMP13:%.*]] = trunc i64 [[INDVARS_IV_NEXT]] to i32
; CHECK-NEXT: [[SHL_1:%.*]] = shl i32 1, [[TMP29]] ; CHECK-NEXT: [[SHL_1:%.*]] = shl i32 1, [[TMP13]]
; CHECK-NEXT: [[AND_1:%.*]] = and i32 [[SHL_1]], [[X]] ; CHECK-NEXT: [[AND_1:%.*]] = and i32 [[SHL_1]], [[X]]
; CHECK-NEXT: [[TOBOOL_1:%.*]] = icmp eq i32 [[AND_1]], 0 ; CHECK-NEXT: [[TOBOOL_1:%.*]] = icmp eq i32 [[AND_1]], 0
; CHECK-NEXT: [[CONV_1:%.*]] = select i1 [[TOBOOL_1]], i8 48, i8 49 ; CHECK-NEXT: [[CONV_1:%.*]] = select i1 [[TOBOOL_1]], i8 48, i8 49
; CHECK-NEXT: [[ARRAYIDX_1:%.*]] = getelementptr inbounds i8, i8* [[S]], i64 [[INDVARS_IV_NEXT]] ; CHECK-NEXT: [[ARRAYIDX_1:%.*]] = getelementptr inbounds i8, i8* [[S]], i64 [[INDVARS_IV_NEXT]]
; CHECK-NEXT: store i8 [[CONV_1]], i8* [[ARRAYIDX_1]], align 1 ; CHECK-NEXT: store i8 [[CONV_1]], i8* [[ARRAYIDX_1]], align 1
; CHECK-NEXT: [[INDVARS_IV_NEXT_1:%.*]] = add nuw nsw i64 [[INDVARS_IV_NEXT]], 1 ; CHECK-NEXT: [[INDVARS_IV_NEXT_1:%.*]] = add nuw nsw i64 [[INDVARS_IV_NEXT]], 1
; CHECK-NEXT: [[TMP30:%.*]] = trunc i64 [[INDVARS_IV_NEXT_1]] to i32 ; CHECK-NEXT: [[TMP14:%.*]] = trunc i64 [[INDVARS_IV_NEXT_1]] to i32
; CHECK-NEXT: [[SHL_2:%.*]] = shl i32 1, [[TMP30]] ; CHECK-NEXT: [[SHL_2:%.*]] = shl i32 1, [[TMP14]]
; CHECK-NEXT: [[AND_2:%.*]] = and i32 [[SHL_2]], [[X]] ; CHECK-NEXT: [[AND_2:%.*]] = and i32 [[SHL_2]], [[X]]
; CHECK-NEXT: [[TOBOOL_2:%.*]] = icmp eq i32 [[AND_2]], 0 ; CHECK-NEXT: [[TOBOOL_2:%.*]] = icmp eq i32 [[AND_2]], 0
; CHECK-NEXT: [[CONV_2:%.*]] = select i1 [[TOBOOL_2]], i8 48, i8 49 ; CHECK-NEXT: [[CONV_2:%.*]] = select i1 [[TOBOOL_2]], i8 48, i8 49
; CHECK-NEXT: [[ARRAYIDX_2:%.*]] = getelementptr inbounds i8, i8* [[S]], i64 [[INDVARS_IV_NEXT_1]] ; CHECK-NEXT: [[ARRAYIDX_2:%.*]] = getelementptr inbounds i8, i8* [[S]], i64 [[INDVARS_IV_NEXT_1]]
; CHECK-NEXT: store i8 [[CONV_2]], i8* [[ARRAYIDX_2]], align 1 ; CHECK-NEXT: store i8 [[CONV_2]], i8* [[ARRAYIDX_2]], align 1
; CHECK-NEXT: [[INDVARS_IV_NEXT_2:%.*]] = add nuw nsw i64 [[INDVARS_IV_NEXT_1]], 1 ; CHECK-NEXT: [[INDVARS_IV_NEXT_2:%.*]] = add nuw nsw i64 [[INDVARS_IV_NEXT_1]], 1
; CHECK-NEXT: [[TMP31:%.*]] = trunc i64 [[INDVARS_IV_NEXT_2]] to i32 ; CHECK-NEXT: [[TMP15:%.*]] = trunc i64 [[INDVARS_IV_NEXT_2]] to i32
; CHECK-NEXT: [[SHL_3:%.*]] = shl i32 1, [[TMP31]] ; CHECK-NEXT: [[SHL_3:%.*]] = shl i32 1, [[TMP15]]
; CHECK-NEXT: [[AND_3:%.*]] = and i32 [[SHL_3]], [[X]] ; CHECK-NEXT: [[AND_3:%.*]] = and i32 [[SHL_3]], [[X]]
; CHECK-NEXT: [[TOBOOL_3:%.*]] = icmp eq i32 [[AND_3]], 0 ; CHECK-NEXT: [[TOBOOL_3:%.*]] = icmp eq i32 [[AND_3]], 0
; CHECK-NEXT: [[CONV_3:%.*]] = select i1 [[TOBOOL_3]], i8 48, i8 49 ; CHECK-NEXT: [[CONV_3:%.*]] = select i1 [[TOBOOL_3]], i8 48, i8 49
; CHECK-NEXT: [[ARRAYIDX_3:%.*]] = getelementptr inbounds i8, i8* [[S]], i64 [[INDVARS_IV_NEXT_2]] ; CHECK-NEXT: [[ARRAYIDX_3:%.*]] = getelementptr inbounds i8, i8* [[S]], i64 [[INDVARS_IV_NEXT_2]]
; CHECK-NEXT: store i8 [[CONV_3]], i8* [[ARRAYIDX_3]], align 1 ; CHECK-NEXT: store i8 [[CONV_3]], i8* [[ARRAYIDX_3]], align 1
; CHECK-NEXT: [[INDVARS_IV_NEXT_3:%.*]] = add nuw nsw i64 [[INDVARS_IV_NEXT_2]], 1 ; CHECK-NEXT: [[INDVARS_IV_NEXT_3:%.*]] = add nuw nsw i64 [[INDVARS_IV_NEXT_2]], 1
; CHECK-NEXT: [[TMP32:%.*]] = trunc i64 [[INDVARS_IV_NEXT_3]] to i32 ; CHECK-NEXT: [[TMP16:%.*]] = trunc i64 [[INDVARS_IV_NEXT_3]] to i32
; CHECK-NEXT: [[SHL_4:%.*]] = shl i32 1, [[TMP32]] ; CHECK-NEXT: [[SHL_4:%.*]] = shl i32 1, [[TMP16]]
; CHECK-NEXT: [[AND_4:%.*]] = and i32 [[SHL_4]], [[X]] ; CHECK-NEXT: [[AND_4:%.*]] = and i32 [[SHL_4]], [[X]]
; CHECK-NEXT: [[TOBOOL_4:%.*]] = icmp eq i32 [[AND_4]], 0 ; CHECK-NEXT: [[TOBOOL_4:%.*]] = icmp eq i32 [[AND_4]], 0
; CHECK-NEXT: [[CONV_4:%.*]] = select i1 [[TOBOOL_4]], i8 48, i8 49 ; CHECK-NEXT: [[CONV_4:%.*]] = select i1 [[TOBOOL_4]], i8 48, i8 49
; CHECK-NEXT: [[ARRAYIDX_4:%.*]] = getelementptr inbounds i8, i8* [[S]], i64 [[INDVARS_IV_NEXT_3]] ; CHECK-NEXT: [[ARRAYIDX_4:%.*]] = getelementptr inbounds i8, i8* [[S]], i64 [[INDVARS_IV_NEXT_3]]
; CHECK-NEXT: store i8 [[CONV_4]], i8* [[ARRAYIDX_4]], align 1 ; CHECK-NEXT: store i8 [[CONV_4]], i8* [[ARRAYIDX_4]], align 1
; CHECK-NEXT: [[INDVARS_IV_NEXT_4:%.*]] = add nuw nsw i64 [[INDVARS_IV_NEXT_3]], 1 ; CHECK-NEXT: [[INDVARS_IV_NEXT_4:%.*]] = add nuw nsw i64 [[INDVARS_IV_NEXT_3]], 1
; CHECK-NEXT: [[TMP33:%.*]] = trunc i64 [[INDVARS_IV_NEXT_4]] to i32 ; CHECK-NEXT: [[TMP17:%.*]] = trunc i64 [[INDVARS_IV_NEXT_4]] to i32
; CHECK-NEXT: [[SHL_5:%.*]] = shl i32 1, [[TMP33]] ; CHECK-NEXT: [[SHL_5:%.*]] = shl i32 1, [[TMP17]]
; CHECK-NEXT: [[AND_5:%.*]] = and i32 [[SHL_5]], [[X]] ; CHECK-NEXT: [[AND_5:%.*]] = and i32 [[SHL_5]], [[X]]
; CHECK-NEXT: [[TOBOOL_5:%.*]] = icmp eq i32 [[AND_5]], 0 ; CHECK-NEXT: [[TOBOOL_5:%.*]] = icmp eq i32 [[AND_5]], 0
; CHECK-NEXT: [[CONV_5:%.*]] = select i1 [[TOBOOL_5]], i8 48, i8 49 ; CHECK-NEXT: [[CONV_5:%.*]] = select i1 [[TOBOOL_5]], i8 48, i8 49
; CHECK-NEXT: [[ARRAYIDX_5:%.*]] = getelementptr inbounds i8, i8* [[S]], i64 [[INDVARS_IV_NEXT_4]] ; CHECK-NEXT: [[ARRAYIDX_5:%.*]] = getelementptr inbounds i8, i8* [[S]], i64 [[INDVARS_IV_NEXT_4]]
; CHECK-NEXT: store i8 [[CONV_5]], i8* [[ARRAYIDX_5]], align 1 ; CHECK-NEXT: store i8 [[CONV_5]], i8* [[ARRAYIDX_5]], align 1
; CHECK-NEXT: [[INDVARS_IV_NEXT_5:%.*]] = add nuw nsw i64 [[INDVARS_IV_NEXT_4]], 1 ; CHECK-NEXT: [[INDVARS_IV_NEXT_5:%.*]] = add nuw nsw i64 [[INDVARS_IV_NEXT_4]], 1
; CHECK-NEXT: [[TMP34:%.*]] = trunc i64 [[INDVARS_IV_NEXT_5]] to i32 ; CHECK-NEXT: [[TMP18:%.*]] = trunc i64 [[INDVARS_IV_NEXT_5]] to i32
; CHECK-NEXT: [[SHL_6:%.*]] = shl i32 1, [[TMP34]] ; CHECK-NEXT: [[SHL_6:%.*]] = shl i32 1, [[TMP18]]
; CHECK-NEXT: [[AND_6:%.*]] = and i32 [[SHL_6]], [[X]] ; CHECK-NEXT: [[AND_6:%.*]] = and i32 [[SHL_6]], [[X]]
; CHECK-NEXT: [[TOBOOL_6:%.*]] = icmp eq i32 [[AND_6]], 0 ; CHECK-NEXT: [[TOBOOL_6:%.*]] = icmp eq i32 [[AND_6]], 0
; CHECK-NEXT: [[CONV_6:%.*]] = select i1 [[TOBOOL_6]], i8 48, i8 49 ; CHECK-NEXT: [[CONV_6:%.*]] = select i1 [[TOBOOL_6]], i8 48, i8 49
; CHECK-NEXT: [[ARRAYIDX_6:%.*]] = getelementptr inbounds i8, i8* [[S]], i64 [[INDVARS_IV_NEXT_5]] ; CHECK-NEXT: [[ARRAYIDX_6:%.*]] = getelementptr inbounds i8, i8* [[S]], i64 [[INDVARS_IV_NEXT_5]]
; CHECK-NEXT: store i8 [[CONV_6]], i8* [[ARRAYIDX_6]], align 1 ; CHECK-NEXT: store i8 [[CONV_6]], i8* [[ARRAYIDX_6]], align 1
; CHECK-NEXT: [[INDVARS_IV_NEXT_6:%.*]] = add nuw nsw i64 [[INDVARS_IV_NEXT_5]], 1 ; CHECK-NEXT: [[INDVARS_IV_NEXT_6:%.*]] = add nuw nsw i64 [[INDVARS_IV_NEXT_5]], 1
; CHECK-NEXT: [[TMP35:%.*]] = trunc i64 [[INDVARS_IV_NEXT_6]] to i32 ; CHECK-NEXT: [[TMP19:%.*]] = trunc i64 [[INDVARS_IV_NEXT_6]] to i32
; CHECK-NEXT: [[SHL_7:%.*]] = shl i32 1, [[TMP35]] ; CHECK-NEXT: [[SHL_7:%.*]] = shl i32 1, [[TMP19]]
; CHECK-NEXT: [[AND_7:%.*]] = and i32 [[SHL_7]], [[X]] ; CHECK-NEXT: [[AND_7:%.*]] = and i32 [[SHL_7]], [[X]]
; CHECK-NEXT: [[TOBOOL_7:%.*]] = icmp eq i32 [[AND_7]], 0 ; CHECK-NEXT: [[TOBOOL_7:%.*]] = icmp eq i32 [[AND_7]], 0
; CHECK-NEXT: [[CONV_7:%.*]] = select i1 [[TOBOOL_7]], i8 48, i8 49 ; CHECK-NEXT: [[CONV_7:%.*]] = select i1 [[TOBOOL_7]], i8 48, i8 49
; CHECK-NEXT: [[ARRAYIDX_7:%.*]] = getelementptr inbounds i8, i8* [[S]], i64 [[INDVARS_IV_NEXT_6]] ; CHECK-NEXT: [[ARRAYIDX_7:%.*]] = getelementptr inbounds i8, i8* [[S]], i64 [[INDVARS_IV_NEXT_6]]
; CHECK-NEXT: store i8 [[CONV_7]], i8* [[ARRAYIDX_7]], align 1 ; CHECK-NEXT: store i8 [[CONV_7]], i8* [[ARRAYIDX_7]], align 1
; CHECK-NEXT: [[INDVARS_IV_NEXT_7]] = add nuw nsw i64 [[INDVARS_IV_NEXT_6]], 1 ; CHECK-NEXT: [[INDVARS_IV_NEXT_7]] = add nuw nsw i64 [[INDVARS_IV_NEXT_6]], 1
; CHECK-NEXT: [[EXITCOND_7:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT_7]], [[WIDE_TRIP_COUNT]] ; CHECK-NEXT: [[EXITCOND_7:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT_7]], [[WIDE_TRIP_COUNT]]
; CHECK-NEXT: br i1 [[EXITCOND_7]], label [[FOR_END_LOOPEXIT_UNR_LCSSA:%.*]], label [[FOR_BODY]] ; CHECK-NEXT: br i1 [[EXITCOND_7]], label [[FOR_END_LOOPEXIT_UNR_LCSSA:%.*]], label [[FOR_BODY]]
▲ Show 20 Lines • Show All 68 Lines • Show Last 20 Lines

llvm/test/Transforms/LoopVectorize/X86/float-induction-x86.ll

Show All 22 Lines
; AUTO_VEC-NEXT: [[ZEXT:%.*]] = zext i32 [[N]] to i64 ; AUTO_VEC-NEXT: [[ZEXT:%.*]] = zext i32 [[N]] to i64
; AUTO_VEC-NEXT: [[MIN_ITERS_CHECK:%.*]] = icmp ult i32 [[N]], 32 ; AUTO_VEC-NEXT: [[MIN_ITERS_CHECK:%.*]] = icmp ult i32 [[N]], 32
; AUTO_VEC-NEXT: br i1 [[MIN_ITERS_CHECK]], label [[FOR_BODY:%.*]], label [[VECTOR_PH:%.*]] ; AUTO_VEC-NEXT: br i1 [[MIN_ITERS_CHECK]], label [[FOR_BODY:%.*]], label [[VECTOR_PH:%.*]]
; AUTO_VEC: vector.ph: ; AUTO_VEC: vector.ph:
; AUTO_VEC-NEXT: [[N_VEC:%.*]] = and i64 [[ZEXT]], 4294967264 ; AUTO_VEC-NEXT: [[N_VEC:%.*]] = and i64 [[ZEXT]], 4294967264
; AUTO_VEC-NEXT: [[CAST_CRD:%.*]] = sitofp i64 [[N_VEC]] to float ; AUTO_VEC-NEXT: [[CAST_CRD:%.*]] = sitofp i64 [[N_VEC]] to float
; AUTO_VEC-NEXT: [[TMP0:%.*]] = fmul fast float [[CAST_CRD]], 5.000000e-01 ; AUTO_VEC-NEXT: [[TMP0:%.*]] = fmul fast float [[CAST_CRD]], 5.000000e-01
; AUTO_VEC-NEXT: [[IND_END:%.*]] = fadd fast float [[TMP0]], 1.000000e+00 ; AUTO_VEC-NEXT: [[IND_END:%.*]] = fadd fast float [[TMP0]], 1.000000e+00
; AUTO_VEC-NEXT: [[TMP1:%.*]] = add nsw i64 [[N_VEC]], -32
; AUTO_VEC-NEXT: [[TMP2:%.*]] = lshr exact i64 [[TMP1]], 5
; AUTO_VEC-NEXT: [[TMP3:%.*]] = add nuw nsw i64 [[TMP2]], 1
; AUTO_VEC-NEXT: [[XTRAITER:%.*]] = and i64 [[TMP3]], 3
; AUTO_VEC-NEXT: [[TMP4:%.*]] = icmp ult i64 [[TMP1]], 96
; AUTO_VEC-NEXT: br i1 [[TMP4]], label [[MIDDLE_BLOCK_UNR_LCSSA:%.*]], label [[VECTOR_PH_NEW:%.*]]
; AUTO_VEC: vector.ph.new:
; AUTO_VEC-NEXT: [[UNROLL_ITER:%.*]] = and i64 [[TMP3]], 1152921504606846972
; AUTO_VEC-NEXT: br label [[VECTOR_BODY:%.*]] ; AUTO_VEC-NEXT: br label [[VECTOR_BODY:%.*]]
; AUTO_VEC: vector.body: ; AUTO_VEC: vector.body:
; AUTO_VEC-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH_NEW]] ], [ [[INDEX_NEXT_3:%.*]], [[VECTOR_BODY]] ] ; AUTO_VEC-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ]
; AUTO_VEC-NEXT: [[VEC_IND:%.*]] = phi <8 x float> [ <float 1.000000e+00, float 1.500000e+00, float 2.000000e+00, float 2.500000e+00, float 3.000000e+00, float 3.500000e+00, float 4.000000e+00, float 4.500000e+00>, [[VECTOR_PH_NEW]] ], [ [[VEC_IND_NEXT_3:%.*]], [[VECTOR_BODY]] ] ; AUTO_VEC-NEXT: [[VEC_IND:%.*]] = phi <8 x float> [ <float 1.000000e+00, float 1.500000e+00, float 2.000000e+00, float 2.500000e+00, float 3.000000e+00, float 3.500000e+00, float 4.000000e+00, float 4.500000e+00>, [[VECTOR_PH]] ], [ [[VEC_IND_NEXT:%.*]], [[VECTOR_BODY]] ]
; AUTO_VEC-NEXT: [[NITER:%.*]] = phi i64 [ [[UNROLL_ITER]], [[VECTOR_PH_NEW]] ], [ [[NITER_NSUB_3:%.*]], [[VECTOR_BODY]] ]
; AUTO_VEC-NEXT: [[STEP_ADD:%.*]] = fadd fast <8 x float> [[VEC_IND]], <float 4.000000e+00, float 4.000000e+00, float 4.000000e+00, float 4.000000e+00, float 4.000000e+00, float 4.000000e+00, float 4.000000e+00, float 4.000000e+00> ; AUTO_VEC-NEXT: [[STEP_ADD:%.*]] = fadd fast <8 x float> [[VEC_IND]], <float 4.000000e+00, float 4.000000e+00, float 4.000000e+00, float 4.000000e+00, float 4.000000e+00, float 4.000000e+00, float 4.000000e+00, float 4.000000e+00>
; AUTO_VEC-NEXT: [[STEP_ADD5:%.*]] = fadd fast <8 x float> [[VEC_IND]], <float 8.000000e+00, float 8.000000e+00, float 8.000000e+00, float 8.000000e+00, float 8.000000e+00, float 8.000000e+00, float 8.000000e+00, float 8.000000e+00> ; AUTO_VEC-NEXT: [[STEP_ADD5:%.*]] = fadd fast <8 x float> [[VEC_IND]], <float 8.000000e+00, float 8.000000e+00, float 8.000000e+00, float 8.000000e+00, float 8.000000e+00, float 8.000000e+00, float 8.000000e+00, float 8.000000e+00>
; AUTO_VEC-NEXT: [[STEP_ADD6:%.*]] = fadd fast <8 x float> [[VEC_IND]], <float 1.200000e+01, float 1.200000e+01, float 1.200000e+01, float 1.200000e+01, float 1.200000e+01, float 1.200000e+01, float 1.200000e+01, float 1.200000e+01> ; AUTO_VEC-NEXT: [[STEP_ADD6:%.*]] = fadd fast <8 x float> [[VEC_IND]], <float 1.200000e+01, float 1.200000e+01, float 1.200000e+01, float 1.200000e+01, float 1.200000e+01, float 1.200000e+01, float 1.200000e+01, float 1.200000e+01>
; AUTO_VEC-NEXT: [[TMP5:%.*]] = getelementptr inbounds float, float* [[A:%.*]], i64 [[INDEX]] ; AUTO_VEC-NEXT: [[TMP1:%.*]] = getelementptr inbounds float, float* [[A:%.*]], i64 [[INDEX]]
; AUTO_VEC-NEXT: [[TMP2:%.*]] = bitcast float* [[TMP1]] to <8 x float>*
; AUTO_VEC-NEXT: store <8 x float> [[VEC_IND]], <8 x float>* [[TMP2]], align 4
; AUTO_VEC-NEXT: [[TMP3:%.*]] = getelementptr inbounds float, float* [[TMP1]], i64 8
; AUTO_VEC-NEXT: [[TMP4:%.*]] = bitcast float* [[TMP3]] to <8 x float>*
; AUTO_VEC-NEXT: store <8 x float> [[STEP_ADD]], <8 x float>* [[TMP4]], align 4
; AUTO_VEC-NEXT: [[TMP5:%.*]] = getelementptr inbounds float, float* [[TMP1]], i64 16
; AUTO_VEC-NEXT: [[TMP6:%.*]] = bitcast float* [[TMP5]] to <8 x float>* ; AUTO_VEC-NEXT: [[TMP6:%.*]] = bitcast float* [[TMP5]] to <8 x float>*
; AUTO_VEC-NEXT: store <8 x float> [[VEC_IND]], <8 x float>* [[TMP6]], align 4 ; AUTO_VEC-NEXT: store <8 x float> [[STEP_ADD5]], <8 x float>* [[TMP6]], align 4
; AUTO_VEC-NEXT: [[TMP7:%.*]] = getelementptr inbounds float, float* [[TMP5]], i64 8 ; AUTO_VEC-NEXT: [[TMP7:%.*]] = getelementptr inbounds float, float* [[TMP1]], i64 24
; AUTO_VEC-NEXT: [[TMP8:%.*]] = bitcast float* [[TMP7]] to <8 x float>* ; AUTO_VEC-NEXT: [[TMP8:%.*]] = bitcast float* [[TMP7]] to <8 x float>*
; AUTO_VEC-NEXT: store <8 x float> [[STEP_ADD]], <8 x float>* [[TMP8]], align 4 ; AUTO_VEC-NEXT: store <8 x float> [[STEP_ADD6]], <8 x float>* [[TMP8]], align 4
; AUTO_VEC-NEXT: [[TMP9:%.*]] = getelementptr inbounds float, float* [[TMP5]], i64 16 ; AUTO_VEC-NEXT: [[INDEX_NEXT]] = add i64 [[INDEX]], 32
; AUTO_VEC-NEXT: [[TMP10:%.*]] = bitcast float* [[TMP9]] to <8 x float>* ; AUTO_VEC-NEXT: [[VEC_IND_NEXT]] = fadd fast <8 x float> [[VEC_IND]], <float 1.600000e+01, float 1.600000e+01, float 1.600000e+01, float 1.600000e+01, float 1.600000e+01, float 1.600000e+01, float 1.600000e+01, float 1.600000e+01>
; AUTO_VEC-NEXT: store <8 x float> [[STEP_ADD5]], <8 x float>* [[TMP10]], align 4 ; AUTO_VEC-NEXT: [[TMP9:%.*]] = icmp eq i64 [[INDEX_NEXT]], [[N_VEC]]
; AUTO_VEC-NEXT: [[TMP11:%.*]] = getelementptr inbounds float, float* [[TMP5]], i64 24 ; AUTO_VEC-NEXT: br i1 [[TMP9]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop !0
; AUTO_VEC-NEXT: [[TMP12:%.*]] = bitcast float* [[TMP11]] to <8 x float>*
; AUTO_VEC-NEXT: store <8 x float> [[STEP_ADD6]], <8 x float>* [[TMP12]], align 4
; AUTO_VEC-NEXT: [[INDEX_NEXT:%.*]] = or i64 [[INDEX]], 32
; AUTO_VEC-NEXT: [[VEC_IND_NEXT:%.*]] = fadd fast <8 x float> [[VEC_IND]], <float 1.600000e+01, float 1.600000e+01, float 1.600000e+01, float 1.600000e+01, float 1.600000e+01, float 1.600000e+01, float 1.600000e+01, float 1.600000e+01>
; AUTO_VEC-NEXT: [[STEP_ADD_1:%.*]] = fadd fast <8 x float> [[VEC_IND]], <float 2.000000e+01, float 2.000000e+01, float 2.000000e+01, float 2.000000e+01, float 2.000000e+01, float 2.000000e+01, float 2.000000e+01, float 2.000000e+01>
; AUTO_VEC-NEXT: [[STEP_ADD5_1:%.*]] = fadd fast <8 x float> [[VEC_IND]], <float 2.400000e+01, float 2.400000e+01, float 2.400000e+01, float 2.400000e+01, float 2.400000e+01, float 2.400000e+01, float 2.400000e+01, float 2.400000e+01>
; AUTO_VEC-NEXT: [[STEP_ADD6_1:%.*]] = fadd fast <8 x float> [[VEC_IND]], <float 2.800000e+01, float 2.800000e+01, float 2.800000e+01, float 2.800000e+01, float 2.800000e+01, float 2.800000e+01, float 2.800000e+01, float 2.800000e+01>
; AUTO_VEC-NEXT: [[TMP13:%.*]] = getelementptr inbounds float, float* [[A]], i64 [[INDEX_NEXT]]
; AUTO_VEC-NEXT: [[TMP14:%.*]] = bitcast float* [[TMP13]] to <8 x float>*
; AUTO_VEC-NEXT: store <8 x float> [[VEC_IND_NEXT]], <8 x float>* [[TMP14]], align 4
; AUTO_VEC-NEXT: [[TMP15:%.*]] = getelementptr inbounds float, float* [[TMP13]], i64 8
; AUTO_VEC-NEXT: [[TMP16:%.*]] = bitcast float* [[TMP15]] to <8 x float>*
; AUTO_VEC-NEXT: store <8 x float> [[STEP_ADD_1]], <8 x float>* [[TMP16]], align 4
; AUTO_VEC-NEXT: [[TMP17:%.*]] = getelementptr inbounds float, float* [[TMP13]], i64 16
; AUTO_VEC-NEXT: [[TMP18:%.*]] = bitcast float* [[TMP17]] to <8 x float>*
; AUTO_VEC-NEXT: store <8 x float> [[STEP_ADD5_1]], <8 x float>* [[TMP18]], align 4
; AUTO_VEC-NEXT: [[TMP19:%.*]] = getelementptr inbounds float, float* [[TMP13]], i64 24
; AUTO_VEC-NEXT: [[TMP20:%.*]] = bitcast float* [[TMP19]] to <8 x float>*
; AUTO_VEC-NEXT: store <8 x float> [[STEP_ADD6_1]], <8 x float>* [[TMP20]], align 4
; AUTO_VEC-NEXT: [[INDEX_NEXT_1:%.*]] = or i64 [[INDEX]], 64
; AUTO_VEC-NEXT: [[VEC_IND_NEXT_1:%.*]] = fadd fast <8 x float> [[VEC_IND]], <float 3.200000e+01, float 3.200000e+01, float 3.200000e+01, float 3.200000e+01, float 3.200000e+01, float 3.200000e+01, float 3.200000e+01, float 3.200000e+01>
; AUTO_VEC-NEXT: [[STEP_ADD_2:%.*]] = fadd fast <8 x float> [[VEC_IND]], <float 3.600000e+01, float 3.600000e+01, float 3.600000e+01, float 3.600000e+01, float 3.600000e+01, float 3.600000e+01, float 3.600000e+01, float 3.600000e+01>
; AUTO_VEC-NEXT: [[STEP_ADD5_2:%.*]] = fadd fast <8 x float> [[VEC_IND]], <float 4.000000e+01, float 4.000000e+01, float 4.000000e+01, float 4.000000e+01, float 4.000000e+01, float 4.000000e+01, float 4.000000e+01, float 4.000000e+01>
; AUTO_VEC-NEXT: [[STEP_ADD6_2:%.*]] = fadd fast <8 x float> [[VEC_IND]], <float 4.400000e+01, float 4.400000e+01, float 4.400000e+01, float 4.400000e+01, float 4.400000e+01, float 4.400000e+01, float 4.400000e+01, float 4.400000e+01>
; AUTO_VEC-NEXT: [[TMP21:%.*]] = getelementptr inbounds float, float* [[A]], i64 [[INDEX_NEXT_1]]
; AUTO_VEC-NEXT: [[TMP22:%.*]] = bitcast float* [[TMP21]] to <8 x float>*
; AUTO_VEC-NEXT: store <8 x float> [[VEC_IND_NEXT_1]], <8 x float>* [[TMP22]], align 4
; AUTO_VEC-NEXT: [[TMP23:%.*]] = getelementptr inbounds float, float* [[TMP21]], i64 8
; AUTO_VEC-NEXT: [[TMP24:%.*]] = bitcast float* [[TMP23]] to <8 x float>*
; AUTO_VEC-NEXT: store <8 x float> [[STEP_ADD_2]], <8 x float>* [[TMP24]], align 4
; AUTO_VEC-NEXT: [[TMP25:%.*]] = getelementptr inbounds float, float* [[TMP21]], i64 16
; AUTO_VEC-NEXT: [[TMP26:%.*]] = bitcast float* [[TMP25]] to <8 x float>*
; AUTO_VEC-NEXT: store <8 x float> [[STEP_ADD5_2]], <8 x float>* [[TMP26]], align 4
; AUTO_VEC-NEXT: [[TMP27:%.*]] = getelementptr inbounds float, float* [[TMP21]], i64 24
; AUTO_VEC-NEXT: [[TMP28:%.*]] = bitcast float* [[TMP27]] to <8 x float>*
; AUTO_VEC-NEXT: store <8 x float> [[STEP_ADD6_2]], <8 x float>* [[TMP28]], align 4
; AUTO_VEC-NEXT: [[INDEX_NEXT_2:%.*]] = or i64 [[INDEX]], 96
; AUTO_VEC-NEXT: [[VEC_IND_NEXT_2:%.*]] = fadd fast <8 x float> [[VEC_IND]], <float 4.800000e+01, float 4.800000e+01, float 4.800000e+01, float 4.800000e+01, float 4.800000e+01, float 4.800000e+01, float 4.800000e+01, float 4.800000e+01>
; AUTO_VEC-NEXT: [[STEP_ADD_3:%.*]] = fadd fast <8 x float> [[VEC_IND]], <float 5.200000e+01, float 5.200000e+01, float 5.200000e+01, float 5.200000e+01, float 5.200000e+01, float 5.200000e+01, float 5.200000e+01, float 5.200000e+01>
; AUTO_VEC-NEXT: [[STEP_ADD5_3:%.*]] = fadd fast <8 x float> [[VEC_IND]], <float 5.600000e+01, float 5.600000e+01, float 5.600000e+01, float 5.600000e+01, float 5.600000e+01, float 5.600000e+01, float 5.600000e+01, float 5.600000e+01>
; AUTO_VEC-NEXT: [[STEP_ADD6_3:%.*]] = fadd fast <8 x float> [[VEC_IND]], <float 6.000000e+01, float 6.000000e+01, float 6.000000e+01, float 6.000000e+01, float 6.000000e+01, float 6.000000e+01, float 6.000000e+01, float 6.000000e+01>
; AUTO_VEC-NEXT: [[TMP29:%.*]] = getelementptr inbounds float, float* [[A]], i64 [[INDEX_NEXT_2]]
; AUTO_VEC-NEXT: [[TMP30:%.*]] = bitcast float* [[TMP29]] to <8 x float>*
; AUTO_VEC-NEXT: store <8 x float> [[VEC_IND_NEXT_2]], <8 x float>* [[TMP30]], align 4
; AUTO_VEC-NEXT: [[TMP31:%.*]] = getelementptr inbounds float, float* [[TMP29]], i64 8
; AUTO_VEC-NEXT: [[TMP32:%.*]] = bitcast float* [[TMP31]] to <8 x float>*
; AUTO_VEC-NEXT: store <8 x float> [[STEP_ADD_3]], <8 x float>* [[TMP32]], align 4
; AUTO_VEC-NEXT: [[TMP33:%.*]] = getelementptr inbounds float, float* [[TMP29]], i64 16
; AUTO_VEC-NEXT: [[TMP34:%.*]] = bitcast float* [[TMP33]] to <8 x float>*
; AUTO_VEC-NEXT: store <8 x float> [[STEP_ADD5_3]], <8 x float>* [[TMP34]], align 4
; AUTO_VEC-NEXT: [[TMP35:%.*]] = getelementptr inbounds float, float* [[TMP29]], i64 24
; AUTO_VEC-NEXT: [[TMP36:%.*]] = bitcast float* [[TMP35]] to <8 x float>*
; AUTO_VEC-NEXT: store <8 x float> [[STEP_ADD6_3]], <8 x float>* [[TMP36]], align 4
; AUTO_VEC-NEXT: [[INDEX_NEXT_3]] = add i64 [[INDEX]], 128
; AUTO_VEC-NEXT: [[VEC_IND_NEXT_3]] = fadd fast <8 x float> [[VEC_IND]], <float 6.400000e+01, float 6.400000e+01, float 6.400000e+01, float 6.400000e+01, float 6.400000e+01, float 6.400000e+01, float 6.400000e+01, float 6.400000e+01>
; AUTO_VEC-NEXT: [[NITER_NSUB_3]] = add i64 [[NITER]], -4
; AUTO_VEC-NEXT: [[NITER_NCMP_3:%.*]] = icmp eq i64 [[NITER_NSUB_3]], 0
; AUTO_VEC-NEXT: br i1 [[NITER_NCMP_3]], label [[MIDDLE_BLOCK_UNR_LCSSA]], label [[VECTOR_BODY]], !llvm.loop !0
; AUTO_VEC: middle.block.unr-lcssa:
; AUTO_VEC-NEXT: [[INDEX_UNR:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT_3]], [[VECTOR_BODY]] ]
; AUTO_VEC-NEXT: [[VEC_IND_UNR:%.*]] = phi <8 x float> [ <float 1.000000e+00, float 1.500000e+00, float 2.000000e+00, float 2.500000e+00, float 3.000000e+00, float 3.500000e+00, float 4.000000e+00, float 4.500000e+00>, [[VECTOR_PH]] ], [ [[VEC_IND_NEXT_3]], [[VECTOR_BODY]] ]
; AUTO_VEC-NEXT: [[LCMP_MOD:%.*]] = icmp eq i64 [[XTRAITER]], 0
; AUTO_VEC-NEXT: br i1 [[LCMP_MOD]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY_EPIL:%.*]]
; AUTO_VEC: vector.body.epil:
; AUTO_VEC-NEXT: [[INDEX_EPIL:%.*]] = phi i64 [ [[INDEX_NEXT_EPIL:%.*]], [[VECTOR_BODY_EPIL]] ], [ [[INDEX_UNR]], [[MIDDLE_BLOCK_UNR_LCSSA]] ]
; AUTO_VEC-NEXT: [[VEC_IND_EPIL:%.*]] = phi <8 x float> [ [[VEC_IND_NEXT_EPIL:%.*]], [[VECTOR_BODY_EPIL]] ], [ [[VEC_IND_UNR]], [[MIDDLE_BLOCK_UNR_LCSSA]] ]
; AUTO_VEC-NEXT: [[EPIL_ITER:%.*]] = phi i64 [ [[EPIL_ITER_SUB:%.*]], [[VECTOR_BODY_EPIL]] ], [ [[XTRAITER]], [[MIDDLE_BLOCK_UNR_LCSSA]] ]
; AUTO_VEC-NEXT: [[STEP_ADD_EPIL:%.*]] = fadd fast <8 x float> [[VEC_IND_EPIL]], <float 4.000000e+00, float 4.000000e+00, float 4.000000e+00, float 4.000000e+00, float 4.000000e+00, float 4.000000e+00, float 4.000000e+00, float 4.000000e+00>
; AUTO_VEC-NEXT: [[STEP_ADD5_EPIL:%.*]] = fadd fast <8 x float> [[VEC_IND_EPIL]], <float 8.000000e+00, float 8.000000e+00, float 8.000000e+00, float 8.000000e+00, float 8.000000e+00, float 8.000000e+00, float 8.000000e+00, float 8.000000e+00>
; AUTO_VEC-NEXT: [[STEP_ADD6_EPIL:%.*]] = fadd fast <8 x float> [[VEC_IND_EPIL]], <float 1.200000e+01, float 1.200000e+01, float 1.200000e+01, float 1.200000e+01, float 1.200000e+01, float 1.200000e+01, float 1.200000e+01, float 1.200000e+01>
; AUTO_VEC-NEXT: [[TMP37:%.*]] = getelementptr inbounds float, float* [[A]], i64 [[INDEX_EPIL]]
; AUTO_VEC-NEXT: [[TMP38:%.*]] = bitcast float* [[TMP37]] to <8 x float>*
; AUTO_VEC-NEXT: store <8 x float> [[VEC_IND_EPIL]], <8 x float>* [[TMP38]], align 4
; AUTO_VEC-NEXT: [[TMP39:%.*]] = getelementptr inbounds float, float* [[TMP37]], i64 8
; AUTO_VEC-NEXT: [[TMP40:%.*]] = bitcast float* [[TMP39]] to <8 x float>*
; AUTO_VEC-NEXT: store <8 x float> [[STEP_ADD_EPIL]], <8 x float>* [[TMP40]], align 4
; AUTO_VEC-NEXT: [[TMP41:%.*]] = getelementptr inbounds float, float* [[TMP37]], i64 16
; AUTO_VEC-NEXT: [[TMP42:%.*]] = bitcast float* [[TMP41]] to <8 x float>*
; AUTO_VEC-NEXT: store <8 x float> [[STEP_ADD5_EPIL]], <8 x float>* [[TMP42]], align 4
; AUTO_VEC-NEXT: [[TMP43:%.*]] = getelementptr inbounds float, float* [[TMP37]], i64 24
; AUTO_VEC-NEXT: [[TMP44:%.*]] = bitcast float* [[TMP43]] to <8 x float>*
; AUTO_VEC-NEXT: store <8 x float> [[STEP_ADD6_EPIL]], <8 x float>* [[TMP44]], align 4
; AUTO_VEC-NEXT: [[INDEX_NEXT_EPIL]] = add i64 [[INDEX_EPIL]], 32
; AUTO_VEC-NEXT: [[VEC_IND_NEXT_EPIL]] = fadd fast <8 x float> [[VEC_IND_EPIL]], <float 1.600000e+01, float 1.600000e+01, float 1.600000e+01, float 1.600000e+01, float 1.600000e+01, float 1.600000e+01, float 1.600000e+01, float 1.600000e+01>
; AUTO_VEC-NEXT: [[EPIL_ITER_SUB]] = add i64 [[EPIL_ITER]], -1
; AUTO_VEC-NEXT: [[EPIL_ITER_CMP:%.*]] = icmp eq i64 [[EPIL_ITER_SUB]], 0
; AUTO_VEC-NEXT: br i1 [[EPIL_ITER_CMP]], label [[MIDDLE_BLOCK]], label [[VECTOR_BODY_EPIL]], !llvm.loop !2
; AUTO_VEC: middle.block: ; AUTO_VEC: middle.block:
; AUTO_VEC-NEXT: [[CMP_N:%.*]] = icmp eq i64 [[N_VEC]], [[ZEXT]] ; AUTO_VEC-NEXT: [[CMP_N:%.*]] = icmp eq i64 [[N_VEC]], [[ZEXT]]
; AUTO_VEC-NEXT: br i1 [[CMP_N]], label [[FOR_END]], label [[FOR_BODY]] ; AUTO_VEC-NEXT: br i1 [[CMP_N]], label [[FOR_END]], label [[FOR_BODY]]
; AUTO_VEC: for.body: ; AUTO_VEC: for.body:
; AUTO_VEC-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY]] ], [ 0, [[FOR_BODY_PREHEADER]] ], [ [[N_VEC]], [[MIDDLE_BLOCK]] ] ; AUTO_VEC-NEXT: [[INDVARS_IV:%.*]] = phi i64 [ [[INDVARS_IV_NEXT:%.*]], [[FOR_BODY]] ], [ 0, [[FOR_BODY_PREHEADER]] ], [ [[N_VEC]], [[MIDDLE_BLOCK]] ]
; AUTO_VEC-NEXT: [[X_06:%.*]] = phi float [ [[CONV1:%.*]], [[FOR_BODY]] ], [ 1.000000e+00, [[FOR_BODY_PREHEADER]] ], [ [[IND_END]], [[MIDDLE_BLOCK]] ] ; AUTO_VEC-NEXT: [[X_06:%.*]] = phi float [ [[CONV1:%.*]], [[FOR_BODY]] ], [ 1.000000e+00, [[FOR_BODY_PREHEADER]] ], [ [[IND_END]], [[MIDDLE_BLOCK]] ]
; AUTO_VEC-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds float, float* [[A]], i64 [[INDVARS_IV]] ; AUTO_VEC-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds float, float* [[A]], i64 [[INDVARS_IV]]
; AUTO_VEC-NEXT: store float [[X_06]], float* [[ARRAYIDX]], align 4 ; AUTO_VEC-NEXT: store float [[X_06]], float* [[ARRAYIDX]], align 4
; AUTO_VEC-NEXT: [[CONV1]] = fadd float [[X_06]], 5.000000e-01 ; AUTO_VEC-NEXT: [[CONV1]] = fadd float [[X_06]], 5.000000e-01
; AUTO_VEC-NEXT: [[INDVARS_IV_NEXT]] = add nuw nsw i64 [[INDVARS_IV]], 1 ; AUTO_VEC-NEXT: [[INDVARS_IV_NEXT]] = add nuw nsw i64 [[INDVARS_IV]], 1
; AUTO_VEC-NEXT: [[TMP45:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT]], [[ZEXT]] ; AUTO_VEC-NEXT: [[TMP10:%.*]] = icmp eq i64 [[INDVARS_IV_NEXT]], [[ZEXT]]
; AUTO_VEC-NEXT: br i1 [[TMP45]], label [[FOR_END]], label [[FOR_BODY]], !llvm.loop !4 ; AUTO_VEC-NEXT: br i1 [[TMP10]], label [[FOR_END]], label [[FOR_BODY]], !llvm.loop !2
; AUTO_VEC: for.end: ; AUTO_VEC: for.end:
; AUTO_VEC-NEXT: ret void ; AUTO_VEC-NEXT: ret void
; ;
entry: entry:
%cmp4 = icmp sgt i32 %N, 0 %cmp4 = icmp sgt i32 %N, 0
br i1 %cmp4, label %for.body.preheader, label %for.end br i1 %cmp4, label %for.body.preheader, label %for.end
for.body.preheader: ; preds = %entry for.body.preheader: ; preds = %entry
▲ Show 20 Lines • Show All 90 Lines • ▼ Show 20 Lines
; AUTO_VEC-NEXT: [[X_06_EPIL:%.*]] = phi float [ [[CONV1_EPIL:%.*]], [[FOR_BODY_EPIL]] ], [ [[X_06_UNR]], [[FOR_END_LOOPEXIT_UNR_LCSSA]] ] ; AUTO_VEC-NEXT: [[X_06_EPIL:%.*]] = phi float [ [[CONV1_EPIL:%.*]], [[FOR_BODY_EPIL]] ], [ [[X_06_UNR]], [[FOR_END_LOOPEXIT_UNR_LCSSA]] ]
; AUTO_VEC-NEXT: [[EPIL_ITER:%.*]] = phi i64 [ [[EPIL_ITER_SUB:%.*]], [[FOR_BODY_EPIL]] ], [ [[XTRAITER]], [[FOR_END_LOOPEXIT_UNR_LCSSA]] ] ; AUTO_VEC-NEXT: [[EPIL_ITER:%.*]] = phi i64 [ [[EPIL_ITER_SUB:%.*]], [[FOR_BODY_EPIL]] ], [ [[XTRAITER]], [[FOR_END_LOOPEXIT_UNR_LCSSA]] ]
; AUTO_VEC-NEXT: [[ARRAYIDX_EPIL:%.*]] = getelementptr inbounds float, float* [[A]], i64 [[INDVARS_IV_EPIL]] ; AUTO_VEC-NEXT: [[ARRAYIDX_EPIL:%.*]] = getelementptr inbounds float, float* [[A]], i64 [[INDVARS_IV_EPIL]]
; AUTO_VEC-NEXT: store float [[X_06_EPIL]], float* [[ARRAYIDX_EPIL]], align 4 ; AUTO_VEC-NEXT: store float [[X_06_EPIL]], float* [[ARRAYIDX_EPIL]], align 4
; AUTO_VEC-NEXT: [[CONV1_EPIL]] = fadd float [[X_06_EPIL]], 5.000000e-01 ; AUTO_VEC-NEXT: [[CONV1_EPIL]] = fadd float [[X_06_EPIL]], 5.000000e-01
; AUTO_VEC-NEXT: [[INDVARS_IV_NEXT_EPIL]] = add nuw nsw i64 [[INDVARS_IV_EPIL]], 1 ; AUTO_VEC-NEXT: [[INDVARS_IV_NEXT_EPIL]] = add nuw nsw i64 [[INDVARS_IV_EPIL]], 1
; AUTO_VEC-NEXT: [[EPIL_ITER_SUB]] = add i64 [[EPIL_ITER]], -1 ; AUTO_VEC-NEXT: [[EPIL_ITER_SUB]] = add i64 [[EPIL_ITER]], -1
; AUTO_VEC-NEXT: [[EPIL_ITER_CMP:%.*]] = icmp eq i64 [[EPIL_ITER_SUB]], 0 ; AUTO_VEC-NEXT: [[EPIL_ITER_CMP:%.*]] = icmp eq i64 [[EPIL_ITER_SUB]], 0
; AUTO_VEC-NEXT: br i1 [[EPIL_ITER_CMP]], label [[FOR_END]], label [[FOR_BODY_EPIL]], !llvm.loop !6 ; AUTO_VEC-NEXT: br i1 [[EPIL_ITER_CMP]], label [[FOR_END]], label [[FOR_BODY_EPIL]], !llvm.loop !4
; AUTO_VEC: for.end: ; AUTO_VEC: for.end:
; AUTO_VEC-NEXT: ret void ; AUTO_VEC-NEXT: ret void
; ;
entry: entry:
%cmp4 = icmp sgt i32 %N, 0 %cmp4 = icmp sgt i32 %N, 0
br i1 %cmp4, label %for.body.preheader, label %for.end br i1 %cmp4, label %for.body.preheader, label %for.end
for.body.preheader: ; preds = %entry for.body.preheader: ; preds = %entry
Show All 23 Lines
; AUTO_VEC-NEXT: [[TMP0:%.*]] = icmp sgt i64 [[N:%.*]], 1 ; AUTO_VEC-NEXT: [[TMP0:%.*]] = icmp sgt i64 [[N:%.*]], 1
; AUTO_VEC-NEXT: [[SMAX:%.*]] = select i1 [[TMP0]], i64 [[N]], i64 1 ; AUTO_VEC-NEXT: [[SMAX:%.*]] = select i1 [[TMP0]], i64 [[N]], i64 1
; AUTO_VEC-NEXT: [[MIN_ITERS_CHECK:%.*]] = icmp ult i64 [[SMAX]], 16 ; AUTO_VEC-NEXT: [[MIN_ITERS_CHECK:%.*]] = icmp ult i64 [[SMAX]], 16
; AUTO_VEC-NEXT: br i1 [[MIN_ITERS_CHECK]], label [[FOR_BODY:%.*]], label [[VECTOR_PH:%.*]] ; AUTO_VEC-NEXT: br i1 [[MIN_ITERS_CHECK]], label [[FOR_BODY:%.*]], label [[VECTOR_PH:%.*]]
; AUTO_VEC: vector.ph: ; AUTO_VEC: vector.ph:
; AUTO_VEC-NEXT: [[N_VEC:%.*]] = and i64 [[SMAX]], 9223372036854775792 ; AUTO_VEC-NEXT: [[N_VEC:%.*]] = and i64 [[SMAX]], 9223372036854775792
; AUTO_VEC-NEXT: [[CAST_CRD:%.*]] = sitofp i64 [[N_VEC]] to double ; AUTO_VEC-NEXT: [[CAST_CRD:%.*]] = sitofp i64 [[N_VEC]] to double
; AUTO_VEC-NEXT: [[TMP1:%.*]] = fmul fast double [[CAST_CRD]], 3.000000e+00 ; AUTO_VEC-NEXT: [[TMP1:%.*]] = fmul fast double [[CAST_CRD]], 3.000000e+00
; AUTO_VEC-NEXT: [[TMP2:%.*]] = add nsw i64 [[N_VEC]], -16
; AUTO_VEC-NEXT: [[TMP3:%.*]] = lshr exact i64 [[TMP2]], 4
; AUTO_VEC-NEXT: [[TMP4:%.*]] = add nuw nsw i64 [[TMP3]], 1
; AUTO_VEC-NEXT: [[XTRAITER:%.*]] = and i64 [[TMP4]], 3
; AUTO_VEC-NEXT: [[TMP5:%.*]] = icmp ult i64 [[TMP2]], 48
; AUTO_VEC-NEXT: br i1 [[TMP5]], label [[MIDDLE_BLOCK_UNR_LCSSA:%.*]], label [[VECTOR_PH_NEW:%.*]]
; AUTO_VEC: vector.ph.new:
; AUTO_VEC-NEXT: [[UNROLL_ITER:%.*]] = and i64 [[TMP4]], 2305843009213693948
; AUTO_VEC-NEXT: br label [[VECTOR_BODY:%.*]] ; AUTO_VEC-NEXT: br label [[VECTOR_BODY:%.*]]
; AUTO_VEC: vector.body: ; AUTO_VEC: vector.body:
; AUTO_VEC-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH_NEW]] ], [ [[INDEX_NEXT_3:%.*]], [[VECTOR_BODY]] ] ; AUTO_VEC-NEXT: [[INDEX:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT:%.*]], [[VECTOR_BODY]] ]
; AUTO_VEC-NEXT: [[VEC_IND:%.*]] = phi <4 x double> [ <double 0.000000e+00, double 3.000000e+00, double 6.000000e+00, double 9.000000e+00>, [[VECTOR_PH_NEW]] ], [ [[VEC_IND_NEXT_3:%.*]], [[VECTOR_BODY]] ] ; AUTO_VEC-NEXT: [[VEC_IND:%.*]] = phi <4 x double> [ <double 0.000000e+00, double 3.000000e+00, double 6.000000e+00, double 9.000000e+00>, [[VECTOR_PH]] ], [ [[VEC_IND_NEXT:%.*]], [[VECTOR_BODY]] ]
; AUTO_VEC-NEXT: [[NITER:%.*]] = phi i64 [ [[UNROLL_ITER]], [[VECTOR_PH_NEW]] ], [ [[NITER_NSUB_3:%.*]], [[VECTOR_BODY]] ]
; AUTO_VEC-NEXT: [[STEP_ADD:%.*]] = fadd fast <4 x double> [[VEC_IND]], <double 1.200000e+01, double 1.200000e+01, double 1.200000e+01, double 1.200000e+01> ; AUTO_VEC-NEXT: [[STEP_ADD:%.*]] = fadd fast <4 x double> [[VEC_IND]], <double 1.200000e+01, double 1.200000e+01, double 1.200000e+01, double 1.200000e+01>
; AUTO_VEC-NEXT: [[STEP_ADD5:%.*]] = fadd fast <4 x double> [[VEC_IND]], <double 2.400000e+01, double 2.400000e+01, double 2.400000e+01, double 2.400000e+01> ; AUTO_VEC-NEXT: [[STEP_ADD5:%.*]] = fadd fast <4 x double> [[VEC_IND]], <double 2.400000e+01, double 2.400000e+01, double 2.400000e+01, double 2.400000e+01>
; AUTO_VEC-NEXT: [[STEP_ADD6:%.*]] = fadd fast <4 x double> [[VEC_IND]], <double 3.600000e+01, double 3.600000e+01, double 3.600000e+01, double 3.600000e+01> ; AUTO_VEC-NEXT: [[STEP_ADD6:%.*]] = fadd fast <4 x double> [[VEC_IND]], <double 3.600000e+01, double 3.600000e+01, double 3.600000e+01, double 3.600000e+01>
; AUTO_VEC-NEXT: [[TMP6:%.*]] = getelementptr double, double* [[A:%.*]], i64 [[INDEX]] ; AUTO_VEC-NEXT: [[TMP2:%.*]] = getelementptr double, double* [[A:%.*]], i64 [[INDEX]]
; AUTO_VEC-NEXT: [[TMP3:%.*]] = bitcast double* [[TMP2]] to <4 x double>*
; AUTO_VEC-NEXT: store <4 x double> [[VEC_IND]], <4 x double>* [[TMP3]], align 8
; AUTO_VEC-NEXT: [[TMP4:%.*]] = getelementptr double, double* [[TMP2]], i64 4
; AUTO_VEC-NEXT: [[TMP5:%.*]] = bitcast double* [[TMP4]] to <4 x double>*
; AUTO_VEC-NEXT: store <4 x double> [[STEP_ADD]], <4 x double>* [[TMP5]], align 8
; AUTO_VEC-NEXT: [[TMP6:%.*]] = getelementptr double, double* [[TMP2]], i64 8
; AUTO_VEC-NEXT: [[TMP7:%.*]] = bitcast double* [[TMP6]] to <4 x double>* ; AUTO_VEC-NEXT: [[TMP7:%.*]] = bitcast double* [[TMP6]] to <4 x double>*
; AUTO_VEC-NEXT: store <4 x double> [[VEC_IND]], <4 x double>* [[TMP7]], align 8 ; AUTO_VEC-NEXT: store <4 x double> [[STEP_ADD5]], <4 x double>* [[TMP7]], align 8
; AUTO_VEC-NEXT: [[TMP8:%.*]] = getelementptr double, double* [[TMP6]], i64 4 ; AUTO_VEC-NEXT: [[TMP8:%.*]] = getelementptr double, double* [[TMP2]], i64 12
; AUTO_VEC-NEXT: [[TMP9:%.*]] = bitcast double* [[TMP8]] to <4 x double>* ; AUTO_VEC-NEXT: [[TMP9:%.*]] = bitcast double* [[TMP8]] to <4 x double>*
; AUTO_VEC-NEXT: store <4 x double> [[STEP_ADD]], <4 x double>* [[TMP9]], align 8 ; AUTO_VEC-NEXT: store <4 x double> [[STEP_ADD6]], <4 x double>* [[TMP9]], align 8
; AUTO_VEC-NEXT: [[TMP10:%.*]] = getelementptr double, double* [[TMP6]], i64 8 ; AUTO_VEC-NEXT: [[INDEX_NEXT]] = add i64 [[INDEX]], 16
; AUTO_VEC-NEXT: [[TMP11:%.*]] = bitcast double* [[TMP10]] to <4 x double>* ; AUTO_VEC-NEXT: [[VEC_IND_NEXT]] = fadd fast <4 x double> [[VEC_IND]], <double 4.800000e+01, double 4.800000e+01, double 4.800000e+01, double 4.800000e+01>
; AUTO_VEC-NEXT: store <4 x double> [[STEP_ADD5]], <4 x double>* [[TMP11]], align 8 ; AUTO_VEC-NEXT: [[TMP10:%.*]] = icmp eq i64 [[INDEX_NEXT]], [[N_VEC]]
; AUTO_VEC-NEXT: [[TMP12:%.*]] = getelementptr double, double* [[TMP6]], i64 12 ; AUTO_VEC-NEXT: br i1 [[TMP10]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY]], !llvm.loop !6
; AUTO_VEC-NEXT: [[TMP13:%.*]] = bitcast double* [[TMP12]] to <4 x double>*
; AUTO_VEC-NEXT: store <4 x double> [[STEP_ADD6]], <4 x double>* [[TMP13]], align 8
; AUTO_VEC-NEXT: [[INDEX_NEXT:%.*]] = or i64 [[INDEX]], 16
; AUTO_VEC-NEXT: [[VEC_IND_NEXT:%.*]] = fadd fast <4 x double> [[VEC_IND]], <double 4.800000e+01, double 4.800000e+01, double 4.800000e+01, double 4.800000e+01>
; AUTO_VEC-NEXT: [[STEP_ADD_1:%.*]] = fadd fast <4 x double> [[VEC_IND]], <double 6.000000e+01, double 6.000000e+01, double 6.000000e+01, double 6.000000e+01>
; AUTO_VEC-NEXT: [[STEP_ADD5_1:%.*]] = fadd fast <4 x double> [[VEC_IND]], <double 7.200000e+01, double 7.200000e+01, double 7.200000e+01, double 7.200000e+01>
; AUTO_VEC-NEXT: [[STEP_ADD6_1:%.*]] = fadd fast <4 x double> [[VEC_IND]], <double 8.400000e+01, double 8.400000e+01, double 8.400000e+01, double 8.400000e+01>
; AUTO_VEC-NEXT: [[TMP14:%.*]] = getelementptr double, double* [[A]], i64 [[INDEX_NEXT]]
; AUTO_VEC-NEXT: [[TMP15:%.*]] = bitcast double* [[TMP14]] to <4 x double>*
; AUTO_VEC-NEXT: store <4 x double> [[VEC_IND_NEXT]], <4 x double>* [[TMP15]], align 8
; AUTO_VEC-NEXT: [[TMP16:%.*]] = getelementptr double, double* [[TMP14]], i64 4
; AUTO_VEC-NEXT: [[TMP17:%.*]] = bitcast double* [[TMP16]] to <4 x double>*
; AUTO_VEC-NEXT: store <4 x double> [[STEP_ADD_1]], <4 x double>* [[TMP17]], align 8
; AUTO_VEC-NEXT: [[TMP18:%.*]] = getelementptr double, double* [[TMP14]], i64 8
; AUTO_VEC-NEXT: [[TMP19:%.*]] = bitcast double* [[TMP18]] to <4 x double>*
; AUTO_VEC-NEXT: store <4 x double> [[STEP_ADD5_1]], <4 x double>* [[TMP19]], align 8
; AUTO_VEC-NEXT: [[TMP20:%.*]] = getelementptr double, double* [[TMP14]], i64 12
; AUTO_VEC-NEXT: [[TMP21:%.*]] = bitcast double* [[TMP20]] to <4 x double>*
; AUTO_VEC-NEXT: store <4 x double> [[STEP_ADD6_1]], <4 x double>* [[TMP21]], align 8
; AUTO_VEC-NEXT: [[INDEX_NEXT_1:%.*]] = or i64 [[INDEX]], 32
; AUTO_VEC-NEXT: [[VEC_IND_NEXT_1:%.*]] = fadd fast <4 x double> [[VEC_IND]], <double 9.600000e+01, double 9.600000e+01, double 9.600000e+01, double 9.600000e+01>
; AUTO_VEC-NEXT: [[STEP_ADD_2:%.*]] = fadd fast <4 x double> [[VEC_IND]], <double 1.080000e+02, double 1.080000e+02, double 1.080000e+02, double 1.080000e+02>
; AUTO_VEC-NEXT: [[STEP_ADD5_2:%.*]] = fadd fast <4 x double> [[VEC_IND]], <double 1.200000e+02, double 1.200000e+02, double 1.200000e+02, double 1.200000e+02>
; AUTO_VEC-NEXT: [[STEP_ADD6_2:%.*]] = fadd fast <4 x double> [[VEC_IND]], <double 1.320000e+02, double 1.320000e+02, double 1.320000e+02, double 1.320000e+02>
; AUTO_VEC-NEXT: [[TMP22:%.*]] = getelementptr double, double* [[A]], i64 [[INDEX_NEXT_1]]
; AUTO_VEC-NEXT: [[TMP23:%.*]] = bitcast double* [[TMP22]] to <4 x double>*
; AUTO_VEC-NEXT: store <4 x double> [[VEC_IND_NEXT_1]], <4 x double>* [[TMP23]], align 8
; AUTO_VEC-NEXT: [[TMP24:%.*]] = getelementptr double, double* [[TMP22]], i64 4
; AUTO_VEC-NEXT: [[TMP25:%.*]] = bitcast double* [[TMP24]] to <4 x double>*
; AUTO_VEC-NEXT: store <4 x double> [[STEP_ADD_2]], <4 x double>* [[TMP25]], align 8
; AUTO_VEC-NEXT: [[TMP26:%.*]] = getelementptr double, double* [[TMP22]], i64 8
; AUTO_VEC-NEXT: [[TMP27:%.*]] = bitcast double* [[TMP26]] to <4 x double>*
; AUTO_VEC-NEXT: store <4 x double> [[STEP_ADD5_2]], <4 x double>* [[TMP27]], align 8
; AUTO_VEC-NEXT: [[TMP28:%.*]] = getelementptr double, double* [[TMP22]], i64 12
; AUTO_VEC-NEXT: [[TMP29:%.*]] = bitcast double* [[TMP28]] to <4 x double>*
; AUTO_VEC-NEXT: store <4 x double> [[STEP_ADD6_2]], <4 x double>* [[TMP29]], align 8
; AUTO_VEC-NEXT: [[INDEX_NEXT_2:%.*]] = or i64 [[INDEX]], 48
; AUTO_VEC-NEXT: [[VEC_IND_NEXT_2:%.*]] = fadd fast <4 x double> [[VEC_IND]], <double 1.440000e+02, double 1.440000e+02, double 1.440000e+02, double 1.440000e+02>
; AUTO_VEC-NEXT: [[STEP_ADD_3:%.*]] = fadd fast <4 x double> [[VEC_IND]], <double 1.560000e+02, double 1.560000e+02, double 1.560000e+02, double 1.560000e+02>
; AUTO_VEC-NEXT: [[STEP_ADD5_3:%.*]] = fadd fast <4 x double> [[VEC_IND]], <double 1.680000e+02, double 1.680000e+02, double 1.680000e+02, double 1.680000e+02>
; AUTO_VEC-NEXT: [[STEP_ADD6_3:%.*]] = fadd fast <4 x double> [[VEC_IND]], <double 1.800000e+02, double 1.800000e+02, double 1.800000e+02, double 1.800000e+02>
; AUTO_VEC-NEXT: [[TMP30:%.*]] = getelementptr double, double* [[A]], i64 [[INDEX_NEXT_2]]
; AUTO_VEC-NEXT: [[TMP31:%.*]] = bitcast double* [[TMP30]] to <4 x double>*
; AUTO_VEC-NEXT: store <4 x double> [[VEC_IND_NEXT_2]], <4 x double>* [[TMP31]], align 8
; AUTO_VEC-NEXT: [[TMP32:%.*]] = getelementptr double, double* [[TMP30]], i64 4
; AUTO_VEC-NEXT: [[TMP33:%.*]] = bitcast double* [[TMP32]] to <4 x double>*
; AUTO_VEC-NEXT: store <4 x double> [[STEP_ADD_3]], <4 x double>* [[TMP33]], align 8
; AUTO_VEC-NEXT: [[TMP34:%.*]] = getelementptr double, double* [[TMP30]], i64 8
; AUTO_VEC-NEXT: [[TMP35:%.*]] = bitcast double* [[TMP34]] to <4 x double>*
; AUTO_VEC-NEXT: store <4 x double> [[STEP_ADD5_3]], <4 x double>* [[TMP35]], align 8
; AUTO_VEC-NEXT: [[TMP36:%.*]] = getelementptr double, double* [[TMP30]], i64 12
; AUTO_VEC-NEXT: [[TMP37:%.*]] = bitcast double* [[TMP36]] to <4 x double>*
; AUTO_VEC-NEXT: store <4 x double> [[STEP_ADD6_3]], <4 x double>* [[TMP37]], align 8
; AUTO_VEC-NEXT: [[INDEX_NEXT_3]] = add i64 [[INDEX]], 64
; AUTO_VEC-NEXT: [[VEC_IND_NEXT_3]] = fadd fast <4 x double> [[VEC_IND]], <double 1.920000e+02, double 1.920000e+02, double 1.920000e+02, double 1.920000e+02>
; AUTO_VEC-NEXT: [[NITER_NSUB_3]] = add i64 [[NITER]], -4
; AUTO_VEC-NEXT: [[NITER_NCMP_3:%.*]] = icmp eq i64 [[NITER_NSUB_3]], 0
; AUTO_VEC-NEXT: br i1 [[NITER_NCMP_3]], label [[MIDDLE_BLOCK_UNR_LCSSA]], label [[VECTOR_BODY]], !llvm.loop !7
; AUTO_VEC: middle.block.unr-lcssa:
; AUTO_VEC-NEXT: [[INDEX_UNR:%.*]] = phi i64 [ 0, [[VECTOR_PH]] ], [ [[INDEX_NEXT_3]], [[VECTOR_BODY]] ]
; AUTO_VEC-NEXT: [[VEC_IND_UNR:%.*]] = phi <4 x double> [ <double 0.000000e+00, double 3.000000e+00, double 6.000000e+00, double 9.000000e+00>, [[VECTOR_PH]] ], [ [[VEC_IND_NEXT_3]], [[VECTOR_BODY]] ]
; AUTO_VEC-NEXT: [[LCMP_MOD:%.*]] = icmp eq i64 [[XTRAITER]], 0
; AUTO_VEC-NEXT: br i1 [[LCMP_MOD]], label [[MIDDLE_BLOCK:%.*]], label [[VECTOR_BODY_EPIL:%.*]]
; AUTO_VEC: vector.body.epil:
; AUTO_VEC-NEXT: [[INDEX_EPIL:%.*]] = phi i64 [ [[INDEX_NEXT_EPIL:%.*]], [[VECTOR_BODY_EPIL]] ], [ [[INDEX_UNR]], [[MIDDLE_BLOCK_UNR_LCSSA]] ]
; AUTO_VEC-NEXT: [[VEC_IND_EPIL:%.*]] = phi <4 x double> [ [[VEC_IND_NEXT_EPIL:%.*]], [[VECTOR_BODY_EPIL]] ], [ [[VEC_IND_UNR]], [[MIDDLE_BLOCK_UNR_LCSSA]] ]
; AUTO_VEC-NEXT: [[EPIL_ITER:%.*]] = phi i64 [ [[EPIL_ITER_SUB:%.*]], [[VECTOR_BODY_EPIL]] ], [ [[XTRAITER]], [[MIDDLE_BLOCK_UNR_LCSSA]] ]
; AUTO_VEC-NEXT: [[STEP_ADD_EPIL:%.*]] = fadd fast <4 x double> [[VEC_IND_EPIL]], <double 1.200000e+01, double 1.200000e+01, double 1.200000e+01, double 1.200000e+01>
; AUTO_VEC-NEXT: [[STEP_ADD5_EPIL:%.*]] = fadd fast <4 x double> [[VEC_IND_EPIL]], <double 2.400000e+01, double 2.400000e+01, double 2.400000e+01, double 2.400000e+01>
; AUTO_VEC-NEXT: [[STEP_ADD6_EPIL:%.*]] = fadd fast <4 x double> [[VEC_IND_EPIL]], <double 3.600000e+01, double 3.600000e+01, double 3.600000e+01, double 3.600000e+01>
; AUTO_VEC-NEXT: [[TMP38:%.*]] = getelementptr double, double* [[A]], i64 [[INDEX_EPIL]]
; AUTO_VEC-NEXT: [[TMP39:%.*]] = bitcast double* [[TMP38]] to <4 x double>*
; AUTO_VEC-NEXT: store <4 x double> [[VEC_IND_EPIL]], <4 x double>* [[TMP39]], align 8
; AUTO_VEC-NEXT: [[TMP40:%.*]] = getelementptr double, double* [[TMP38]], i64 4
; AUTO_VEC-NEXT: [[TMP41:%.*]] = bitcast double* [[TMP40]] to <4 x double>*
; AUTO_VEC-NEXT: store <4 x double> [[STEP_ADD_EPIL]], <4 x double>* [[TMP41]], align 8
; AUTO_VEC-NEXT: [[TMP42:%.*]] = getelementptr double, double* [[TMP38]], i64 8
; AUTO_VEC-NEXT: [[TMP43:%.*]] = bitcast double* [[TMP42]] to <4 x double>*
; AUTO_VEC-NEXT: store <4 x double> [[STEP_ADD5_EPIL]], <4 x double>* [[TMP43]], align 8
; AUTO_VEC-NEXT: [[TMP44:%.*]] = getelementptr double, double* [[TMP38]], i64 12
; AUTO_VEC-NEXT: [[TMP45:%.*]] = bitcast double* [[TMP44]] to <4 x double>*
; AUTO_VEC-NEXT: store <4 x double> [[STEP_ADD6_EPIL]], <4 x double>* [[TMP45]], align 8
; AUTO_VEC-NEXT: [[INDEX_NEXT_EPIL]] = add i64 [[INDEX_EPIL]], 16
; AUTO_VEC-NEXT: [[VEC_IND_NEXT_EPIL]] = fadd fast <4 x double> [[VEC_IND_EPIL]], <double 4.800000e+01, double 4.800000e+01, double 4.800000e+01, double 4.800000e+01>
; AUTO_VEC-NEXT: [[EPIL_ITER_SUB]] = add i64 [[EPIL_ITER]], -1
; AUTO_VEC-NEXT: [[EPIL_ITER_CMP:%.*]] = icmp eq i64 [[EPIL_ITER_SUB]], 0
; AUTO_VEC-NEXT: br i1 [[EPIL_ITER_CMP]], label [[MIDDLE_BLOCK]], label [[VECTOR_BODY_EPIL]], !llvm.loop !8
; AUTO_VEC: middle.block: ; AUTO_VEC: middle.block:
; AUTO_VEC-NEXT: [[CMP_N:%.*]] = icmp eq i64 [[SMAX]], [[N_VEC]] ; AUTO_VEC-NEXT: [[CMP_N:%.*]] = icmp eq i64 [[SMAX]], [[N_VEC]]
; AUTO_VEC-NEXT: [[TMP46:%.*]] = add nsw i64 [[N_VEC]], -1 ; AUTO_VEC-NEXT: [[TMP11:%.*]] = add nsw i64 [[N_VEC]], -1
; AUTO_VEC-NEXT: [[CAST_CMO:%.*]] = sitofp i64 [[TMP46]] to double ; AUTO_VEC-NEXT: [[CAST_CMO:%.*]] = sitofp i64 [[TMP11]] to double
; AUTO_VEC-NEXT: [[TMP47:%.*]] = fmul fast double [[CAST_CMO]], 3.000000e+00 ; AUTO_VEC-NEXT: [[TMP12:%.*]] = fmul fast double [[CAST_CMO]], 3.000000e+00
; AUTO_VEC-NEXT: br i1 [[CMP_N]], label [[FOR_END:%.*]], label [[FOR_BODY]] ; AUTO_VEC-NEXT: br i1 [[CMP_N]], label [[FOR_END:%.*]], label [[FOR_BODY]]
; AUTO_VEC: for.body: ; AUTO_VEC: for.body:
; AUTO_VEC-NEXT: [[I:%.*]] = phi i64 [ [[I_NEXT:%.*]], [[FOR_BODY]] ], [ 0, [[ENTRY:%.*]] ], [ [[N_VEC]], [[MIDDLE_BLOCK]] ] ; AUTO_VEC-NEXT: [[I:%.*]] = phi i64 [ [[I_NEXT:%.*]], [[FOR_BODY]] ], [ 0, [[ENTRY:%.*]] ], [ [[N_VEC]], [[MIDDLE_BLOCK]] ]
; AUTO_VEC-NEXT: [[J:%.*]] = phi double [ [[J_NEXT:%.*]], [[FOR_BODY]] ], [ 0.000000e+00, [[ENTRY]] ], [ [[TMP1]], [[MIDDLE_BLOCK]] ] ; AUTO_VEC-NEXT: [[J:%.*]] = phi double [ [[J_NEXT:%.*]], [[FOR_BODY]] ], [ 0.000000e+00, [[ENTRY]] ], [ [[TMP1]], [[MIDDLE_BLOCK]] ]
; AUTO_VEC-NEXT: [[TMP0:%.*]] = getelementptr double, double* [[A]], i64 [[I]] ; AUTO_VEC-NEXT: [[TMP0:%.*]] = getelementptr double, double* [[A]], i64 [[I]]
; AUTO_VEC-NEXT: store double [[J]], double* [[TMP0]], align 8 ; AUTO_VEC-NEXT: store double [[J]], double* [[TMP0]], align 8
; AUTO_VEC-NEXT: [[I_NEXT]] = add nuw nsw i64 [[I]], 1 ; AUTO_VEC-NEXT: [[I_NEXT]] = add nuw nsw i64 [[I]], 1
; AUTO_VEC-NEXT: [[J_NEXT]] = fadd fast double [[J]], 3.000000e+00 ; AUTO_VEC-NEXT: [[J_NEXT]] = fadd fast double [[J]], 3.000000e+00
; AUTO_VEC-NEXT: [[EXITCOND:%.*]] = icmp eq i64 [[I_NEXT]], [[SMAX]] ; AUTO_VEC-NEXT: [[COND:%.*]] = icmp slt i64 [[I_NEXT]], [[N]]
; AUTO_VEC-NEXT: br i1 [[EXITCOND]], label [[FOR_END]], label [[FOR_BODY]], !llvm.loop !9 ; AUTO_VEC-NEXT: br i1 [[COND]], label [[FOR_BODY]], label [[FOR_END]], !llvm.loop !7
; AUTO_VEC: for.end: ; AUTO_VEC: for.end:
; AUTO_VEC-NEXT: [[J_LCSSA:%.*]] = phi double [ [[TMP47]], [[MIDDLE_BLOCK]] ], [ [[J]], [[FOR_BODY]] ] ; AUTO_VEC-NEXT: [[J_LCSSA:%.*]] = phi double [ [[TMP12]], [[MIDDLE_BLOCK]] ], [ [[J]], [[FOR_BODY]] ]
; AUTO_VEC-NEXT: ret double [[J_LCSSA]] ; AUTO_VEC-NEXT: ret double [[J_LCSSA]]
; ;
entry: entry:
br label %for.body br label %for.body
for.body: for.body:
%i = phi i64 [ 0, %entry ], [%i.next, %for.body] %i = phi i64 [ 0, %entry ], [%i.next, %for.body]
%j = phi double [ 0.0, %entry ], [ %j.next, %for.body ] %j = phi double [ 0.0, %entry ], [ %j.next, %for.body ]
▲ Show 20 Lines • Show All 71 Lines • ▼ Show 20 Lines
; AUTO_VEC-NEXT: [[J_EPIL:%.*]] = phi double [ [[J_NEXT_EPIL:%.*]], [[FOR_BODY_EPIL]] ], [ [[J_UNR]], [[FOR_END_UNR_LCSSA]] ] ; AUTO_VEC-NEXT: [[J_EPIL:%.*]] = phi double [ [[J_NEXT_EPIL:%.*]], [[FOR_BODY_EPIL]] ], [ [[J_UNR]], [[FOR_END_UNR_LCSSA]] ]
; AUTO_VEC-NEXT: [[EPIL_ITER:%.*]] = phi i64 [ [[EPIL_ITER_SUB:%.*]], [[FOR_BODY_EPIL]] ], [ [[XTRAITER]], [[FOR_END_UNR_LCSSA]] ] ; AUTO_VEC-NEXT: [[EPIL_ITER:%.*]] = phi i64 [ [[EPIL_ITER_SUB:%.*]], [[FOR_BODY_EPIL]] ], [ [[XTRAITER]], [[FOR_END_UNR_LCSSA]] ]
; AUTO_VEC-NEXT: [[TMP0_EPIL:%.*]] = getelementptr double, double* [[A]], i64 [[I_EPIL]] ; AUTO_VEC-NEXT: [[TMP0_EPIL:%.*]] = getelementptr double, double* [[A]], i64 [[I_EPIL]]
; AUTO_VEC-NEXT: store double [[J_EPIL]], double* [[TMP0_EPIL]], align 8 ; AUTO_VEC-NEXT: store double [[J_EPIL]], double* [[TMP0_EPIL]], align 8
; AUTO_VEC-NEXT: [[I_NEXT_EPIL]] = add nuw nsw i64 [[I_EPIL]], 1 ; AUTO_VEC-NEXT: [[I_NEXT_EPIL]] = add nuw nsw i64 [[I_EPIL]], 1
; AUTO_VEC-NEXT: [[J_NEXT_EPIL]] = fadd double [[J_EPIL]], 3.000000e+00 ; AUTO_VEC-NEXT: [[J_NEXT_EPIL]] = fadd double [[J_EPIL]], 3.000000e+00
; AUTO_VEC-NEXT: [[EPIL_ITER_SUB]] = add i64 [[EPIL_ITER]], -1 ; AUTO_VEC-NEXT: [[EPIL_ITER_SUB]] = add i64 [[EPIL_ITER]], -1
; AUTO_VEC-NEXT: [[EPIL_ITER_CMP:%.*]] = icmp eq i64 [[EPIL_ITER_SUB]], 0 ; AUTO_VEC-NEXT: [[EPIL_ITER_CMP:%.*]] = icmp eq i64 [[EPIL_ITER_SUB]], 0
; AUTO_VEC-NEXT: br i1 [[EPIL_ITER_CMP]], label [[FOR_END]], label [[FOR_BODY_EPIL]], !llvm.loop !10 ; AUTO_VEC-NEXT: br i1 [[EPIL_ITER_CMP]], label [[FOR_END]], label [[FOR_BODY_EPIL]], !llvm.loop !8
; AUTO_VEC: for.end: ; AUTO_VEC: for.end:
; AUTO_VEC-NEXT: [[J_LCSSA:%.*]] = phi double [ [[J_LCSSA_PH]], [[FOR_END_UNR_LCSSA]] ], [ [[J_EPIL]], [[FOR_BODY_EPIL]] ] ; AUTO_VEC-NEXT: [[J_LCSSA:%.*]] = phi double [ [[J_LCSSA_PH]], [[FOR_END_UNR_LCSSA]] ], [ [[J_EPIL]], [[FOR_BODY_EPIL]] ]
; AUTO_VEC-NEXT: ret double [[J_LCSSA]] ; AUTO_VEC-NEXT: ret double [[J_LCSSA]]
; ;
entry: entry:
br label %for.body br label %for.body
for.body: for.body:
Show All 16 Lines

llvm/test/Transforms/SpeculateAroundPHIs/convergent.ll

Show First 20 Lines • Show All 62 Lines • ▼ Show 20 Lines
} }
; Otherwise identical function which should be transformed. ; Otherwise identical function which should be transformed.
define i32 @test_reference(i1 %flag, i32 %arg) #2 { define i32 @test_reference(i1 %flag, i32 %arg) #2 {
; CHECK-LABEL: @test_reference( ; CHECK-LABEL: @test_reference(
; CHECK-NEXT: entry: ; CHECK-NEXT: entry:
; CHECK-NEXT: br i1 [[FLAG:%.*]], label [[A:%.*]], label [[B:%.*]] ; CHECK-NEXT: br i1 [[FLAG:%.*]], label [[A:%.*]], label [[B:%.*]]
; CHECK: a: ; CHECK: a:
; CHECK-NEXT: [[SUM_0:%.*]] = call i32 @llvm.regular(i32 7)
; CHECK-NEXT: br label [[EXIT:%.*]] ; CHECK-NEXT: br label [[EXIT:%.*]]
; CHECK: b: ; CHECK: b:
; CHECK-NEXT: [[SUM_1:%.*]] = call i32 @llvm.regular(i32 11)
; CHECK-NEXT: br label [[EXIT]] ; CHECK-NEXT: br label [[EXIT]]
; CHECK: exit: ; CHECK: exit:
; CHECK-NEXT: [[SUM_PHI:%.*]] = phi i32 [ [[SUM_0]], [[A]] ], [ [[SUM_1]], [[B]] ] ; CHECK-NEXT: [[P:%.*]] = phi i32 [ 7, [[A]] ], [ 11, [[B]] ]
; CHECK-NEXT: ret i32 [[SUM_PHI]] ; CHECK-NEXT: [[SUM:%.*]] = call i32 @llvm.regular(i32 [[P]])
; CHECK-NEXT: ret i32 [[SUM]]
; ;
entry: entry:
br i1 %flag, label %a, label %b br i1 %flag, label %a, label %b
a: a:
br label %exit br label %exit
b: b:
Show All 12 Lines