[VE] Make VE official

[simoll]

Summary

Add VE to the list of official targets.

Proposal: https://lists.llvm.org/pipermail/llvm-dev/2021-November/153571.html

[vchuravy]

👏

[rengolin]

Does this need to come after D113093?

[simoll]

In D113247#3114954, @rengolin wrote:

Does this need to come after D113093?

Not strictly.
When D113093 is committed and we throw the switch from 'experimental' to 'official' with D113247, the clang-ve-ninja builder will be green. If VE is official first and compiler-rt fixed second, the VE builder will be failing as we make VE official.

[lattner]

Please propose this on llvm-dev if you haven't already

[rengolin]

This has been proposed on llvm-dev for almost a month and has only attracted two reviews (me and @reames) and I believe all raised points were addressed, including a now consistently green and fast buildbot.

I personally believe the VE community is ready to pick up the work to keep the target up-to-date and CI green.

This now looks good to me, thanks!

[simoll]

Great! I'll proceed with the plan we discussed on the mailing list tomorrow:

1. Connect the VE worker to main buildbot instance.
2. Add VE to the LLVM target list, thereby making it official (D113247).
3. Update llvm-zorg to build VE as an official target and enable buildbot reporting for VE (D113389).
4. Attentively follow all builders until the expected aftermath has settled.

[simoll]

In D113247#3161757, @simoll wrote:

1. Connect the VE worker to main buildbot instance. [DONE]

First clang-ve-ninja build of a recent commit with main buildbot instance: https://lab.llvm.org/buildbot/#/builders/91/builds/5
There were a few stale build requests in the pipeline before that.

[simoll]

In D113247#3161757, @simoll wrote:

2. Add VE to the LLVM target list, thereby making it official (D113247). [DONE]

commit a9d1d00b865ab6f6e75dcd649362a7c5cf01d168 (HEAD -> arcpatch-D113247, origin/main, origin/HEAD)
Author: Simon Moll <[e-mail]>
Date:   Wed Dec 1 13:21:39 2021 +0100

3. Update llvm-zorg to build VE as an official target and enable buildbot reporting for VE (D113389). [DONE]

commit feb754795a1d21d4644351b360f805e8e449026b (HEAD -> arcpatch-D113389, origin/main, origin/HEAD)
Author: Simon Moll <[e-mail]>
Date:   Wed Dec 1 13:22:16 2021 +0100

Now following up on potential build failures.

[simoll]

Unresolved

• sanitizer-x86_64-linux-fast (https://lab.llvm.org/buildbot/#/builders/5/builds/15384/steps/13/logs/stdio) . MachineInstr/iterator corrupted in lib/CodeGen/ExpandPostRAPseudos.cpp?

Cause identified

• clang-ppc64le-rhel (https://lab.llvm.org/buildbot/#/builders/57/builds/12628): Default linker override set (-DCLANG_DEFAULT_LINKER=lld). test/Driver/ve-toolchain.cpp test expecting nld to be the linker regardless.

Resolved

• clang-x64-windows-msvc (https://lab.llvm.org/buildbot/#/builders/123/builds/7509). Failure due to non-POSIX-compliant stream redirection in VE codegen tests. Fix by commit 444013d324fe2 "[VE][NFC] Use POSIX-compatible stream redirection"

[RKSimon]

I'm seeing this on MSVC builds:

E:\llvm\ninja>ninja llc
[5/4/2/10] Building CXX object lib\Target\VE\MCTa...c\CMakeFiles\LLVMVEDesc.dir\VEInstPrinter.cpp.obj
E:\llvm\llvm-project\llvm\lib\Target\VE\MCTargetDesc\VEInstPrinter.cpp(30): warning C4515: 'VE': namespace uses itself

[simoll]

Cause identified

• sanitizer-x86_64-linux-fast (https://lab.llvm.org/buildbot/#/builders/5/builds/15384/steps/13/logs/stdio) . Use-after-free (at least partially responsible).
• (Warning) E:\llvm\llvm-project\llvm\lib\Target\VE\MCTargetDesc\VEInstPrinter.cpp(30): warning C4515: 'VE': namespace uses itself

Resolved

• clang-ppc64le-rhel (https://lab.llvm.org/buildbot/#/builders/57/builds/12628): Default linker override set (-DCLANG_DEFAULT_LINKER=lld). Fixed by commit 6dd0bfad0d3c7b "[Clang][VE] Fix toolchain test when -DCLANG_DEFAULT_LINKER=lld in use"

[kda]

This looks like it is breaking the buildbots.
https://lab.llvm.org/buildbot/#/builders/5/builds/15384

log extract:

==27278==ERROR: AddressSanitizer: use-after-poison on address 0x62100113d058 at pc 0x0000097c0934 bp 0x7fff65832680 sp 0x7fff65832678
READ of size 8 at 0x62100113d058 thread T0
    #0 0x97c0933 in getOpcode /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/include/llvm/CodeGen/MachineInstr.h:489:39
    #1 0x97c0933 in (anonymous namespace)::ExpandPostRA::runOnMachineFunction(llvm::MachineFunction&) /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/CodeGen/ExpandPostRAPseudos.cpp:204:18
    #2 0x9acc148 in llvm::MachineFunctionPass::runOnFunction(llvm::Function&) /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/CodeGen/MachineFunctionPass.cpp:72:13
    #3 0xa9172cc in llvm::FPPassManager::runOnFunction(llvm::Function&) /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1439:27
    #4 0xa932af0 in llvm::FPPassManager::runOnModule(llvm::Module&) /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1485:16
    #5 0xa9194e8 in runOnModule /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1554:27
    #6 0xa9194e8 in llvm::legacy::PassManagerImpl::run(llvm::Module&) /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:542:44
    #7 0x4f2d66d in compileModule /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/tools/llc/llc.cpp:716:8
    #8 0x4f2d66d in main /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/tools/llc/llc.cpp:417:22
    #9 0x7f61f777b09a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2409a)
    #10 0x4e6c4f9 in _start (/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan/bin/llc+0x4e6c4f9)
0x62100113d058 is located 1880 bytes inside of 4096-byte region [0x62100113c900,0x62100113d900)
allocated by thread T0 here:
    #0 0x4f241dd in operator new(unsigned long) /b/s/w/ir/cache/builder/src/third_party/llvm/compiler-rt/lib/asan/asan_new_delete.cpp:95:3
    #1 0x53dc2e6 in Allocate /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/include/llvm/Support/AllocatorBase.h:85:12
    #2 0x53dc2e6 in llvm::BumpPtrAllocatorImpl<llvm::MallocAllocator, 4096ul, 4096ul, 128ul>::StartNewSlab() /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/include/llvm/Support/Allocator.h:335:21
    #3 0x53dc002 in llvm::BumpPtrAllocatorImpl<llvm::MallocAllocator, 4096ul, 4096ul, 128ul>::Allocate(unsigned long, llvm::Align) /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/include/llvm/Support/Allocator.h:190:5
    #4 0x9aa12f7 in Allocate /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/include/llvm/Support/Allocator.h:204:12
    #5 0x9aa12f7 in operator new<llvm::MallocAllocator, 4096UL, 4096UL, 128UL> /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/include/llvm/Support/Allocator.h:437:20
    #6 0x9aa12f7 in llvm::MachineFunction::init() /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/CodeGen/MachineFunction.cpp:161:15
    #7 0x9b64651 in llvm::MachineModuleInfo::getOrCreateMachineFunction(llvm::Function&) /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/CodeGen/MachineModuleInfo.cpp:303:14
    #8 0x9acbe50 in llvm::MachineFunctionPass::runOnFunction(llvm::Function&) /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/CodeGen/MachineFunctionPass.cpp:44:29
    #9 0xa9172cc in llvm::FPPassManager::runOnFunction(llvm::Function&) /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1439:27
    #10 0xa932af0 in llvm::FPPassManager::runOnModule(llvm::Module&) /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1485:16
    #11 0xa9194e8 in runOnModule /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:1554:27
    #12 0xa9194e8 in llvm::legacy::PassManagerImpl::run(llvm::Module&) /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/lib/IR/LegacyPassManager.cpp:542:44
    #13 0x4f2d66d in compileModule /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/tools/llc/llc.cpp:716:8
    #14 0x4f2d66d in main /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/tools/llc/llc.cpp:417:22
    #15 0x7f61f777b09a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2409a)
SUMMARY: AddressSanitizer: use-after-poison /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/include/llvm/CodeGen/MachineInstr.h:489:39 in getOpcode
Shadow bytes around the buggy address:
  0x0c428021f9b0: 00 00 f7 f7 00 00 00 00 00 00 00 00 00 f7 00 00
  0x0c428021f9c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c428021f9d0: 00 00 00 00 00 00 00 00 00 00 f7 00 00 00 00 00
  0x0c428021f9e0: 00 00 00 00 f7 00 00 00 00 00 00 00 00 00 00 00
  0x0c428021f9f0: 00 00 00 00 00 f7 00 00 00 00 00 00 00 00 00 f7
=>0x0c428021fa00: 00 00 00 00 00 00 00 00 f7 f7 f7[f7]f7 f7 f7 f7
  0x0c428021fa10: f7 f7 f7 00 00 00 00 00 00 00 00 f7 00 00 00 00
  0x0c428021fa20: 00 00 00 00 00 f7 00 00 00 00 00 00 00 00 f7 00
  0x0c428021fa30: 00 00 00 00 00 00 00 00 f7 00 00 00 00 f7 00 00
  0x0c428021fa40: 00 00 00 00 00 00 f7 00 00 00 00 00 00 00 00 00
  0x0c428021fa50: 00 00 00 00 00 00 00 f7 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==27278==ABORTING
/b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/CodeGen/VE/VELIntrinsics/vfmk.ll:1691:16: error: CHECK-LABEL: expected string not found in input
; CHECK-LABEL: vfmkslenan_mvl:
               ^
<stdin>:1338:17: note: scanning from here
vfmksgenan_mvml: 
                ^
Input file: <stdin>
Check file: /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm/test/CodeGen/VE/VELIntrinsics/vfmk.ll
-dump-input=help explains the following input dump.
Input was:
<<<<<<
            .
            .
            .
         1333:  .size vfmksgenan_mvl, .Lfunc_end110-vfmksgenan_mvl 
         1334:  # -- End function 
         1335:  .globl vfmksgenan_mvml # -- Begin function vfmksgenan_mvml 
         1336:  .p2align 4 
         1337:  .type vfmksgenan_mvml,@function 
         1338: vfmksgenan_mvml: 
label:1691                     X error: no match found
>>>>>>
--
********************
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90.. 
********************
Failed Tests (2):
  LLVM :: CodeGen/VE/VELIntrinsics/lvm.ll
  LLVM :: CodeGen/VE/VELIntrinsics/vfmk.ll

[simoll]

Status: VE reverted from official target list / clang-ve-ninja reporting to main buildbot instance and building VE as experimental target again

Unresolved

• sanitizer-x86_64-linux-fast LLVM :: CodeGen/VE/VELIntrinsics/vfmk.ll (https://lab.llvm.org/buildbot/#/builders/5/builds/15393) - fixing locally now

Cause identified

• (Warning) E:\llvm\llvm-project\llvm\lib\Target\VE\MCTargetDesc\VEInstPrinter.cpp(30): warning C4515: 'VE': namespace uses itself

Resolved

• sanitizer-x86_64-linux-fast LLVM :: CodeGen/VE/VELIntrinsics/lvm.ll (https://lab.llvm.org/buildbot/#/builders/5/builds/15384/steps/13/logs/stdio) - fixed by commit 435d44bf8ab "[VE][NFC] Fix use-after-free in VEInstrInfo"

[simoll]

In D113247#3165604, @kda wrote:

This looks like it is breaking the buildbots.
https://lab.llvm.org/buildbot/#/builders/5/builds/15384
[..]
Failed Tests (2):

LLVM :: CodeGen/VE/VELIntrinsics/lvm.ll
LLVM :: CodeGen/VE/VELIntrinsics/vfmk.ll

These are two use-after-free bugs that i could reproduce locally with an asan-enabled build. I have pushed to fixes that resolve this locally:

• commit e37000f3bff
• commit 435d44bf8ab

I will revert the revert of D113247 when these two patches have passed clang-ve-ninja buildbot with a green status (just to double check..).

[simoll]

Status: VE relanded as official target / clang-ve-ninja reporting to main buildbot instance and building VE as official target again

Cause identified

• (Warning) E:\llvm\llvm-project\llvm\lib\Target\VE\MCTargetDesc\VEInstPrinter.cpp(30): warning C4515: 'VE': namespace uses itself
• clang-x64-windows-msvc (https://lab.llvm.org/buildbot/#/builders/123/builds/7520) - Windows path separators unexpected in VE toolchain tests .. testing a patch locally

Resolved

• sanitizer-x86_64-linux-fast LLVM :: CodeGen/VE/VELIntrinsics/lvm.ll (https://lab.llvm.org/buildbot/#/builders/5/builds/15384/steps/13/logs/stdio) - fixed by commit 435d44bf8ab "[VE][NFC] Fix use-after-free in VEInstrInfo"
• sanitizer-x86_64-linux-fast LLVM :: CodeGen/VE/VELIntrinsics/vfmk.ll (https://lab.llvm.org/buildbot/#/builders/5/builds/15393) - fixing locally now

[simoll]

Cautiously calling the transition to official complete.. there may be some slow results trickling in still. Will wait until next week before lifting the feature freeze on VE.

One issue that showed in clang's test/Driver/ve-toolchain tests is the unfortunate behavior of the -DCLANG_DEFAULT_LINKER cmake flag and the -fuse-ld=ld option: when some toolchain does not support the configured default linker (eg VE, whose only is named nld), linker invocations by that toolchain will be invalid. Proposing a fix for that.

llvm/CMakeLists.txt

set(LLVM_ALL_TARGETS
   Lanai
   Mips
   MSP430
   NVPTX
   PowerPC
   RISCV
   Sparc
   SystemZ
+  VE
   WebAssembly
   X86
   XCore
   )
 
 # List of targets with JIT support:
 set(LLVM_TARGETS_WITH_JIT X86 PowerPC AArch64 ARM Mips SystemZ)