Hi Louis,

https://reviews.llvm.org/D100704

Hi, I'm actually wondering how this is working for anyone.

Updating diff to most recent: Want to see if CI actually starts working..

Apologies. I uploaded the full diff again, rather than the refinement I wanted to.

• Made ViewMISChedDAG a global option to match ForceTopDown et al.
• Optimized includes in VLIWMachineScheduler.h
• Replaced vectors in VLIWMachineScheduler.h with SmallVector
• Ran clang-format on all changes, which includes the entirety of the VLIWMachineScheduler files, as they are treated as 'new' files

As-is, this patch will have the unintended effect that Clang and the
LLVM libraries (not only the runtime ones like libc++) will also be
built with -fPIC in the release.

In D113150#3118950, @kparzysz wrote:

I committed a small patch (https://reviews.llvm.org/rGa721ddbae983), it changes releaseTopNode and releaseBottomNode, you'll need to rebase before merging.

Performance tests look ok.

Thanks!

Comments for review added.

This review is not yet formatted, but I'll upload a formatted version prior to commit once we find the changes sufficient. I wanted to avoid unrelated code changes from being injected into the review.

You're welcome! Surprising that no downstream clang was running with the old PM and assertions...

Perhaps you could reproduce my error -round-trip-args on the -cc1 command line?

The trigger for the remark I'm seeing is llvm::shouldInline in InlineAdvisor.cpp ((https://github.com/llvm/llvm-project/blob/2240deb9766cc080b351016b0d7f975d7249b113/llvm/lib/Transforms/IPO/Inliner.cpp#L425) which is called in a top-level AlwaysInlinerLegacyPass

I'll take a quick look tomorrow, but the general idea is that on calling ParseOptimizationRemark on line 1909 with a -cc1 command line containing -Rpass=inline -Rno-pass, Opts.OptimizationRemarkMissed and Opts.OptimizationRemarkAnalysis are set to valid patterns (Regex is ".*", Pattern is "", and Kind is RK_Missing). This happens around line 1909 in the change set.

Have they added a way to do that? I recall there being some discussions, but as far as I know, I'd have to XFAIL the whole test, not just specific lines.
Either way, this isn't a blocking issue on our end. I just wanted to bring it up in case someone else has the same problem and can see from where it originates.

In rGe1ed02181ffcfea7de952d252a5e25953c822251#1028369, @aeubanks wrote:

I don't think that -Rpass turns on pass-misses/analysis, that'd be -Rpass-analysis/-Rpass-missed.

Pass manager problems? That crossed my mind because we have been slow on the uptake of the new PM, but seeing the behavior in the clang function made me believe that it couldn't have been the case. Wouldn't just making -Rpass not turn on pass-misses and pass-analysis solve the issue?

In this commit, our team is seeing a failure in the optimization_remark.c test you modified, but not on a line you changed. The failure occurs on line 16, and I'm trying to understand why other upstream targets aren't seeing the same.
The relevant command line options are:

Adding relevant people from my team.

Hi all,

In D105310#2854283, @efriedma wrote:

I suspect this affects arm64_32, at least in terms of the generated SelectionDAG.

I want to apologize for missing such obvious things. Perhaps I used all of my observational power on my initial investigation. Of course it checks the triple for *hf and then the leading arch for arm, since that's how the property is added to the triple (*-eabihf)

Oh! Sorry, I glazed over that block and missed the set. I suspect that an architecture called '^arm.*hf$' doesn't cause issues in builtins specifically because it seems there's no special casing for anything besides 'armhf'

In rG8bd2722f65cfd7883ed9769f7bad3ff50e4c6905#982445, @mstorsjo wrote:

The toplevel CMakeLists also does happen to have such a case though, see https://github.com/llvm/llvm-project/blob/main/compiler-rt/CMakeLists.txt#L111-L114. IMO that also would lose all the details between various arm architecture variants. But maybe that just doesn't trigger for ones that are sensitive enough for the distinction to matter?

In rG8bd2722f65cfd7883ed9769f7bad3ff50e4c6905#982263, @mstorsjo wrote:

I'm sorry about that. Do I guess correctly that you're building with LLVM_ENABLE_PER_TARGET_RUNTIME_DIR enabled, as otherwise, clang would just look for all files in the same directory, and expect all of them to be named libclang_rt.builtins-arm.a or something like that? In such a setup, I guess that one doesn't want the normalization - but I hadn't really expect it to break anything either.

This commit has broken our downstream toolchain builds which utilizes the llvm runtimes system to compile multiple sets of libraries with a just-built cross compiler.

I believe our teams' downstream Darwin (cross compiler to ARM) builds are failing due to this commit:

In llvm/test/DebugInfo/dwarfdump-debug-frame-simple.test

In D53005#2064021, @fpichet wrote:

I am working on an out of tree target and this would be useful.
So what is the status of this patch?

In D83174#2230763, @aaron.ballman wrote:

Unfortunately, I had to revert the change as it was causing some buildbots to fail. I reverted in 58c305f466d1f78adb10e7295b9bc9fc192a6e09. Some failures include:

http://lab.llvm.org:8011/builders/clang-cmake-armv7-quick/builds/20294
http://lab.llvm.org:8011/builders/clang-ppc64le-linux-multistage/builds/13600

I'm not certain why b.h is not being found by those builders. Do you mind investigating?

In rG04a6ea5d77e7613a5e1398ddf2a0fcb4e1cea41c#942027, @aemerson wrote:

In rG04a6ea5d77e7613a5e1398ddf2a0fcb4e1cea41c#942000, @JamesNagurne wrote:

This commit is causing build failures in my downstream ARM repository, as well as at least one buildbot (http://lab.llvm.org:8011/builders/clang-cmake-x86_64-avx2-linux)

The errors seem to stem from the std::tuple implementation. Could it be that the buildbot and my downstream are using older, unsupported library implementations?

Could be, but I'm not certain. In any case, I've just changed them to use std::make_tuple now instead in ed353445248.

This commit is causing build failures in my downstream ARM repository, as well as at least one buildbot (http://lab.llvm.org:8011/builders/clang-cmake-x86_64-avx2-linux)

Abandoning in lieu of fix in D85215

Moved test to X86 subdirectory

Either of these options sounds good to me.

I apologize for the constant early changes. I forget how phabricator catches and marks things as being referenced, so I was trying to get it to work.
This is a patch fix for D69384, which has broken ARM build bots. I present this as a band-aid since this feature and its test seems to be rather generic.

In D69384#2190796, @ostannard wrote:

This test case has been failing on many of the ARM and AArch64 bots all weekend, e.g. http://lab.llvm.org:8011/builders/clang-cmake-armv7-quick/builds/19408, would you mind looking into this?

Here's the log of one of the failures (from http://lab.llvm.org:8011/builders/clang-cmake-armv7-quick/builds/19408/steps/ninja%20check%201/logs/FAIL%3A%20LLVM%3A%3Acoldentrycount.ll):

******************** TEST 'LLVM :: Transforms/HotColdSplit/coldentrycount.ll' FAILED ********************
Script:
--
: 'RUN: at line 4';   /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/opt -hotcoldsplit -hotcoldsplit-threshold=0 -codegenprepare -S < /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/llvm/llvm/test/Transforms/HotColdSplit/coldentrycount.ll | /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/FileCheck /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/llvm/llvm/test/Transforms/HotColdSplit/coldentrycount.ll
--
Exit Code: 2

Command Output (stderr):
--
LLVM ERROR: Trying to construct TargetPassConfig without a target machine. Scheduling a CodeGen pass without a target triple set?
PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace.
Stack dump:
0.	Program arguments: /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/opt -hotcoldsplit -hotcoldsplit-threshold=0 -codegenprepare -S 
FileCheck error: '<stdin>' is empty.
FileCheck command line:  /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/FileCheck /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/llvm/llvm/test/Transforms/HotColdSplit/coldentrycount.ll

--

********************

In D72947#2047140, @beanz wrote:

Sorry for being late to the party on this.

There's really only a handful of variables in LLVMConfig.cmake that can actually be different between LLVM and the project using LLVM. Might it make sense to just handle those as special? We already do handle LLVM_PTHREAD_LIB specially for exactly the same reason you describe in the description.

The problem here is the method we reset the variables in the cache back to their cached value.

Actually, it's very fortuitous that you reminded of this review today.

When @phosek gets back to it, one of you would need to commit it on my behalf.

Unfortunately I don't have commit access.
Could one of you get this onto the upstream?

This commit broke our validations. Specifically, the tests in llvm/test/tools/opt-viewer

@bkramer committed a fix for a crash in LegalizeFloatTypes where this operator appeared in SoftPromoteHalfResult

Thanks @phosek

In D72947#1850273, @phosek wrote:

This looks reasonable to me, @smeenai & @beanz do you see any problem with this?

In D72952#1841048, @mstorsjo wrote:

In D72952#1840841, @mstorsjo wrote:

In D72952#1840794, @mstorsjo wrote:

This broke my build, for both mingw and linux, with both cmake 3.10.2 and 3.16.2 - the assembly files don't get built at all. Reproducible on linux with a plain "cmake .." (in llvm-project/libunwind/build) without further args.

Is some custom build config needed for this to work?

Ok, so it seems this works when built via the main llvm cmakefile, but not standalone. So I guess the libunwind cmakefiles needs some extras for hooking this up, that are included only when built standalone.

I fixed this in https://reviews.llvm.org/rG0e0c65264aeb6f66b6f711884c55cdbf66d975f6, but building assembly files as assembly instead of C breaks building for mingw, for which I sent an additional patch at D73436, essentially conditionally reverting this change for mingw targets.

Moved option into a block guarded by COMPILER_RT_STANDLONE_BUILD, to indicate that it is an option that is available when not utilizing a linked LLVM toolchain and its configuration.

Updated to reflect master -> diff 2, rather than diff 1 -> diff 2

In D72952#1827668, @smeenai wrote:

The diff here looks weird ... it looks like a diff against your previous diff instead of a diff against master? (Phabricator says the line you're deleting has LANGUAGE ASM instead of LANGUAGE C, as it does on master.)

Confirmed that cmake correctly identifies '.S' as ASM, and uses CMAKE_ASM_FLAGS

@phosek or really any other committer, could we get this onto master?

Sadly, I do not have commit access. Could you do the honors @phosek ?

In D71570#1792087, @phosek wrote:

I'm fine with this change in general, but I'm concerned that no matter what order we choose, this might break users who rely on the opposite order. Maybe it'd be better to modify the logic to skip the second loop if the first loop matches, that way we wouldn't rely on any particular order.

This foreach loop goes over each variable, yes?
If so, then I'm not sure this change does what is required.

This commit has broken a few build bots.

Sadly I'm home from the weekend, so I can't post more diffs for the renaming.
Here's a link to a github search though! I think this would be all the places: https://github.com/llvm/llvm-project/search?q=iterface&unscoped_q=iterface

In D63978#1706714, @plotfi wrote:

In D63978#1706502, @JamesNagurne wrote:

In D63978#1706448, @plotfi wrote:

In D63978#1706420, @JamesNagurne wrote:

Our team maintains a downstream embedded ARM clang distribution and some tests from this commit have begun to fail for us.
For a number of these tests, there was a REQUIRES: x86-registered-target at the top, which has now been removed. Specifically, externstatic.c, merge-conflict-test.c, object-float.c, and object.c are failing.

object* tests seem to be based on object.cpp, which had the REQUIRES line, and externstatic.c also had that line prior to the change.
I see that @compnerd suggested the removal, but were you certain that these tests would work on clang toolchains for which x86 is not a registered target?

For a failure example, here the output of lit for our toolchain. If you can make sense of it, I'd appreciate input on how we can fix or work around it:

> <WORKDIR>/arm-llvm/Release/llvm/bin/clang -c -o - -emit-interface-stubs <WORKDIR>/llvm-project/clang/test/InterfaceStubs/object.c | <WORKDIR>/arm-llvm/Release/llvm/bin/FileCheck -check-prefix=CHECK-TAPI <WORKDIR>/llvm-project/clang/test/InterfaceStubs/object.c
<WORKDIR>/llvm-project/clang/test/InterfaceStubs/object.c:5:16: error: CHECK-TAPI: expected string not found in input
// CHECK-TAPI: data: { Type: Object, Size: 4 }
               ^
<stdin>:1:1: note: scanning from here
--- !experimental-ifs-v1
^

And when run without FileCheck, our raw output:

> <WORKDIR>/arm-llvm/Release/llvm/bin/clang -c -o - -emit-interface-stubs <WORKDIR>/llvm-project/clang/test/InterfaceStubs/object.c
--- !experimental-ifs-v1
IfsVersion: 1.0
Triple: thumbv7em-ti-none-eabihf
ObjectFileFormat: ELF
Symbols:
...

I am sorry for this James. I can add back the REQUIRES lines for now and coordinate with you on making sure your downstream bots are not affected again if the REQUIRES are removed again.
By chance are your bots accessible publicly?

Sadly, they are not. It's on our list of things to investigate, but we don't have the resources to do such a thing quite yet.
I'm looking into the 'arm7*' buildbots to see if they are built similar to ours so I am not leaving you entirely without something to look at. However, if it seems to be common knowledge to always include an X86 target, I think I can talk to my team and change up what we do.

These buildbots seem to also do LLVM_TARGETS_TO_BUILD=ARM, and then set the default target triple to a non-x86 triple (the host's)

That could point towards us being in error here. I'll investigate things a little further, and update when I get the chance.
To be clear: this feature should work for any ELF target, correct?

Yes, it is designed to work for all ELF targets but at the moment it is still in an early state. I am on the llvm IRC as zer0_ BTW

In D63978#1706448, @plotfi wrote:

In D63978#1706420, @JamesNagurne wrote:

Our team maintains a downstream embedded ARM clang distribution and some tests from this commit have begun to fail for us.
For a number of these tests, there was a REQUIRES: x86-registered-target at the top, which has now been removed. Specifically, externstatic.c, merge-conflict-test.c, object-float.c, and object.c are failing.

object* tests seem to be based on object.cpp, which had the REQUIRES line, and externstatic.c also had that line prior to the change.
I see that @compnerd suggested the removal, but were you certain that these tests would work on clang toolchains for which x86 is not a registered target?

For a failure example, here the output of lit for our toolchain. If you can make sense of it, I'd appreciate input on how we can fix or work around it:

> <WORKDIR>/arm-llvm/Release/llvm/bin/clang -c -o - -emit-interface-stubs <WORKDIR>/llvm-project/clang/test/InterfaceStubs/object.c | <WORKDIR>/arm-llvm/Release/llvm/bin/FileCheck -check-prefix=CHECK-TAPI <WORKDIR>/llvm-project/clang/test/InterfaceStubs/object.c
<WORKDIR>/llvm-project/clang/test/InterfaceStubs/object.c:5:16: error: CHECK-TAPI: expected string not found in input
// CHECK-TAPI: data: { Type: Object, Size: 4 }
               ^
<stdin>:1:1: note: scanning from here
--- !experimental-ifs-v1
^

And when run without FileCheck, our raw output:

> <WORKDIR>/arm-llvm/Release/llvm/bin/clang -c -o - -emit-interface-stubs <WORKDIR>/llvm-project/clang/test/InterfaceStubs/object.c
--- !experimental-ifs-v1
IfsVersion: 1.0
Triple: thumbv7em-ti-none-eabihf
ObjectFileFormat: ELF
Symbols:
...

I am sorry for this James. I can add back the REQUIRES lines for now and coordinate with you on making sure your downstream bots are not affected again if the REQUIRES are removed again.
By chance are your bots accessible publicly?

Our team maintains a downstream embedded ARM clang distribution and some tests from this commit have begun to fail for us.
For a number of these tests, there was a REQUIRES: x86-registered-target at the top, which has now been removed. Specifically, externstatic.c, merge-conflict-test.c, object-float.c, and object.c are failing.

Ah! My apologies. I had not pulled anything new down, as our process tends to hard-stop until an issue is resolved (something I'm thinking needs to be a little more flexible).

As mentioned in an inline comment, the test only works if the sizeof(int) != sizeof(int *) and sizeof(unsigned long long) == sizeof(int)

In D65907#1645474, @arphaman wrote:

In D65907#1643800, @JamesNagurne wrote:

In D65907#1643650, @arphaman wrote:

No the windows test failure was different, there were no Deps at all. I'm currently investigating it on a windows VM.

@JamesNagurne I think there's some issue with the working directory, which is not added in your case. Which platform are you running your build/test on? Which cmake options are you using?

I apologize for not giving such information in the first reply. Unfortunately this isn't an easy remote reproduction, as our ToolChain and some integral changes aren't upstreamed. This is an embedded ARM cross-compiled on Linux. Might be able to reproduce with arm-none-none-eabi.
LLVM is built as an external project. Looking at the build system, it looks like we have the CMAKE_ARGS:

-DLLVM_DEFAULT_TARGET_TRIPLE=arm-ti-none-eabi
-DLLVM_EXTERNAL_CLANG_SOURCE_DIR=${CMAKE_SOURCE_DIR}/llvm-project/clang
-DLLVM_TARGETS_TO_BUILD=ARM
-DCMAKE_BUILD_TYPE=Release
-DCMAKE_CXX_COMPILER=clang++
-DCMAKE_C_COMPILER=clang
-DLLVM_USE_LINKER=gold
-GNinja

Nothing suspicious, except maybe the default triple and ARM target.
Looking at our (not upstream) toolchain file, we do have some RTLibs, LibInternal, libcxx, and System include changes, along with a -nostdsysteminc to avoid pulling host includes into our cross compiler. However, none of this should affect general "#include" behavior, correct?
Another glance at your changes don't seem to affect any target-specific handling either, at least directly.

Yes, I don't see anything in your changes that is an obvious thing to blame.

I might have to just bite the bullet and drop into the test with a debugger.

I fixed the Windows issue, and it was completely unrelated to your issue. It looks like the FileManager isn't constructing absolute paths when accessing the files, which is somewhat unexpected. It would be useful to debug invocations of getFileEntryRef, to see if the InterndFilename in the function is getting changed into an absolute path or not.

In D65907#1643650, @arphaman wrote:

No the windows test failure was different, there were no Deps at all. I'm currently investigating it on a windows VM.

@JamesNagurne I think there's some issue with the working directory, which is not added in your case. Which platform are you running your build/test on? Which cmake options are you using?