Page MenuHomePhabricator

[llvm] Update IR verifier to reject non-power-of-2 alignment assume bundles (PR48713).
AbandonedPublic

Authored by rmansfield on Sep 28 2021, 8:50 AM.

Details

Diff Detail

Unit TestsFailed

TimeTest
150 msx64 debian > HWAddressSanitizer-x86_64.TestCases/Linux::aligned_alloc-alignment.cpp
Script: -- : 'RUN: at line 1'; /var/lib/buildkite-agent/builds/llvm-project/build/./bin/clang --driver-mode=g++ -m64 -gline-tables-only -fsanitize=hwaddress -fuse-ld=lld -fsanitize-hwaddress-experimental-aliasing -mcmodel=large -mllvm -hwasan-generate-tags-with-calls=1 -mllvm -hwasan-globals -mllvm -hwasan-use-short-granules -mllvm -hwasan-instrument-landing-pads=0 -mllvm -hwasan-instrument-personality-functions -O0 /var/lib/buildkite-agent/builds/llvm-project/compiler-rt/test/hwasan/TestCases/Linux/aligned_alloc-alignment.cpp -o /var/lib/buildkite-agent/builds/llvm-project/build/projects/compiler-rt/test/hwasan/X86_64/TestCases/Linux/Output/aligned_alloc-alignment.cpp.tmp
60 msx64 debian > ORC-x86_64-linux.TestCases/Linux/x86-64::trivial-cxa-atexit.S
Script: -- : 'RUN: at line 3'; /var/lib/buildkite-agent/builds/llvm-project/build/./bin/clang -m64 -c -o /var/lib/buildkite-agent/builds/llvm-project/build/projects/compiler-rt/test/orc/X86_64LinuxConfig/TestCases/Linux/x86-64/Output/trivial-cxa-atexit.S.tmp /var/lib/buildkite-agent/builds/llvm-project/compiler-rt/test/orc/TestCases/Linux/x86-64/trivial-cxa-atexit.S
80 msx64 debian > ORC-x86_64-linux.TestCases/Linux/x86-64::trivial-static-initializer.S
Script: -- : 'RUN: at line 7'; /var/lib/buildkite-agent/builds/llvm-project/build/./bin/clang -m64 -c -o /var/lib/buildkite-agent/builds/llvm-project/build/projects/compiler-rt/test/orc/X86_64LinuxConfig/TestCases/Linux/x86-64/Output/trivial-static-initializer.S.tmp /var/lib/buildkite-agent/builds/llvm-project/compiler-rt/test/orc/TestCases/Linux/x86-64/trivial-static-initializer.S
90 msx64 debian > ORC-x86_64-linux.TestCases/Linux/x86-64::trivial-tls.S
Script: -- : 'RUN: at line 1'; /var/lib/buildkite-agent/builds/llvm-project/build/./bin/clang -m64 -c -o /var/lib/buildkite-agent/builds/llvm-project/build/projects/compiler-rt/test/orc/X86_64LinuxConfig/TestCases/Linux/x86-64/Output/trivial-tls.S.tmp /var/lib/buildkite-agent/builds/llvm-project/compiler-rt/test/orc/TestCases/Linux/x86-64/trivial-tls.S
2,830 msx64 debian > Scudo-x86_64.Scudo-x86_64::aligned-new.cpp
Script: -- : 'RUN: at line 1'; /var/lib/buildkite-agent/builds/llvm-project/build/./bin/clang -m64 -pthread -fPIE -pie -O0 -UNDEBUG -ldl -Wl,--gc-sections -lrt --driver-mode=g++ -std=c++11 -fsanitize=scudo -std=c++1z -faligned-allocation /var/lib/buildkite-agent/builds/llvm-project/compiler-rt/test/scudo/aligned-new.cpp -o /var/lib/buildkite-agent/builds/llvm-project/build/projects/compiler-rt/test/scudo/X86_64LinuxConfig/Output/aligned-new.cpp.tmp

Event Timeline

rmansfield created this revision.Sep 28 2021, 8:50 AM
rmansfield requested review of this revision.Sep 28 2021, 8:50 AM
Herald added a project: Restricted Project. · View Herald TranscriptSep 28 2021, 8:50 AM

I think this looks good, need to update some tests though.

I think this looks good, need to update some tests though.

Thanks for the review. The compiler-rt tests appear to be failing because the clang change not to emit the non-power of two assume bundle doesn't seem to be applied and the tests generate IR that is now being rejected.

e.g.

/var/lib/buildkite-agent/builds/llvm-project/compiler-rt/test/hwasan/TestCases/Linux/aligned_alloc-alignment.cpp:15:27: warning: requested alignment is not a power of 2 [-Wnon-power-of-two-alignment]
  void *p = aligned_alloc(17, 100);
                          ^~
alignment must be a power of 2
  call void @llvm.assume(i1 true) [ "align"(i8* %call, i64 17) ], !dbg !19
in function main
fatal error: error in backend: Broken function found, compilation aborted!

With both patches applied the tests pass locally. I'm not sure how to link the reviews such the build bots will build with both changes applied.

nikic added inline comments.Sep 28 2021, 11:11 AM
llvm/lib/IR/Verifier.cpp
4668

I believe this should be Assert(Alignment && Alignment->getValue().isPowerOf2()). That is, we should be requiring the alignment to be a constant. I don't think it makes sense to enforce power of two only for constant alignments, but leave non-constant alignments unconstrained -- and non-constant alignments don't really make sense to me in this context.

@jdoerfert wasn't the alignment intentionally been left as not being required to be constant?

@jdoerfert wasn't the alignment intentionally been left as not being required to be constant?

Yes. Though, I'm not even sure we emit/use non-constant alignments right now. If we emit them we would use them if they are proven to be constant later.
More elaborate support was eventually planned though. I would not require constants here, I overlooked that earlier.

nikic added a comment.Oct 2 2021, 7:57 AM

@jdoerfert wasn't the alignment intentionally been left as not being required to be constant?

Yes. Though, I'm not even sure we emit/use non-constant alignments right now. If we emit them we would use them if they are proven to be constant later.
More elaborate support was eventually planned though. I would not require constants here, I overlooked that earlier.

Well, if we don't require constants here then we can't add this check -- this means that some valid input IR (with non-constant non-power-of-two alignment) could become invalid after folding.

Allowing non-constants here doesn't make a great deal of sense to me, in terms of practical usefulness. My understanding was that these operand bundles are supposed to model the corresponding metadata, and that doesn't allow non-constant values either.

nikic requested changes to this revision.Oct 2 2021, 8:00 AM

Marking this as changes requested, because we have to either forbid non-constants, or we have to give up on this approach and say that consuming code needs to expect and handle invalid alignments. I don't think there's any in-between options here.

This revision now requires changes to proceed.Oct 2 2021, 8:00 AM

@jdoerfert wasn't the alignment intentionally been left as not being required to be constant?

Yes. Though, I'm not even sure we emit/use non-constant alignments right now. If we emit them we would use them if they are proven to be constant later.
More elaborate support was eventually planned though. I would not require constants here, I overlooked that earlier.

Well, if we don't require constants here then we can't add this check -- this means that some valid input IR (with non-constant non-power-of-two alignment) could become invalid after folding.

That is not wrong but also not new, I'd think. The verifier does local verification and after transformations things may be exposed that were missed before. It is always invalid for someone to put a non-power-of-two alignment somewhere and the only questions is when we identify that and reject it.

Allowing non-constants here doesn't make a great deal of sense to me, in terms of practical usefulness. My understanding was that these operand bundles are supposed to model the corresponding metadata, and that doesn't allow non-constant values either.

That is true except that we have alignment source annotation that allows non-constant values. E.g., the alignment of the return is the second call argument to this function. If we don't allow non-constant alignments here we loose the ability to express these source annotations in IR.

Marking this as changes requested, because we have to either forbid non-constants, or we have to give up on this approach and say that consuming code needs to expect and handle invalid alignments. I don't think there's any in-between options here.

I don't think the argument against an in-between option is actually something we have to worry about. As mentioned before, the only impact is that we would reject problematic code later rather than never.

nikic added a comment.Oct 2 2021, 3:27 PM

Well, if we don't require constants here then we can't add this check -- this means that some valid input IR (with non-constant non-power-of-two alignment) could become invalid after folding.

That is not wrong but also not new, I'd think. The verifier does local verification and after transformations things may be exposed that were missed before. It is always invalid for someone to put a non-power-of-two alignment somewhere and the only questions is when we identify that and reject it.

You're right, but...

Allowing non-constants here doesn't make a great deal of sense to me, in terms of practical usefulness. My understanding was that these operand bundles are supposed to model the corresponding metadata, and that doesn't allow non-constant values either.

That is true except that we have alignment source annotation that allows non-constant values. E.g., the alignment of the return is the second call argument to this function. If we don't allow non-constant alignments here we loose the ability to express these source annotations in IR.

...actually making use of non-constant alignments would still require the frontend to ensure that the dynamic alignment is a power of two, otherwise it may fail verification after folding. I guess there are cases where this can indeed be ensured (alignment of the form 1 << dynamic), but the particular alloc_align case here does not fall in that category. If we declared that non-power-of-2 alignments are illegal IR (as opposed to just UB), then clang wouldn't actually be able to emit the dynamic alignment case, because it could not guarantee that the resulting IR is valid.

After looking at the alloc_align attribute use-case, I've come around to the idea that non-constant alignments can be useful (e.g. so we don't have to drop the info just because the call is happening through a wrapper). In which case I think we should just give up on this, and keep non-power-of-2 alignment as UB rather than invalid IR. Callers will just have to deal with it. Though possibly there could be a helper that only extracts the constant power of two case from the operand bundle and ignores other cases.

Well, if we don't require constants here then we can't add this check -- this means that some valid input IR (with non-constant non-power-of-two alignment) could become invalid after folding.

That is not wrong but also not new, I'd think. The verifier does local verification and after transformations things may be exposed that were missed before. It is always invalid for someone to put a non-power-of-two alignment somewhere and the only questions is when we identify that and reject it.

You're right, but...

Allowing non-constants here doesn't make a great deal of sense to me, in terms of practical usefulness. My understanding was that these operand bundles are supposed to model the corresponding metadata, and that doesn't allow non-constant values either.

That is true except that we have alignment source annotation that allows non-constant values. E.g., the alignment of the return is the second call argument to this function. If we don't allow non-constant alignments here we loose the ability to express these source annotations in IR.

...actually making use of non-constant alignments would still require the frontend to ensure that the dynamic alignment is a power of two, otherwise it may fail verification after folding. I guess there are cases where this can indeed be ensured (alignment of the form 1 << dynamic), but the particular alloc_align case here does not fall in that category. If we declared that non-power-of-2 alignments are illegal IR (as opposed to just UB), then clang wouldn't actually be able to emit the dynamic alignment case, because it could not guarantee that the resulting IR is valid.

After looking at the alloc_align attribute use-case, I've come around to the idea that non-constant alignments can be useful (e.g. so we don't have to drop the info just because the call is happening through a wrapper). In which case I think we should just give up on this, and keep non-power-of-2 alignment as UB rather than invalid IR. Callers will just have to deal with it. Though possibly there could be a helper that only extracts the constant power of two case from the operand bundle and ignores other cases.

And if someone thinks this isn't fun enough yet, passing invalid alignment (not a power of two) isn't actually UB in C: http://www.open-std.org/jtc1/sc22/wg14/www/docs/summary.htm#dr_460

I am worried we might need to check the alignment requirement at the use site instead after all...

rmansfield abandoned this revision.Apr 5 2022, 8:43 AM
Herald added a project: Restricted Project. · View Herald TranscriptApr 5 2022, 8:43 AM