Use-after-return sanitizer binary metadata

[dvyukov]

Summary

Currently per-function metadata consists of:
(start-pc, size, features)

This adds a new UAR feature and if it's set an additional element:
(start-pc, size, features, stack-args-size)

[dvyukov]

WIP

[dvyukov]

WIP

[dvyukov]

WIP

[dvyukov]

Marco, does this look reasonable? Any high-level comments?

There are lots of plumbing to add a new flag and integrate a new pass, so you can look only at:
llvm/lib/Transforms/Instrumentation/SanitizerBinaryMetadata.cpp
llvm/lib/CodeGen/SanitizerMetadata.cpp

As far as I understand the stack layout is only known in machine passes via MachineFrameInfo,
so I had to add a machine pass.

[dvyukov]

Unrelated, but looking at the metadata format more closely, I think this can benefit from LEB128-like varlen encoding.
Function size is small.
Features are very small.
Stack args size is small.
Function start (if we encode it from the end of the previous one) is very small as well.
So potentially it can reduce the entry size from 16 bytes to just 4.
I am just thinking if we use 10x more metadata in future, the size can grow from percents of binary to tens of percents.

[dvyukov]

take args alignment into account

[melver]

In D136078#3867627, @dvyukov wrote:

Unrelated, but looking at the metadata format more closely, I think this can benefit from LEB128-like varlen encoding.
Function size is small.
Features are very small.
Stack args size is small.
Function start (if we encode it from the end of the previous one) is very small as well.

Function start is currently encoded as a relative relocation (see AsmPrinter). Anything else, including attempting to use a previous function does not work because of section merging, potential discarding of functions by linker (ffunction-sections), etc.

We could implement compression in the linker, which would be the most reliable. Otherwise we can only play with encoding of the aux constants.

So potentially it can reduce the entry size from 16 bytes to just 4.
I am just thinking if we use 10x more metadata in future, the size can grow from percents of binary to tens of percents.

If that's the case, I think the only option is to implement compression in the linker.

llvm/include/llvm/Transforms/Instrumentation.h

166

inline constexpr

But overall, I'd prefer if these wouldn't need to be defined here, but instead in SanitizerBinaryMetadata.h.

Also, re naming "SanitizerMetadata" is already overloaded and we need to stick to SanitizerBinaryMetadata :-/

llvm/lib/CodeGen/SanitizerMetadata.cpp

1

SanitizerMetadata is already overloaded for other purposes (see clang/lib/CodeGen/SanitizerMetadata.cpp).

(Which is why I had to choose SanitizerBinaryMetadata)

47

I think you also need to check the target section name matches (in operand 0). Because having !pcsections does not imply SanitizerMetadata enabled.

Perhaps at this point it might make sense to move MetadataInfo to SanitizerBinaryMetadata.h (as SanitizerBinaryMetadataInfo perhaps ... long I know :-/ .. unfortunately "SanitizerMetadata" is already overloaded elsewhere -- SanitizerBinaryMDInfo?).

49

APInt

llvm/lib/IR/MDBuilder.cpp

176

This is not great. We still want MDNode interning to work for these.

Doesn't replaceOperandWith() work where you want to replace the operand?

[dvyukov]

rebase and adressed comments

[dvyukov]

PTAL

I've named everything consistently as "SanitizeBinaryMetadata",
fixed interning of metadata, and added a test.

[dvyukov]

llvm/lib/CodeGen/SanitizerBinaryMetadata.cpp

79 ↗

(On Diff #478168)

Should this be a new method on MDBuilder?

[melver]

llvm/include/llvm/Transforms/Instrumentation/SanitizerBinaryMetadata.h

38 ↗

(On Diff #478168)

inline constexpr char[]

llvm/lib/CodeGen/SanitizerBinaryMetadata.cpp

79 ↗

(On Diff #478168)

As discussed, this could just use the MDBuilder, and then extract the operand out of the new MDNode without using the full MDNode.

llvm/lib/Transforms/Instrumentation/SanitizerBinaryMetadata.cpp

242

I think this is no longer PerInstrFeatureMask, but should instead become FeatureMask.

245

Comment why also || Options.UAR (because PerInstrFeatureMask setting of UAR bit is deferred?).

252

What if Options.Covered==true?

Will it still emit some UAR metadata or will it emit something it shouldn't?

Should the F.isVarArg() check be done above in if (PerInstrFeatureMask || (Options.UAR && !F.isVarArg()) ? Then you wouldn't need the PerInstrFeatureMask && RequiresCovered change below and it could still just check RequiresCovered as before.

[melver]

llvm/test/Instrumentation/SanitizerBinaryMetadata/uar.cpp

1 ↗

(On Diff #478168)

For LLVM IR tests, consider if utils/update_test_checks.py and for MIR utils/update_mir_test_checks.py could help.

Ideally, there'd be no .cpp runtime test, but I also think writing the .ll and .mir tests manually is impossible.

[dvyukov]

addressed comments

[dvyukov]

PTAL

llvm/lib/Transforms/Instrumentation/SanitizerBinaryMetadata.cpp

252

What if Options.Covered==true?
Will it still emit some UAR metadata or will it emit something it shouldn't?

If Options.Covered==true && F.isVarArg(), we emit covered metadata with features=0. This looks WAI.

Should the F.isVarArg() check be done above in if (PerInstrFeatureMask || (Options.UAR && !F.isVarArg()) ? Then you wouldn't need the PerInstrFeatureMask && RequiresCovered change below and it could still just check RequiresCovered as before.

It's tricky.
I forgot why I structured the code this way, but I added a new test for all permutations of covered/atomics/uar and it shows the following breakage with your proposed change:

- CHECK-AU: ellipsis: features=1 stack_args=0
+ CHECK-AU: ellipsis: features=3 stack_args=0

llvm/test/Instrumentation/SanitizerBinaryMetadata/covered.cpp

1 ↗

(On Diff #478569)

This is the new test.

[melver]

LGTM

llvm/include/llvm/CodeGen/MachinePassRegistry.def

205

I called the LLVM IR pass just "sanmd-module".
So this could just be "machine-sanmd".

llvm/test/Instrumentation/SanitizerBinaryMetadata/common.h

54 ↗

(On Diff #478569)

Could make this have its real signature, i.e. add version,start,end args. Then could also check in del that start,end matches, like above.

llvm/test/Instrumentation/SanitizerBinaryMetadata/covered.cpp

1 ↗

(On Diff #478569)

Nice!

[melver]

llvm/test/Instrumentation/SanitizerBinaryMetadata/uar.cpp

19 ↗

(On Diff #478569)

Is this comment out of place?

[dvyukov]

addressed comments

[dvyukov]

llvm/test/Instrumentation/SanitizerBinaryMetadata/uar.cpp

19 ↗

(On Diff #478569)

If a function is completely empty, it won't be marked with UAR feature metadata.
This is super-primitive approximation of the actual analysis we should do (the function has addr-taken/escaped locals/arguments).
So I needed at least somebody for the function.

[melver]

llvm/test/Instrumentation/SanitizerBinaryMetadata/uar.cpp

19 ↗

(On Diff #478569)

Ack

[melver]

llvm/lib/CodeGen/SanitizerBinaryMetadata.cpp

57 ↗

(On Diff #478615)

Probably should be getNumOperands()?

https://reviews.llvm.org/rGdbb11309665fdf7d411d25cc8f2e6a0c8f658143

-DLLVM_ENABLE_ASSERTIONS=On will catch it.

[dvyukov]

fixed the debug build

[dvyukov]

Kazu, thanks for taking care of the revert.

I've fixed the assert and tested with -DLLVM_ENABLE_ASSERTIONS=On.

[dvyukov]

FTR ./llvm/test/CodeGen/AMDGPU/llc-pipeline.ll also failed as it hardcodes all passes:
https://lab.llvm.org/buildbot/#/builders/123/builds/14397

[dvyukov]

In D136078#3959643, @dvyukov wrote:

FTR ./llvm/test/CodeGen/AMDGPU/llc-pipeline.ll also failed as it hardcodes all passes:
https://lab.llvm.org/buildbot/#/builders/123/builds/14397

Also these:

LLVM :: CodeGen/X86/O0-pipeline.ll
LLVM :: CodeGen/X86/opt-pipeline.ll

need to grep for something.

[dvyukov]

moved *.cpp tests to clang/test/
updated tests that hardcode pipeline passes
fixed metadata update

[dvyukov]

PTAL

moved *.cpp tests to clang/test/
updated tests that hardcode pipeline passes
fixed metadata update

This did not work, it also changed all shared instances:

const auto *NewMD = MDB.createPCSections(
    {{Section.getString(), {Features, IRB.getInt32(Size)}}});
MD->replaceOperandWith(1, NewMD->getOperand(1));

so I changed it to:

F.setMetadata(LLVMContext::MD_pcsections,
              MDB.createPCSections(
                  {{Section.getString(), {Features, IRB.getInt32(Size)}}}));

[melver]

AFAIK Windows bot will break: https://buildkite.com/llvm-project/premerge-checks/builds/124003#0184c872-af5b-47c0-938c-b31ee8808241

Need to restrict test to Linux only.

[dvyukov]

restrict tests to linux

[mysterymath]

Hi, it looks like this change is causing Fuchsia's linux-x64 clang builders to fail: https://luci-milo.appspot.com/ui/p/fuchsia/builders/toolchain.ci/clang-linux-x64/b8796062278266465473/overview

Would you mind taking a look and fixing forward, or that ends up being difficult, reverting until this can be fixed?

[aeubanks]

clang/test/Instrumentation/SanitizerBinaryMetadata/common.h

1–3 ↗

(On Diff #478922)

tests in clang/test cannot assume that standard headers are available, the tests should be hermetic. could you revert and reland with a proper test fix?

[wolfgangp]

covered.cpp and uar.cpp seem to fail on ubuntu bots here.

[dvyukov]

@melver Re this failure:

******************** TEST 'Clang :: Instrumentation/SanitizerBinaryMetadata/uar.cpp' FAILED ********************
Script:
--
: 'RUN: at line 4';   /home/buildbot/as-worker-91/clang-with-lto-ubuntu/build/stage1/bin/clang --driver-mode=g++ /home/buildbot/as-worker-91/clang-with-lto-ubuntu/llvm-project/clang/test/Instrumentation/SanitizerBinaryMetadata/uar.cpp -o /home/buildbot/as-worker-91/clang-with-lto-ubuntu/build/stage1/tools/clang/test/Instrumentation/SanitizerBinaryMetadata/Output/uar.cpp.tmp -fexperimental-sanitize-metadata=covered,uar && /home/buildbot/as-worker-91/clang-with-lto-ubuntu/build/stage1/tools/clang/test/Instrumentation/SanitizerBinaryMetadata/Output/uar.cpp.tmp | /home/buildbot/as-worker-91/clang-with-lto-ubuntu/build/stage1/bin/FileCheck /home/buildbot/as-worker-91/clang-with-lto-ubuntu/llvm-project/clang/test/Instrumentation/SanitizerBinaryMetadata/uar.cpp
--
Exit Code: 1
Command Output (stderr):
--
/usr/local/bin/ld: sanmd_covered has both ordered [`sanmd_covered[_Z7consumeIjET_RPKcS2_]' in /tmp/lit-tmp-2jisr97l/uar-d5d8d4.o] and unordered [`sanmd_covered[__dummy_sanmd_covered]' in /tmp/lit-tmp-2jisr97l/uar-d5d8d4.o] sections
/usr/local/bin/ld: final link failed: Bad value
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)

https://lab.llvm.org/buildbot/#/builders/124/builds/5759/steps/7/logs/stdio

This looks like a latent issue which is just exposed by the test.

As far as I understand this happens because this dummy variable somehow ends up in an "unordered" section, while other "real" metadata objects end up in "ordered" sections:

void SanitizerBinaryMetadata::createZeroSizedObjectInSection(
    Type *Ty, StringRef SectionSuffix) {
  auto *DummyInit = ConstantAggregateZero::get(ArrayType::get(Ty, 0));
  auto *DummyEntry = new GlobalVariable(Mod, DummyInit->getType(), true,
                                        GlobalVariable::ExternalLinkage,
                                        DummyInit, "__dummy_" + SectionSuffix);
  DummyEntry->setSection(getSectionName(SectionSuffix));
  DummyEntry->setVisibility(GlobalValue::HiddenVisibility);
  if (TargetTriple.supportsCOMDAT())
    DummyEntry->setComdat(Mod.getOrInsertComdat(DummyEntry->getName()));
  // Make sure the section isn't discarded by gc-sections.
  appendToUsed(Mod, DummyEntry);
}

I don't see any method on GlobalVariable to set the section to "ordered".

The only idea I have is to print dummy object in AsmPrinter the same way we print all other metadata objects (then I assume it will have the same properties as other objects).

Do you have any other ideas on how to dead with this?

[dvyukov]

Re this failure:

project/clang/test/Instrumentation/SanitizerBinaryMetadata/covered.cpp
--
Exit Code: 2

Command Output (stderr):
--
covered.cpp.tmp: /b/s/w/ir/x/w/llvm-llvm-project/clang/test/Instrumentation/SanitizerBinaryMetadata/common.h:22: T consume(const char *&, const char *) [T = unsigned int]: Assertion `pos <= end' failed.
FileCheck error: '<stdin>' is empty.
FileCheck command line:  /b/s/w/ir/x/w/staging/llvm_build/bin/FileCheck -check-prefix=CHECK-A /b/s/w/ir/x/w/llvm-llvm-project/clang/test/Instrumentation/SanitizerBinaryMetadata/covered.cpp

https://logs.chromium.org/logs/fuchsia/buildbucket/cr-buildbucket/8796062278266465473/+/u/clang/test/stdout

Not sure why it run on fuchsia at all and the strange failure mode.
But I hope this will go away when we move tests to compiler-rt.

[melver]

In D136078#3964100, @dvyukov wrote:

@melver Re this failure:

******************** TEST 'Clang :: Instrumentation/SanitizerBinaryMetadata/uar.cpp' FAILED ********************
Script:
--
: 'RUN: at line 4';   /home/buildbot/as-worker-91/clang-with-lto-ubuntu/build/stage1/bin/clang --driver-mode=g++ /home/buildbot/as-worker-91/clang-with-lto-ubuntu/llvm-project/clang/test/Instrumentation/SanitizerBinaryMetadata/uar.cpp -o /home/buildbot/as-worker-91/clang-with-lto-ubuntu/build/stage1/tools/clang/test/Instrumentation/SanitizerBinaryMetadata/Output/uar.cpp.tmp -fexperimental-sanitize-metadata=covered,uar && /home/buildbot/as-worker-91/clang-with-lto-ubuntu/build/stage1/tools/clang/test/Instrumentation/SanitizerBinaryMetadata/Output/uar.cpp.tmp | /home/buildbot/as-worker-91/clang-with-lto-ubuntu/build/stage1/bin/FileCheck /home/buildbot/as-worker-91/clang-with-lto-ubuntu/llvm-project/clang/test/Instrumentation/SanitizerBinaryMetadata/uar.cpp
--
Exit Code: 1
Command Output (stderr):
--
/usr/local/bin/ld: sanmd_covered has both ordered [`sanmd_covered[_Z7consumeIjET_RPKcS2_]' in /tmp/lit-tmp-2jisr97l/uar-d5d8d4.o] and unordered [`sanmd_covered[__dummy_sanmd_covered]' in /tmp/lit-tmp-2jisr97l/uar-d5d8d4.o] sections
/usr/local/bin/ld: final link failed: Bad value
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)

https://lab.llvm.org/buildbot/#/builders/124/builds/5759/steps/7/logs/stdio

This looks like a latent issue which is just exposed by the test.

As far as I understand this happens because this dummy variable somehow ends up in an "unordered" section, while other "real" metadata objects end up in "ordered" sections:

void SanitizerBinaryMetadata::createZeroSizedObjectInSection(
    Type *Ty, StringRef SectionSuffix) {
  auto *DummyInit = ConstantAggregateZero::get(ArrayType::get(Ty, 0));
  auto *DummyEntry = new GlobalVariable(Mod, DummyInit->getType(), true,
                                        GlobalVariable::ExternalLinkage,
                                        DummyInit, "__dummy_" + SectionSuffix);
  DummyEntry->setSection(getSectionName(SectionSuffix));
  DummyEntry->setVisibility(GlobalValue::HiddenVisibility);
  if (TargetTriple.supportsCOMDAT())
    DummyEntry->setComdat(Mod.getOrInsertComdat(DummyEntry->getName()));
  // Make sure the section isn't discarded by gc-sections.
  appendToUsed(Mod, DummyEntry);
}

I don't see any method on GlobalVariable to set the section to "ordered".

The only idea I have is to print dummy object in AsmPrinter the same way we print all other metadata objects (then I assume it will have the same properties as other objects).

Do you have any other ideas on how to dead with this?

It's strange we've not encountered this elsewhere - this must be some special (old?) linker. I'll investigate...

[melver]

Do you have any other ideas on how to dead with this?

It's strange we've not encountered this elsewhere - this must be some special (old?) linker. I'll investigate...

GNU ld <= 2.35 does not support mixed SHF_LINK_ORDER and non-SHF_LINK_ORDER sections.

This is about the relative order of the section w.r.t. other sections it references (we link it to the function section, see MCObjectFileInfo::getPCSection) when the linker merges the sections. From what I can find, we need to retain SHF_LINK_ORDER to support gc-sections properly.

This is a good summary: https://maskray.me/blog/2021-01-31-metadata-sections-comdat-and-shf-link-order
@MaskRay

I'm going to try and see how I can add the SHF_LINK_ORDER attribute to the section output for the dummy variable.

[melver]

In D136078#3966187, @melver wrote:

Do you have any other ideas on how to dead with this?

It's strange we've not encountered this elsewhere - this must be some special (old?) linker. I'll investigate...

GNU ld <= 2.35 does not support mixed SHF_LINK_ORDER and non-SHF_LINK_ORDER sections.

This is about the relative order of the section w.r.t. other sections it references (we link it to the function section, see MCObjectFileInfo::getPCSection) when the linker merges the sections. From what I can find, we need to retain SHF_LINK_ORDER to support gc-sections properly.

This is a good summary: https://maskray.me/blog/2021-01-31-metadata-sections-comdat-and-shf-link-order
@MaskRay

I'm going to try and see how I can add the SHF_LINK_ORDER attribute to the section output for the dummy variable.

This should fix it: https://reviews.llvm.org/D139224 - at least whatever I was able to reproduce with the linker version I found (2.31) seems to work.

[MaskRay]

SanitizerBinaryMetadata::createZeroSizedObjectInSection creates __dummy_*, but why is it needed? (There is no comment.)

[melver]

In D136078#3969061, @MaskRay wrote:

SanitizerBinaryMetadata::createZeroSizedObjectInSection creates __dummy_*, but why is it needed? (There is no comment.)

// Create a 0-sized object in a section, so that the section is not discarded
// if all inputs have been discarded.

We need the __begin and __end markers even if there ends up being no binary metadata.

[MaskRay]

In D136078#3969091, @melver wrote:

In D136078#3969061, @MaskRay wrote:

SanitizerBinaryMetadata::createZeroSizedObjectInSection creates __dummy_*, but why is it needed? (There is no comment.)

// Create a 0-sized object in a section, so that the section is not discarded
// if all inputs have been discarded.

We need the __begin and __end markers even if there ends up being no binary metadata.

Thanks! I saw the comment associated with createZeroSizedObjectInSection but wasn't so sure if it was only for section based garbage collection.
If that's the case, D139276 ExternalWeakLinkage would be a more size efficient fix.

[dvyukov]

Moved tests to compiler-rt and rebased to HEAD.

[dvyukov]

PTAL

[thakis]

This seems to break tests: http://45.33.8.238/linux/93224/step_12.txt

Can you take a look?

[dvyukov]

In D136078#3970792, @thakis wrote:

This seems to break tests: http://45.33.8.238/linux/93224/step_12.txt

Can you take a look?

Looking.

[dvyukov]

Fix for:

-- Supported architectures for crt: aarch64
CMake Error at compiler-rt/cmake/config-ix.cmake:244 (message):
  Unsupported architecture: x86_64
Call Stack (most recent call first):
  compiler-rt/cmake/config-ix.cmake:280 (get_target_flags_for_arch)
  compiler-rt/test/metadata/CMakeLists.txt:7 (get_test_cc_for_arch)
-- Configuring incomplete, errors occurred!

https://reviews.llvm.org/D139323

[dvyukov]

In D136078#3970792, @thakis wrote:

This seems to break tests: http://45.33.8.238/linux/93224/step_12.txt

Can you take a look?

Sent a fix: https://reviews.llvm.org/D139325

[dvyukov]

FTR 2 breakage fixes merged:
https://github.com/llvm/llvm-project/commit/2a05bd212e3e8aaed818ee23464f4d1fe0b0596d
https://github.com/llvm/llvm-project/commit/08742e72a34e835e6fc3c696eabe6045c78d6289

clang/include/clang/Basic/CodeGenOptions.h

#include "clang/Basic/CodeGenOptions.def"
   bool hasSanitizeCoverage() const {
     return SanitizeCoverageType || SanitizeCoverageIndirectCalls ||
            SanitizeCoverageTraceCmp || SanitizeCoverageTraceLoads ||
            SanitizeCoverageTraceStores || SanitizeCoverageControlFlow;
   }
 
   // Check if any one of SanitizeBinaryMetadata* is enabled.
   bool hasSanitizeBinaryMetadata() const {
-    return SanitizeBinaryMetadataCovered || SanitizeBinaryMetadataAtomics;
+    return SanitizeBinaryMetadataCovered || SanitizeBinaryMetadataAtomics ||
+           SanitizeBinaryMetadataUAR;
   }
 };
 
 }  // end namespace clang
 
 #endif

clang/include/clang/Basic/CodeGenOptions.def

 CODEGENOPT(SanitizeCoveragePCTable, 1, 0) ///< Create a PC Table.
 CODEGENOPT(SanitizeCoverageControlFlow, 1, 0) ///< Collect control flow
 CODEGENOPT(SanitizeCoverageNoPrune, 1, 0) ///< Disable coverage pruning.
 CODEGENOPT(SanitizeCoverageStackDepth, 1, 0) ///< Enable max stack depth tracing
 CODEGENOPT(SanitizeCoverageTraceLoads, 1, 0) ///< Enable tracing of loads.
 CODEGENOPT(SanitizeCoverageTraceStores, 1, 0) ///< Enable tracing of stores.
 CODEGENOPT(SanitizeBinaryMetadataCovered, 1, 0) ///< Emit PCs for covered functions.
 CODEGENOPT(SanitizeBinaryMetadataAtomics, 1, 0) ///< Emit PCs for atomic operations.
+CODEGENOPT(SanitizeBinaryMetadataUAR, 1, 0) ///< Emit PCs for start of functions
+                                            ///< that are subject for use-after-return checking.
 CODEGENOPT(SanitizeStats     , 1, 0) ///< Collect statistics for sanitizers.
 CODEGENOPT(SimplifyLibCalls  , 1, 1) ///< Set when -fbuiltin is enabled.
 CODEGENOPT(SoftFloat         , 1, 0) ///< -soft-float.
 CODEGENOPT(SpeculativeLoadHardening, 1, 0) ///< Enable speculative load hardening.
 CODEGENOPT(FineGrainedBitfieldAccesses, 1, 0) ///< Enable fine-grained bitfield accesses.
 CODEGENOPT(StrictEnums       , 1, 0) ///< Optimize based on strict enum definition.
 CODEGENOPT(StrictVTablePointers, 1, 0) ///< Optimize based on the strict vtable pointers
 CODEGENOPT(TimePasses        , 1, 0) ///< Set when -ftime-report or -ftime-report= is enabled.

clang/include/clang/Driver/Options.td

 def fexperimental_sanitize_metadata_EQ_covered
     : Flag<["-"], "fexperimental-sanitize-metadata=covered">,
       HelpText<"Emit PCs for code covered with binary analysis sanitizers">,
       MarshallingInfoFlag<CodeGenOpts<"SanitizeBinaryMetadataCovered">>;
 def fexperimental_sanitize_metadata_EQ_atomics
     : Flag<["-"], "fexperimental-sanitize-metadata=atomics">,
       HelpText<"Emit PCs for atomic operations used by binary analysis sanitizers">,
       MarshallingInfoFlag<CodeGenOpts<"SanitizeBinaryMetadataAtomics">>;
+def fexperimental_sanitize_metadata_EQ_uar
+    : Flag<["-"], "fexperimental-sanitize-metadata=uar">,
+      HelpText<"Emit PCs for start of functions that are subject for use-after-return checking.">,
+      MarshallingInfoFlag<CodeGenOpts<"SanitizeBinaryMetadataUAR">>;
 def fpatchable_function_entry_offset_EQ
     : Joined<["-"], "fpatchable-function-entry-offset=">, MetaVarName<"<M>">,
       HelpText<"Generate M NOPs before function entry">,
       MarshallingInfoInt<CodeGenOpts<"PatchableFunctionEntryOffset">>;
 def fprofile_instrument_EQ : Joined<["-"], "fprofile-instrument=">,
     HelpText<"Enable PGO instrumentation">, Values<"none,clang,llvm,csllvm">,
     NormalizedValuesScope<"CodeGenOptions">,
     NormalizedValues<["ProfileNone", "ProfileClangInstr", "ProfileIRInstr", "ProfileCSIRInstr"]>,

clang/lib/CodeGen/BackendUtil.cpp

getSancovOptsFromCGOpts(const CodeGenOptions &CGOpts) {
   return Opts;
 }
 
 static SanitizerBinaryMetadataOptions
 getSanitizerBinaryMetadataOptions(const CodeGenOptions &CGOpts) {
   SanitizerBinaryMetadataOptions Opts;
   Opts.Covered = CGOpts.SanitizeBinaryMetadataCovered;
   Opts.Atomics = CGOpts.SanitizeBinaryMetadataAtomics;
+  Opts.UAR = CGOpts.SanitizeBinaryMetadataUAR;
   return Opts;
 }
 
 // Check if ASan should use GC-friendly instrumentation for globals.
 // First of all, there is no point if -fdata-sections is off (expect for MachO,
 // where this is not a factor). Also, on ELF this feature requires an assembler
 // extension that only works with -integrated-as at the moment.
 static bool asanUseGlobalsGC(const Triple &T, const CodeGenOptions &CGOpts) {

clang/lib/Driver/SanitizerArgs.cpp

enum CoverageFeature {
   CoverageTraceLoads = 1 << 16,
   CoverageTraceStores = 1 << 17,
   CoverageControlFlow = 1 << 18,
 };
 
 enum BinaryMetadataFeature {
   BinaryMetadataCovered = 1 << 0,
   BinaryMetadataAtomics = 1 << 1,
+  BinaryMetadataUAR = 1 << 2,
 };
 
 /// Parse a -fsanitize= or -fno-sanitize= argument's values, diagnosing any
 /// invalid components. Returns a SanitizerMask.
 static SanitizerMask parseArgValues(const Driver &D, const llvm::opt::Arg *A,
                                     bool DiagnoseErrors);
 
 /// Parse -f(no-)?sanitize-coverage= flag values, diagnosing any invalid
addSpecialCaseListOpt(
       Args, CmdArgs, "-fsanitize-coverage-allowlist=", CoverageAllowlistFiles);
   addSpecialCaseListOpt(Args, CmdArgs, "-fsanitize-coverage-ignorelist=",
                         CoverageIgnorelistFiles);
 
   // Translate available BinaryMetadataFeatures to corresponding clang-cc1
   // flags. Does not depend on any other sanitizers.
   const std::pair<int, std::string> BinaryMetadataFlags[] = {
       std::make_pair(BinaryMetadataCovered, "covered"),
-      std::make_pair(BinaryMetadataAtomics, "atomics")};
+      std::make_pair(BinaryMetadataAtomics, "atomics"),
+      std::make_pair(BinaryMetadataUAR, "uar")};
   for (const auto &F : BinaryMetadataFlags) {
     if (BinaryMetadataFeatures & F.first)
       CmdArgs.push_back(
           Args.MakeArgString("-fexperimental-sanitize-metadata=" + F.second));
   }
 
   if (TC.getTriple().isOSWindows() && needsUbsanRt()) {
     // Instruct the code generator to embed linker directives in the object file
assert(
       A->getOption().matches(
           options::OPT_fno_experimental_sanitize_metadata_EQ));
   int Features = 0;
   for (int i = 0, n = A->getNumValues(); i != n; ++i) {
     const char *Value = A->getValue(i);
     int F = llvm::StringSwitch<int>(Value)
                 .Case("covered", BinaryMetadataCovered)
                 .Case("atomics", BinaryMetadataAtomics)
+                .Case("uar", BinaryMetadataUAR)
                 .Case("all", ~0)
                 .Default(0);
     if (F == 0 && DiagnoseErrors)
       D.Diag(clang::diag::err_drv_unsupported_option_argument)
           << A->getOption().getName() << Value;
     Features |= F;
   }
   return Features;

llvm/include/llvm/CodeGen/CodeGenPassBuilder.h

if (TM.Options.EnableIPRA)
     // Collect register usage information and produce a register mask of
     // clobbered registers, to be used to optimize call sites.
     addPass(RegUsageInfoCollectorPass());
 
   addPass(FuncletLayoutPass());
 
   addPass(StackMapLivenessPass());
   addPass(LiveDebugValuesPass());
+  addPass(MachineSanitizerMetadata());
 
   if (TM.Options.EnableMachineOutliner && getOptLevel() != CodeGenOpt::None &&
       Opt.EnableMachineOutliner != RunOutliner::NeverOutline) {
     bool RunOnAllFunctions =
         (Opt.EnableMachineOutliner == RunOutliner::AlwaysOutline);
     bool AddOutliner = RunOnAllFunctions || TM.Options.SupportsDefaultOutlining;
     if (AddOutliner)
       addPass(MachineOutlinerPass(RunOnAllFunctions));

llvm/include/llvm/CodeGen/MachinePassRegistry.def

 DUMMY_MACHINE_FUNCTION_PASS("ra-pbqp", RAPBQPPass, ())
 DUMMY_MACHINE_FUNCTION_PASS("legalizer", LegalizerPass, ())
 DUMMY_MACHINE_FUNCTION_PASS("irtranslator", IRTranslatorPass, ())
 DUMMY_MACHINE_FUNCTION_PASS("regbankselect", RegBankSelectPass, ())
 DUMMY_MACHINE_FUNCTION_PASS("instruction-select", InstructionSelectPass, ())
 DUMMY_MACHINE_FUNCTION_PASS("reset-machine-function", ResetMachineFunctionPass, ())
 DUMMY_MACHINE_FUNCTION_PASS("machineverifier", MachineVerifierPass, ())
 DUMMY_MACHINE_FUNCTION_PASS("print-machine-cycles", MachineCycleInfoPrinterPass, ())
+DUMMY_MACHINE_FUNCTION_PASS("machine-sanmd", MachineSanitizerMetadata, ())
 #undef DUMMY_MACHINE_FUNCTION_PASS

llvm/include/llvm/CodeGen/Passes.h

namespace llvm {
   /// bundles (created earlier, e.g. during pre-RA scheduling).
   extern char &FinalizeMachineBundlesID;
 
   /// StackMapLiveness - This pass analyses the register live-out set of
   /// stackmap/patchpoint intrinsics and attaches the calculated information to
   /// the intrinsic for later emission to the StackMap.
   extern char &StackMapLivenessID;
 
+  extern char &MachineSanitizerMetadataID;
+
   /// RemoveRedundantDebugValues pass.
   extern char &RemoveRedundantDebugValuesID;
 
   /// MachineCFGPrinter pass.
   extern char &MachineCFGPrinterID;
 
   /// LiveDebugValues pass
   extern char &LiveDebugValuesID;

llvm/include/llvm/InitializePasses.h

 void initializeMachineLICMPass(PassRegistry&);
 void initializeMachineLoopInfoPass(PassRegistry&);
 void initializeMachineModuleInfoWrapperPassPass(PassRegistry &);
 void initializeMachineOptimizationRemarkEmitterPassPass(PassRegistry&);
 void initializeMachineOutlinerPass(PassRegistry&);
 void initializeMachinePipelinerPass(PassRegistry&);
 void initializeMachinePostDominatorTreePass(PassRegistry&);
 void initializeMachineRegionInfoPassPass(PassRegistry&);
+void initializeMachineSanitizerMetadataPass(PassRegistry &);
 void initializeMachineSchedulerPass(PassRegistry&);
 void initializeMachineSinkingPass(PassRegistry&);
 void initializeMachineTraceMetricsPass(PassRegistry&);
 void initializeMachineVerifierPassPass(PassRegistry&);
 void initializeMemCpyOptLegacyPassPass(PassRegistry&);
 void initializeMemDepPrinterPass(PassRegistry&);
 void initializeMemDerefPrinterPass(PassRegistry&);
 void initializeMemoryDependenceWrapperPassPass(PassRegistry&);

llvm/include/llvm/Transforms/Instrumentation.h

struct SanitizerCoverageOptions {
 
   SanitizerCoverageOptions() = default;
 };
 
 /// Options for SanitizerBinaryMetadata.
 struct SanitizerBinaryMetadataOptions {
   bool Covered = false;
   bool Atomics = false;
+  bool UAR = false;
   SanitizerBinaryMetadataOptions() = default;
 };
 
+constexpr int kSanitizerMetadataAtomicsBit = 0;
+constexpr int kSanitizerMetadataUARBit = 1;
+
+constexpr uint32_t kSanitizerMetadataNone = 0;
+constexpr uint32_t kSanitizerMetadataAtomics = 1
+                                               << kSanitizerMetadataAtomicsBit;
+constexpr uint32_t kSanitizerMetadataUAR = 1 << kSanitizerMetadataUARBit;
+
 /// Calculate what to divide by to scale counts.
 ///
 /// Given the maximum count, calculate a divisor that will scale all the
 /// weights to strictly less than std::numeric_limits<uint32_t>::max().
 static inline uint64_t calculateCountScale(uint64_t MaxCount) {
   return MaxCount < std::numeric_limits<uint32_t>::max()
              ? 1
              : MaxCount / std::numeric_limits<uint32_t>::max() + 1;

llvm/lib/CodeGen/CMakeLists.txt

add_llvm_component_library(LLVMCodeGen
   RegUsageInfoCollector.cpp
   RegUsageInfoPropagate.cpp
   ReplaceWithVeclib.cpp
   ResetMachineFunctionPass.cpp
   RegisterBank.cpp
   RegisterBankInfo.cpp
   SafeStack.cpp
   SafeStackLayout.cpp
+  SanitizerMetadata.cpp
   ScheduleDAG.cpp
   ScheduleDAGInstrs.cpp
   ScheduleDAGPrinter.cpp
   ScoreboardHazardRecognizer.cpp
   SelectOptimize.cpp
   ShadowStackGCLowering.cpp
   ShrinkWrap.cpp
   SjLjEHPrepare.cpp

llvm/lib/CodeGen/CodeGen.cpp

void llvm::initializeCodeGen(PassRegistry &Registry) {
   initializeMachineDominatorTreePass(Registry);
   initializeMachineFunctionPrinterPassPass(Registry);
   initializeMachineLICMPass(Registry);
   initializeMachineLoopInfoPass(Registry);
   initializeMachineModuleInfoWrapperPassPass(Registry);
   initializeMachineOptimizationRemarkEmitterPassPass(Registry);
   initializeMachineOutlinerPass(Registry);
   initializeMachinePipelinerPass(Registry);
+  initializeMachineSanitizerMetadataPass(Registry);
   initializeModuloScheduleTestPass(Registry);
   initializeMachinePostDominatorTreePass(Registry);
   initializeMachineRegionInfoPassPass(Registry);
   initializeMachineSchedulerPass(Registry);
   initializeMachineSinkingPass(Registry);
   initializeMachineVerifierPassPass(Registry);
   initializeOptimizePHIsPass(Registry);
   initializePEIPass(Registry);

llvm/lib/CodeGen/SanitizerMetadata.cpp

+//===- SanitizerMetadata.cpp ----------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file is a part of SanitizerBinaryMetadata.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/CodeGen/MachineFrameInfo.h"
+#include "llvm/CodeGen/MachineFunction.h"
+#include "llvm/CodeGen/MachineFunctionPass.h"
+#include "llvm/CodeGen/Passes.h"
+#include "llvm/IR/IRBuilder.h"
+#include "llvm/IR/MDBuilder.h"
+#include "llvm/InitializePasses.h"
+#include "llvm/Pass.h"
+#include "llvm/Transforms/Instrumentation.h"
+#include <algorithm>
+
+using namespace llvm;
+
+class MachineSanitizerMetadata : public MachineFunctionPass {
+public:
+  static char ID;
+
+  MachineSanitizerMetadata();
+  bool runOnMachineFunction(MachineFunction &F) override;
+};
+
+INITIALIZE_PASS(MachineSanitizerMetadata, "machine-sanmd",
+                "Machine Sanitizer Binary Metadata", false, false)
+
+char MachineSanitizerMetadata::ID = 0;
+char &llvm::MachineSanitizerMetadataID = MachineSanitizerMetadata::ID;
+
+MachineSanitizerMetadata::MachineSanitizerMetadata() : MachineFunctionPass(ID) {
+  initializeMachineSanitizerMetadataPass(*PassRegistry::getPassRegistry());
+}
+
+bool MachineSanitizerMetadata::runOnMachineFunction(MachineFunction &MF) {
+  MDNode *MD = MF.getFunction().getMetadata(LLVMContext::MD_pcsections);
+  if (!MD)
+    return false;
+  auto &AuxMDs = *cast<MDTuple>(MD->getOperand(1));
+  const auto &Features = cast<ConstantAsMetadata>(AuxMDs.getOperand(0))
+                             ->getValue()
+                             ->getUniqueInteger();
+  if (!Features[kSanitizerMetadataUARBit])
+    return false;
+  // Append stack args size to the function metadata.
+  int64_t Size = 0;
+  uint64_t Align = 0;
+  const MachineFrameInfo &MFI = MF.getFrameInfo();
+  for (int i = -1; i >= (int)-MFI.getNumFixedObjects(); --i) {
+    Size = std::max(Size, MFI.getObjectOffset(i) + MFI.getObjectSize(i));
+    Align = std::max(Align, MFI.getObjectAlign(i).value());
+  }
+  Size = (Size + Align - 1) & ~(Align - 1);
+  auto &F = MF.getFunction();
+  IRBuilder<> IRB(F.getContext());
+  MDBuilder MDB(F.getContext());
+  AuxMDs.push_back(MDB.createConstant(IRB.getInt32(Size)));
+  return false;
+}

llvm/lib/CodeGen/TargetPassConfig.cpp

if (TM->Options.EnableIPRA)
     addPass(createRegUsageInfoCollector());
 
   // FIXME: Some backends are incompatible with running the verifier after
   // addPreEmitPass.  Maybe only pass "false" here for those targets?
   addPass(&FuncletLayoutID);
 
   addPass(&StackMapLivenessID);
   addPass(&LiveDebugValuesID);
+  addPass(&MachineSanitizerMetadataID);
 
   if (TM->Options.EnableMachineOutliner && getOptLevel() != CodeGenOpt::None &&
       EnableMachineOutliner != RunOutliner::NeverOutline) {
     bool RunOnAllFunctions =
         (EnableMachineOutliner == RunOutliner::AlwaysOutline);
     bool AddOutliner =
         RunOnAllFunctions || TM->Options.SupportsDefaultOutlining;
     if (AddOutliner)

llvm/lib/IR/MDBuilder.cpp

for (const auto &Entry : Sections) {
 
     // If auxiliary data for this section exists, append it.
     const SmallVector<Constant *> &AuxConsts = Entry.second;
     if (!AuxConsts.empty()) {
       SmallVector<Metadata *, 1> AuxMDs;
       AuxMDs.reserve(AuxConsts.size());
       for (Constant *C : AuxConsts)
         AuxMDs.push_back(createConstant(C));
-      Ops.push_back(MDNode::get(Context, AuxMDs));
+      // MachineSanitizerMetadata pass may modify it later.
+      Ops.push_back(MDNode::getDistinct(Context, AuxMDs));
     }
   }
 
   return MDNode::get(Context, Ops);
 }
 
 MDNode *MDBuilder::createAnonymousAARoot(StringRef Name, MDNode *Extra) {
   SmallVector<Metadata *, 3> Args(1, nullptr);

llvm/lib/Transforms/Instrumentation/SanitizerBinaryMetadata.cpp

public:
   const uint32_t FeatureMask;
 
   static const MetadataInfo Covered;
   static const MetadataInfo Atomics;
 
 private:
   // Forbid construction elsewhere.
   explicit constexpr MetadataInfo(StringRef FunctionPrefix,
-                                  StringRef SectionSuffix, int Feature)
+                                  StringRef SectionSuffix, uint32_t Feature)
       : FunctionPrefix(FunctionPrefix), SectionSuffix(SectionSuffix),
-        FeatureMask(Feature != -1 ? (1u << Feature) : 0) {}
+        FeatureMask(Feature) {}
 };
-const MetadataInfo MetadataInfo::Covered{"__sanitizer_metadata_covered",
-                                         "sanmd_covered", -1};
-const MetadataInfo MetadataInfo::Atomics{"__sanitizer_metadata_atomics",
-                                         "sanmd_atomics", 0};
+const MetadataInfo MetadataInfo::Covered{
+    "__sanitizer_metadata_covered", "sanmd_covered", kSanitizerMetadataNone};
+const MetadataInfo MetadataInfo::Atomics{
+    "__sanitizer_metadata_atomics", "sanmd_atomics", kSanitizerMetadataAtomics};
 
 // The only instances of MetadataInfo are the constants above, so a set of
 // them may simply store pointers to them. To deterministically generate code,
 // we need to use a set with stable iteration order, such as SetVector.
 using MetadataInfoSet = SetVector<const MetadataInfo *>;
 
 //===--- Command-line options ---------------------------------------------===//
 
 cl::opt<bool> ClEmitCovered("sanitizer-metadata-covered",
                             cl::desc("Emit PCs for covered functions."),
                             cl::Hidden, cl::init(false));
 cl::opt<bool> ClEmitAtomics("sanitizer-metadata-atomics",
                             cl::desc("Emit PCs for atomic operations."),
                             cl::Hidden, cl::init(false));
+cl::opt<bool> ClEmitUAR("sanitizer-metadata-uar",
+                        cl::desc("Emit PCs for start of functions that are "
+                                 "subject for use-after-return checking"),
+                        cl::Hidden, cl::init(false));
 
 //===--- Statistics -------------------------------------------------------===//
 
 STATISTIC(NumMetadataCovered, "Metadata attached to covered functions");
 STATISTIC(NumMetadataAtomics, "Metadata attached to atomics");
+STATISTIC(NumMetadataUAR, "Metadata attached to UAR functions");
 
 //===----------------------------------------------------------------------===//
 
 // Apply opt overrides.
 SanitizerBinaryMetadataOptions &&
 transformOptionsFromCl(SanitizerBinaryMetadataOptions &&Opts) {
   Opts.Covered |= ClEmitCovered;
   Opts.Atomics |= ClEmitAtomics;
+  Opts.UAR |= ClEmitUAR;
   return std::move(Opts);
 }
 
 class SanitizerBinaryMetadata {
 public:
   SanitizerBinaryMetadata(Module &M, SanitizerBinaryMetadataOptions Opts)
       : Mod(M), Options(transformOptionsFromCl(std::move(Opts))),
         TargetTriple(M.getTargetTriple()), IRB(M.getContext()) {
private:
 
   // Determines which set of metadata to collect for this instruction.
   //
   // Returns true if covered metadata is required to unambiguously interpret
   // other metadata. For example, if we are interested in atomics metadata, any
   // function with memory operations (atomic or not) requires covered metadata
   // to determine if a memory operation is atomic or not in modules compiled
   // with SanitizerBinaryMetadata.
-  bool runOn(Instruction &I, MetadataInfoSet &MIS, MDBuilder &MDB);
+  bool runOn(Instruction &I, MetadataInfoSet &MIS, MDBuilder &MDB,
+             uint32_t &PerInstrFeatureMask);
 
   // Get start/end section marker pointer.
   GlobalVariable *getSectionMarker(const Twine &MarkerName, Type *Ty);
 
   // Create a 0-sized object in a section, so that the section is not discarded
   // if all inputs have been discarded.
   void createZeroSizedObjectInSection(Type *Ty, StringRef SectionSuffix);
 
void SanitizerBinaryMetadata::runOn(Function &F, MetadataInfoSet &MIS) {
   // Don't touch available_externally functions, their actual body is elsewhere.
   if (F.getLinkage() == GlobalValue::AvailableExternallyLinkage)
     return;
 
   MDBuilder MDB(F.getContext());
 
   // The metadata features enabled for this function, stored along covered
   // metadata (if enabled).
   uint32_t PerInstrFeatureMask = getEnabledPerInstructionFeature();
   // Don't emit unnecessary covered metadata for all functions to save space.
   bool RequiresCovered = false;
-  if (PerInstrFeatureMask) {
+  if (PerInstrFeatureMask || Options.UAR) {
     for (BasicBlock &BB : F)
       for (Instruction &I : BB)
-        RequiresCovered |= runOn(I, MIS, MDB);
+        RequiresCovered |= runOn(I, MIS, MDB, PerInstrFeatureMask);
   }
 
+  if (F.isVarArg())
+    PerInstrFeatureMask &= ~kSanitizerMetadataUAR;
+  if (PerInstrFeatureMask & kSanitizerMetadataUAR)
+    NumMetadataUAR++;
+
   // Covered metadata is always emitted if explicitly requested, otherwise only
   // if some other metadata requires it to unambiguously interpret it for
   // modules compiled with SanitizerBinaryMetadata.
-  if (Options.Covered || RequiresCovered) {
+  if (Options.Covered || (PerInstrFeatureMask && RequiresCovered)) {
     NumMetadataCovered++;
     const auto *MI = &MetadataInfo::Covered;
     MIS.insert(MI);
     const StringRef Section = getSectionName(MI->SectionSuffix);
     // The feature mask will be placed after the size (32 bit) of the function,
     // so in total one covered entry will use `sizeof(void*) + 4 + 4`.
     Constant *CFM = IRB.getInt32(PerInstrFeatureMask);
     F.setMetadata(LLVMContext::MD_pcsections,
                   MDB.createPCSections({{Section, {CFM}}}));
   }
 }
 
 bool SanitizerBinaryMetadata::runOn(Instruction &I, MetadataInfoSet &MIS,
-                                    MDBuilder &MDB) {
+                                    MDBuilder &MDB,
+                                    uint32_t &PerInstrFeatureMask) {
   SmallVector<const MetadataInfo *, 1> InstMetadata;
   bool RequiresCovered = false;
 
+  if (Options.UAR) {
+    for (unsigned i = 0; i < I.getNumOperands(); ++i) {
+      const Value *V = I.getOperand(i);
+      // TODO(dvyukov): check if V is an address of alloca/function arg.
+      // See isSafeAndProfitableToSinkLoad for addr-taken allocas
+      // and DeadArgumentEliminationPass::removeDeadStuffFromFunction
+      // for iteration over function args.
+      if (V) {
+        RequiresCovered = true;
+        PerInstrFeatureMask |= kSanitizerMetadataUAR;
+      }
+    }
+  }
+
   if (Options.Atomics && I.mayReadOrWriteMemory()) {
     auto SSID = getAtomicSyncScopeID(&I);
     if (SSID.has_value() && SSID.value() != SyncScope::SingleThread) {
       NumMetadataAtomics++;
       InstMetadata.push_back(&MetadataInfo::Atomics);
     }
     RequiresCovered = true;
   }