This is an archive of the discontinued LLVM Phabricator instance.

[Driver] Fix ToolChain::getSanitizerArgs
ClosedPublic

Authored by yaxunl on Oct 8 2021, 10:33 AM.

Details

Summary

The driver uses class SanitizerArgs to store parsed sanitizer arguments. It keeps a cached
SanitizerArgs object in ToolChain and uses it for different jobs. This does not work if
the sanitizer options are different for different jobs, which could happen when an
offloading toolchain translates the options for different jobs.

To fix this, SanitizerArgs should be created by using the actual arguments passed
to jobs instead of the original arguments passed to the driver, since the toolchain
may change the original arguments. And the sanitizer arguments should be diagnose
once.

This patch also fixes HIP toolchain for handling -fgpu-sanitize: a warning is emitted
for GPU's not supporting sanitizer and skipped. This is for backward compatibility
with existing -fsanitize options. -fgpu-sanitize is also turned on by default.

Diff Detail

Event Timeline

yaxunl created this revision.Oct 8 2021, 10:33 AM
yaxunl requested review of this revision.Oct 8 2021, 10:33 AM
tra added a comment.Oct 8 2021, 11:19 AM

I'm curious why we need the cache at all. While the construction of sanitizer args is hairy, it's only called few times from the driver and will be lost in the noise compared to everything else.

clang/include/clang/Driver/ToolChain.h
165

Using argument list as a proxy for a job is somewhat questionable. ArgList may not necessarily be related to a particular job at all.

Can we use JobAction as the key instead and default to full reconstruction of sanitizer args where it's not available? Or just always construct them without caching?

I'm curious why we need the cache at all. While the construction of sanitizer args is hairy, it's only called few times from the driver and will be lost in the noise compared to everything else.

It is for avoiding repeated diagnostics. For example, -fsanitize=xxx is passed to clang and ld. If it is parsed twice, the diagnostics are emitted twice. There are lit tests to prevent the repeated diagnostics.

For this reason, using job action as cache key will not work since it will not prevent repeated diagnostics. For most non-offloading toolchain, ld and clang job action will not create new ArgList, therefore they share the same ArgList and avoid repeated parsing. The ArgList persists for the whole life span of driver, therefore no life time issue.

tra added a reviewer: eugenis.Oct 8 2021, 3:37 PM
tra added a subscriber: eugenis.

I'm curious why we need the cache at all. While the construction of sanitizer args is hairy, it's only called few times from the driver and will be lost in the noise compared to everything else.

It is for avoiding repeated diagnostics. For example, -fsanitize=xxx is passed to clang and ld. If it is parsed twice, the diagnostics are emitted twice. There are lit tests to prevent the repeated diagnostics.
For this reason, using job action as cache key will not work since it will not prevent repeated diagnostics.

Fair enough. Perhaps that's what we need to refactor first. We could separate processing of the arguments from diagnosing issues there. I.e. something like getSanitizerArgs(const llvm::opt::ArgList &JobArgs, bool diag = false) and call it once with diag=true. Or just make diag a static local var and just enable diags on the first call.

For most non-offloading toolchain, ld and clang job action will not create new ArgList, therefore they share the same ArgList and avoid repeated parsing. The ArgList persists for the whole life span of driver, therefore no life time issue.

It may work now, but to me it looks like a dependence on an implementation detail. I do not think it's reasonable to assume that ArgList pointer is immutable and that it coexists with particular job.

@eugenis -- WDYT?

Don't we want to diagnose the problems in the job's command line? What kind of changes can the driver do there that would affect SanitizerArgs?

I wonder if diagnostics should still be performed on the job args, but presented to the user differently, mainly because we can no longer refer to the user-provided command line?

I'm curious why we need the cache at all. While the construction of sanitizer args is hairy, it's only called few times from the driver and will be lost in the noise compared to everything else.

It is for avoiding repeated diagnostics. For example, -fsanitize=xxx is passed to clang and ld. If it is parsed twice, the diagnostics are emitted twice. There are lit tests to prevent the repeated diagnostics.
For this reason, using job action as cache key will not work since it will not prevent repeated diagnostics.

Fair enough. Perhaps that's what we need to refactor first. We could separate processing of the arguments from diagnosing issues there. I.e. something like getSanitizerArgs(const llvm::opt::ArgList &JobArgs, bool diag = false) and call it once with diag=true. Or just make diag a static local var and just enable diags on the first call.

For most non-offloading toolchain, ld and clang job action will not create new ArgList, therefore they share the same ArgList and avoid repeated parsing. The ArgList persists for the whole life span of driver, therefore no life time issue.

It may work now, but to me it looks like a dependence on an implementation detail. I do not think it's reasonable to assume that ArgList pointer is immutable and that it coexists with particular job.

@eugenis -- WDYT?

Don't we want to diagnose the problems in the job's command line? What kind of changes can the driver do there that would affect SanitizerArgs?

I wonder if diagnostics should still be performed on the job args, but presented to the user differently, mainly because we can no longer refer to the user-provided command line?

For offloading toolchain, the driver could remove -fsanitize for certain devices if they do not support them. Consider the following use case:

-fsanitize=address is passed by the user to clang driver, clang driver constructs 4 jobs by passing translated arguments to the tools:

  1. clang for host: -fsanitize=address is passed to clang tool
  1. ld for host: -fsantize=address is passed to host linker tool
  1. clang for device 1: -fsanitize=address is not passed to clang tool
  1. clang for device 2: -fsanitize=address is passed to clang tool

This demonstrates the necessity to parse the sanitizer arguments per job instead of per toolchain, since the same toolchain may parse different sanitizer arguments for different jobs.

If we need not avoid repeated diagnostics, we just need to parse the arguments for each job.

However, if we passed an invalid sanitizer argument to jobs 1, 2, and 4, we will get 3 identical diagnostics. This is annoying. That is why there are lit tests to prevent that.

Basically we need a way to know if the same argument list has been diagnosed. The simplest way is to check the pointer to the argument list since toolchain will reuse the original argument list if it does not change it. In this case, jobs 1, 2 and 4 share the same pointer to argument list, therefore they will be diagnosed only once. Since they are the same, they can be cached by pointer to the argument list.

This is also why separating diagnosing and parsing of sanitizer args does not help avoid repeated diagnostics for different jobs, since different jobs may have different sanitizer args. Then you have to have a map IsDiagnosed[JobArg] to track whether a job arg has been diagnosed. However, if you have that, then it is not too much different than having a cache for SanitizerArgs.

Right, but a cache for SanitizerArgs is not enough to avoid repeated diagnostics, is it? Ex. if I request a non-existing sanitizer, I think I would get errors from host arg parsing, as well as from each of device1 and device2, because each device will have a unique ArgList.

Is it even possible for the driver to introduce new diagnostics in offload SanitizerArgs parsing? Is it possible to catch those cases ahead of time, when parsing SanitizerArgs for the first time, by looking at a target triple or something? That would be the most user friendly approach.

Right, but a cache for SanitizerArgs is not enough to avoid repeated diagnostics, is it? Ex. if I request a non-existing sanitizer, I think I would get errors from host arg parsing, as well as from each of device1 and device2, because each device will have a unique ArgList.

Is it even possible for the driver to introduce new diagnostics in offload SanitizerArgs parsing? Is it possible to catch those cases ahead of time, when parsing SanitizerArgs for the first time, by looking at a target triple or something? That would be the most user friendly approach.

I think it makes sense to assume offloading toolchain will not introduce extra diagnostics inside SanitizerArgs than the default toolchain, therefore it should be OK to let the default toolchain check sanitizer args for once. I will try make that change.

yaxunl updated this revision to Diff 379806.Oct 14 2021, 12:53 PM
yaxunl edited the summary of this revision. (Show Details)

diagnose sanitizer args only once.

yaxunl marked an inline comment as done.Oct 20 2021, 7:53 AM

ping

yaxunl added a comment.Nov 1 2021, 7:29 AM

ping. I have made changes so that the diagnostics about sanitizer args are only emitted once. Any further changes or concerns? Thanks.

The approach looks reasonable to me.

yaxunl added a comment.Nov 8 2021, 8:12 AM

@tra Any further concerns? Thanks.

tra added a comment.Nov 8 2021, 10:46 AM

I'll defer to @eugenis. Overall it looks OK to be.

@eugenis Any further changes needed? Thanks.

eugenis added inline comments.Nov 10 2021, 1:58 PM
clang/lib/Driver/ToolChain.cpp
124

SanitizerArgs SanArgs(*this, JobArgs, !SanitizerArgsChecked);
SanitizerArgsChecked = true;
return SanArgs;

clang/lib/Driver/ToolChains/FreeBSD.cpp
471

it looks like we do a lot of sanitizerargs recomputing. Is it worth it to try and cache it?

yaxunl updated this revision to Diff 386518.Nov 11 2021, 7:55 AM
yaxunl marked an inline comment as done.

Revised by Evgenii's comments

yaxunl marked an inline comment as done.Nov 11 2021, 7:59 AM
yaxunl added inline comments.
clang/lib/Driver/ToolChain.cpp
124

done

clang/lib/Driver/ToolChains/FreeBSD.cpp
471

In general, SanitizerArgs is per job arguments, not per toolchain. There is no good way to cache it.

On the other hand, I would expect time spent in argument parsing is trivial compared to time spent in compilation for common programs.

eugenis accepted this revision.Nov 11 2021, 1:02 PM

LGTM

This revision is now accepted and ready to land.Nov 11 2021, 1:02 PM
This revision was landed with ongoing or failed builds.Nov 11 2021, 2:17 PM
This revision was automatically updated to reflect the committed changes.
yaxunl marked an inline comment as done.
Herald added a project: Restricted Project. · View Herald TranscriptNov 11 2021, 2:17 PM