[clang] Add the flag -ffile-reproducible

[ayzhao]

Summary

When Clang generates the path prefix (i.e. the path of the directory
where the file is) when generating FILE, __builtin_FILE(), and
std::source_location, Clang uses the platform-specific path separator
character of the build environment where Clang _itself_ is built. This
leads to inconsistencies in Chrome builds where Clang running on
non-Windows environments uses the forward slash (/) path separator
while Clang running on Windows builds uses the backslash (\) path
separator. To fix this, we add a flag -ffile-reproducible (and its
inverse, -fno-file-reproducible) to have Clang use the target's
platform-specific file separator character.

Additionally, the existing flags -fmacro-prefix-map and
-ffile-prefix-map now both imply -ffile-reproducible. This can be
overriden by setting -fno-file-reproducible.

[0]: https://crbug.com/1310767

[ayzhao]

Remove errant diff

[thakis]

As said elsewhere, I think this is too simplistic.

It's good to make this dependent on the triple, instead of on the host platform. But we should have a flag that controls which slash direction to use on windows triples, since different people will want different things.

And since most people who use clang-cl run it on Windows, the default for that flag should imho stay a backslash (but projects can add the forward direction flag if they want).

[thakis]

(I don't think the argument about '\' in include lines applies to this patch (?))

[hans]

we should have a flag that controls which slash direction to use on windows triples, since different people will want different things.
And since most people who use clang-cl run it on Windows, the default for that flag should imho stay a backslash (but projects can add the forward direction flag if they want).

I guess this is the part that's not entirely clear to me. Do people really *want* the backslash? Would anything break if we just use the forward slash?

This is MSVC's current behaviour:

C:\src\tmp>type a.cc foo\a.h

a.cc


#include "foo/a.h"

int main() { f(); return 0; }

foo\a.h


#include <stdio.h>

inline void f() { printf("%s\n", __FILE__); }

C:\src\tmp>cl /nologo a.cc && a.exe
a.cc
C:\src\tmp\foo/a.h

I think

C:\src\tmp\foo/a.h

looks pretty weird, and canonicalizing on slashes would be an improvement even if it diverges from MSVC.

(I don't think the argument about '\' in include lines applies to this patch (?))

It kinda does in that developers, even on Windows, already use forward slashes to refer to included files:

#include "foo/bar.h"

and so having the compiler prefer forward slashes for the whole filename is in line with that.

While this would make clang-cl diverge from MSVC, maybe this is one of those cases where we want to diverge because clang-cl's way is better. Unless it breaks something.

clang/lib/AST/Expr.cpp

2193–2194

I was going to say perhaps we should put a comment about why Windows needs different treatment, but as we'd need to put the comment in three different places, maybe this (remapPathPrefix() and make_preferred()) should be factored out to a utility function?

[hans]

Maybe Martin or Aaron have opinions here too?

[ayzhao]

clang/lib/AST/Expr.cpp

2193–2194

Extracting these lines into a separate function (say, for example, handle_file_macro(...)?) makes sense. However, I'm not sure which file this function would belong:

• The function would need to accept a LangOpts and a TargetInfo object. Both of these classes are Clang specific.
• It shouldn't belong in llvm::sys::path::Style because the file declaring that namespace exists in the llvm directory, which shouldn't depend on things in Clang.
• We could put it in LangOpts, but TargetInfo.h includes LangOpts.h, so this would create a circular dependency.
• TargetInfo doesn't make much sense as it doesn't have any code generation / language specific logic.

WDYT?

[mstorsjo]

When targeting Windows, the path separator used when targeting Windows depends on the build environment when Clang _itself_ is built. This leads to inconsistencies in Chrome builds where Clang running on non-Windows environments uses the forward slash (/) path separator while Clang running on Windows builds uses the backslash (\) path separator. To fix this, we make Clang use forward slashes whenever it is targeting Windows.

IMO, this problem formulation is a bit too narrow/specific - you'd have the same phenomenon if you'd cross compile things on Windows, targeting e.g. Linux. So I think the condition for normalizing paths would be if the build host OS is Windows, not based on the target OS. (I.e. in the current patch form, the normalization that is chosen, when targeting Windows from Unix, is a no-op anyway.)

Then secondly - if the source paths are relative paths, making them OS-agnostic in this way does make sense. But if they are absolute, they are pretty much by definition specific to the build host, and can't be expected to make sense on a different platform. So for such a case, there's less reason to try to normalize the paths. But as Windows path handling in most cases does tolerate paths with forward slashes, I guess it's ok to do anyway.

Overall, I think this is a sensible thing to do. I'm not entirely sure if an option for the behaviour is needed or not though - I guess that's the main question.

[thakis]

Do you know what cl.exe does to paths in pdb files? Does it write mixed-slashiness for foo/bar.h includes, or does it write backslashes throughout?

Windows can handle slashes, but several tools can't. I worry that if we do something different than cl, some random corner case might break (dbghelp, or some source server thing or some custom debug info processor somewhere).

Then secondly - if the source paths are relative paths, making them OS-agnostic in this way does make sense. But if they are absolute, they are pretty much by definition specific to the build host,

We use /pdbsourcepath:X:\fake\prefix to write a deterministic absolute path to the output file at link time (and -fdebug-compilation-dir . to get deterministic compiler output – see https://blog.llvm.org/2019/11/deterministic-builds-with-clang-and-lld.html). The motivation is to get exactly the same output when building on linux and windows hosts.

[mstorsjo]

In D122766#3420925, @thakis wrote:

Windows can handle slashes, but several tools can't. I worry that if we do something different than cl, some random corner case might break (dbghelp, or some source server thing or some custom debug info processor somewhere).

Yup, that's a valid concern. Especially PDB handling is pretty particular about requiring backslashes, as far as I've seen.

Then secondly - if the source paths are relative paths, making them OS-agnostic in this way does make sense. But if they are absolute, they are pretty much by definition specific to the build host,

We use /pdbsourcepath:X:\fake\prefix to write a deterministic absolute path to the output file at link time (and -fdebug-compilation-dir . to get deterministic compiler output – see https://blog.llvm.org/2019/11/deterministic-builds-with-clang-and-lld.html). The motivation is to get exactly the same output when building on linux and windows hosts.

Ok, nice.

Do you agree that if we go down this path doing this, we should also do it whenever the host OS is windows, i.e. also for windows->linux cross compilation, for consistency?

[thakis]

Do you agree that if we go down this path doing this, we should also do it whenever the host OS is windows, i.e. also for windows->linux cross compilation, for consistency?

Yes, to me it makes sense that this should depend on the target triple, not on the host system (ie always use slashes when targeting non-Windows, and jury still a bit out on what exactly to do when targeting Windows).

[hans]

In D122766#3420925, @thakis wrote:

Do you know what cl.exe does to paths in pdb files? Does it write mixed-slashiness for foo/bar.h includes, or does it write backslashes throughout?

Looks like just backslashes:

cl /nologo /Zi a.cc && a.exe
llvm-pdbutil.exe dump --files \src\tmp\a.pdb | grep foo
- (MD5: B97B3B57F55C87D06A0379A4FBCA51F0) C:\src\tmp\foo\a.h

(To make sure llvm-pdbutil didn't canonicalize, I looked at the pdb in vim and the backslashes are there.)

It appears that lld does the same.

Windows can handle slashes, but several tools can't. I worry that if we do something different than cl, some random corner case might break (dbghelp, or some source server thing or some custom debug info processor somewhere).

Since cl is already mixing back and forward slashes, any tools consuming FILE can presumably handle both. So diverging msvc by canonicalizing here doesn't seem that risky, we just have to decide in which direction.

clang/lib/AST/Expr.cpp

2193–2194

It would have to be somewhere in Clang.
Maybe it could be a method in Preprocessor.

[ayzhao]

Extract logic into a separate function

[aaron.ballman]

In D122766#3419303, @hans wrote:

Maybe Martin or Aaron have opinions here too?

Naively, this seems wrong to me. Yes, Windows sometimes allows you to use forward slashes, but that is not the path separator which Windows users are most often presented with in terms of documentation, screen shots, presentation in Explorer, etc. My feeling is that the default behavior on Windows needs to be to use backslashes and not forward slashes. Having an option to canonicalize in the other direction would be reasonable, as does differing what the default is based on the target.

[hans]

My feeling is that the default behavior on Windows needs to be to use backslashes and not forward slashes.

Okay, how would folks feel about always canonicalizing __FILE__ etc. to use backslashes when targeting Windows?

[dexonsmith]

In D122766#3429552, @hans wrote:

My feeling is that the default behavior on Windows needs to be to use backslashes and not forward slashes.

Okay, how would folks feel about always canonicalizing __FILE__ etc. to use backslashes when targeting Windows?

FWIW (I have little Windows experience and no stake in this!), defaulting to backslashes more sense to me. Also seems easy enough to add a preprocessor option to configure this.

I'm curious though, if you're building on POSIX (so the file starts out as POSIX) and targeting Windows, does calling make_preferred() actually give you something reasonable in the general case? Or should it be used *only* when there's a hit in LangOpts.remapPathPrefix()? Sorry if this has been discussed and rejected; I haven't followed the full thread; feel free to ignore if it's not relevant; but it seems to me that when -ffile-prefix-map is NOT being used, then __FILE__ ought to correspond to the build environment (if anything, it should use Style::native).

[thakis]

Normalizing to \ seems better to me than normalizing to / too.

I abstractly like changing as little as necessary at every stage, which in this case would mean changing just the slashiness of slashes that clang itself adds, so I'm still weakly in favor of that. But I also won't insist on it.

In D122766#3431468, @dexonsmith wrote:

In D122766#3429552, @hans wrote:

My feeling is that the default behavior on Windows needs to be to use backslashes and not forward slashes.

Okay, how would folks feel about always canonicalizing __FILE__ etc. to use backslashes when targeting Windows?

FWIW (I have little Windows experience and no stake in this!), defaulting to backslashes more sense to me. Also seems easy enough to add a preprocessor option to configure this.

I'm curious though, if you're building on POSIX (so the file starts out as POSIX) and targeting Windows, does calling make_preferred() actually give you something reasonable in the general case? Or should it be used *only* when there's a hit in LangOpts.remapPathPrefix()? Sorry if this has been discussed and rejected; I haven't followed the full thread; feel free to ignore if it's not relevant; but it seems to me that when -ffile-prefix-map is NOT being used, then __FILE__ ought to correspond to the build environment (if anything, it should use Style::native).

(We don't use -ffile-prefix-map. As https://blog.llvm.org/2019/11/deterministic-builds-with-clang-and-lld.html explains, using it means your commandlines *must* be machine-dependent, which we don't want. So we use -ffile-compilation-dir=. instead which has the same effect but doesn't have that drawback.)

[ayzhao]

So, the general consensus seems to be that we should use backslashes when targeting Windows.

I implemented using only backslashes for Windows; however, clang/test/SemaCXX/destructor.cpp fails when running on Linux with the following error (among other errors, but the one below is the most important).

...
error: 'error' diagnostics seen but not expected:
  Line 32: '<REDACTED_PATH_TO_LLVM_CHECKOUT_WITH_BACKSLASHES>\\clang\\test\\SemaCXX\\destructor.cpp' file not found
...

The reason for this is that the test has Clang target windows and the test also has the statement #include __FILE__.

One way to fix this would be to have every macro that accepts a path internally convert the path to the build environment's path format, but TBH I'm not sure. What do you all think?

[dexonsmith]

In D122766#3450298, @ayzhao wrote:

So, the general consensus seems to be that we should use backslashes when targeting Windows.

I implemented using only backslashes for Windows; however, clang/test/SemaCXX/destructor.cpp fails when running on Linux with the following error (among other errors, but the one below is the most important).

...
error: 'error' diagnostics seen but not expected:
  Line 32: '<REDACTED_PATH_TO_LLVM_CHECKOUT_WITH_BACKSLASHES>\\clang\\test\\SemaCXX\\destructor.cpp' file not found
...

The reason for this is that the test has Clang target windows and the test also has the statement #include __FILE__.

One way to fix this would be to have every macro that accepts a path internally convert the path to the build environment's path format, but TBH I'm not sure. What do you all think?

Wow, #include __FILE__ is kind of amazing...

It sounds like you're suggesting changing the #include directive to start canonicalizing its argument. I'm not a fan of that idea; header search is hard enough to reason about.

I suggest, instead:

• Only canonicalize __FILE__ for the target platform when there's a command-line flag that suggests it's okay. I suggest adding -ffile-reproducible, which could be implied by -ffile-prefix-map but also specified separately when -ffile-prefix-map isn't being used.
• When __FILE__ is being canonicalized, it's okay to fail on #include __FILE__ if the canonicalized path doesn't work as an argument to #include in the current build environment.

This would mean that -ffile-reproducible makes #include __FILE__ non-portable, which would be fine as long as #include __FILE__ isn't load-bearing for real life usecases. My guess is that it'd be okay?

What do others think?

[dexonsmith]

In D122766#3450348, @dexonsmith wrote:

• Only canonicalize __FILE__ for the target platform when there's a command-line flag that suggests it's okay. I suggest adding -ffile-reproducible, which could be implied by -ffile-prefix-map but also specified separately when -ffile-prefix-map isn't being used.

(Could also be implied by -ffile-compilation-dir=)

[hans]

In D122766#3450298, @ayzhao wrote:

So, the general consensus seems to be that we should use backslashes when targeting Windows.

I implemented using only backslashes for Windows; however, clang/test/SemaCXX/destructor.cpp fails when running on Linux with the following error (among other errors, but the one below is the most important).

...
error: 'error' diagnostics seen but not expected:
  Line 32: '<REDACTED_PATH_TO_LLVM_CHECKOUT_WITH_BACKSLASHES>\\clang\\test\\SemaCXX\\destructor.cpp' file not found
...

The reason for this is that the test has Clang target windows and the test also has the statement #include __FILE__.

That seems like an unusual use case of __FILE__. I wonder if we could just change the test to #include "destructor.cpp" instead and not worry about the case of #include __FILE__ in cross-builds targeting Windows.

[ayzhao]

In D122766#3451775, @hans wrote:

In D122766#3450298, @ayzhao wrote:

So, the general consensus seems to be that we should use backslashes when targeting Windows.

I implemented using only backslashes for Windows; however, clang/test/SemaCXX/destructor.cpp fails when running on Linux with the following error (among other errors, but the one below is the most important).

...
error: 'error' diagnostics seen but not expected:
  Line 32: '<REDACTED_PATH_TO_LLVM_CHECKOUT_WITH_BACKSLASHES>\\clang\\test\\SemaCXX\\destructor.cpp' file not found
...

The reason for this is that the test has Clang target windows and the test also has the statement #include __FILE__.

That seems like an unusual use case of __FILE__. I wonder if we could just change the test to #include "destructor.cpp" instead and not worry about the case of #include __FILE__ in cross-builds targeting Windows.

This line was introduced in D37235 (coincidentally written by @thakis), and in that review, @rsmith made a comment on this exact line that said

Do we guarantee that __FILE__ names a path that can be used to include the current file? In other tests, we add -include %s to the RUN: line to model this situation.

[ayzhao]

Implement -ffile-reproducible

[ayzhao]

missing newline

[ayzhao]

another missing newline

[ayzhao]

In D122766#3450348, @dexonsmith wrote:

In D122766#3450298, @ayzhao wrote:

So, the general consensus seems to be that we should use backslashes when targeting Windows.

I implemented using only backslashes for Windows; however, clang/test/SemaCXX/destructor.cpp fails when running on Linux with the following error (among other errors, but the one below is the most important).

...
error: 'error' diagnostics seen but not expected:
  Line 32: '<REDACTED_PATH_TO_LLVM_CHECKOUT_WITH_BACKSLASHES>\\clang\\test\\SemaCXX\\destructor.cpp' file not found
...

The reason for this is that the test has Clang target windows and the test also has the statement #include __FILE__.

One way to fix this would be to have every macro that accepts a path internally convert the path to the build environment's path format, but TBH I'm not sure. What do you all think?

Wow, #include __FILE__ is kind of amazing...

It sounds like you're suggesting changing the #include directive to start canonicalizing its argument. I'm not a fan of that idea; header search is hard enough to reason about.

I suggest, instead:

• Only canonicalize __FILE__ for the target platform when there's a command-line flag that suggests it's okay. I suggest adding -ffile-reproducible, which could be implied by -ffile-prefix-map but also specified separately when -ffile-prefix-map isn't being used.
• When __FILE__ is being canonicalized, it's okay to fail on #include __FILE__ if the canonicalized path doesn't work as an argument to #include in the current build environment.

This would mean that -ffile-reproducible makes #include __FILE__ non-portable, which would be fine as long as #include __FILE__ isn't load-bearing for real life usecases. My guess is that it'd be okay?

What do others think?

Patch has been updated to implement -ffile-reproducible

[aaron.ballman]

clang/include/clang/Driver/Options.td

3020–3025

These are both getting a bit long, can you wrap to 80 col?

clang/lib/Driver/ToolChains/Clang.cpp

1489–1490

I'm reasonably sure this does the same thing (but you should check to be sure).

clang/lib/Frontend/CompilerInvocation.cpp

4081–4084

I was hoping there was a clean way to do this using hasFlag() but I wasn't really excited by what I was getting from that; I think this is okay, but no need to qualify the option names.

clang/lib/Lex/PPMacroExpansion.cpp

1895–1900

LLVM style nit for single-line if/else clauses (the diff view makes it look more scary than it is).

[ayzhao]

code review comments

[hans]

I still wish this could just be the default behavior, but this flag seems a good compromise.

LGTM (but please wait for Aaron too)

clang/include/clang/Driver/Options.td

1516

I think "build environment" is maybe confusing, and "host" might be the better word.

[ayzhao]

Clarify --fno-file-reproducible HelpText

[aaron.ballman]

LGTM!

clang/include/clang/Basic/LangOptions.h

public:
   bool IsHeaderFile = false;
 
   /// The default stream kind used for HIP kernel launching.
   GPUDefaultStreamKind GPUDefaultStream;
 
   /// The seed used by the randomize structure layout feature.
   std::string RandstructSeed;
 
+  /// Indicates whether the __FILE__ macro should use the target's
+  /// platform-specific file separator or whether it should use the build
+  /// environment's platform-specific file separator.
+  ///
+  /// The plaform-specific path separator is the backslash(\) for Windows and
+  /// forward slash (/) elsewhere.
+  bool UseTargetPathSeparator = false;
+
   LangOptions();
 
   /// Set language defaults for the given input language and
   /// language standard in the given LangOptions object.
   ///
   /// \param Opts - The LangOptions object to set up.
   /// \param Lang - The input language.
   /// \param T - The target triple.
#include "clang/Basic/LangOptions.def"
 
   bool hasWasmExceptions() const {
     return getExceptionHandling() == ExceptionHandlingKind::Wasm;
   }
 
   bool isSYCL() const { return SYCLIsDevice || SYCLIsHost; }
 
   /// Remap path prefix according to -fmacro-prefix-path option.
-  void remapPathPrefix(SmallString<256> &Path) const;
+  void remapPathPrefix(SmallVectorImpl<char> &Path) const;
 };
 
 /// Floating point control options
 class FPOptionsOverride;
 class FPOptions {
 public:
   // We start by defining the layout.
   using storage_type = uint16_t;

clang/include/clang/Driver/Options.td

 def : Flag<["-"], "fno-expensive-optimizations">, Group<clang_ignored_gcc_optimization_f_Group>;
 def fextdirs_EQ : Joined<["-"], "fextdirs=">, Group<f_Group>;
 def : Flag<["-"], "fdefer-pop">, Group<clang_ignored_gcc_optimization_f_Group>;
 def : Flag<["-"], "fno-defer-pop">, Group<clang_ignored_gcc_optimization_f_Group>;
 def : Flag<["-"], "fextended-identifiers">, Group<clang_ignored_f_Group>;
 def : Flag<["-"], "fno-extended-identifiers">, Group<f_Group>, Flags<[Unsupported]>;
 def fhosted : Flag<["-"], "fhosted">, Group<f_Group>;
 def fdenormal_fp_math_EQ : Joined<["-"], "fdenormal-fp-math=">, Group<f_Group>, Flags<[CC1Option]>;
+def ffile_reproducible : Flag<["-"], "ffile-reproducible">, Group<f_Group>,
+  Flags<[CoreOption, CC1Option]>,
+  HelpText<"Use the target's platform-specific path separator character when "
+           "expanding the __FILE__ macro">;
+def fno_file_reproducible : Flag<["-"], "fno-file-reproducible">,
+  Group<f_Group>, Flags<[CoreOption, CC1Option]>,
+  HelpText<"Use the host's platform-specific path separator character when "
+           "expanding the __FILE__ macro">;
 def ffp_eval_method_EQ : Joined<["-"], "ffp-eval-method=">, Group<f_Group>, Flags<[CC1Option]>,
   HelpText<"Specifies the evaluation method to use for floating-point arithmetic.">,
   Values<"source,double,extended">, NormalizedValuesScope<"LangOptions">,
   NormalizedValues<["FEM_Source", "FEM_Double", "FEM_Extended"]>,
   MarshallingInfoEnum<LangOpts<"FPEvalMethod">, "FEM_UnsetOnCommandLine">;
 def ffp_model_EQ : Joined<["-"], "ffp-model=">, Group<f_Group>, Flags<[NoXarchOption]>,
   HelpText<"Controls the semantics of floating-point calculations.">;
 def ffp_exception_behavior_EQ : Joined<["-"], "ffp-exception-behavior=">, Group<f_Group>, Flags<[CC1Option]>,
: Joined<["-"], "fdebug-prefix-map=">, Group<f_Group>,
     Flags<[CC1Option,CC1AsOption]>,
     HelpText<"remap file source paths in debug info">;
 def fcoverage_prefix_map_EQ
   : Joined<["-"], "fcoverage-prefix-map=">, Group<f_Group>,
     Flags<[CC1Option]>,
     HelpText<"remap file source paths in coverage mapping">;
 def ffile_prefix_map_EQ
   : Joined<["-"], "ffile-prefix-map=">, Group<f_Group>,
-    HelpText<"remap file source paths in debug info, predefined preprocessor macros and __builtin_FILE()">;
+    HelpText<"remap file source paths in debug info, predefined preprocessor "
+             "macros and __builtin_FILE(). Implies -ffile-reproducible.">;
 def fmacro_prefix_map_EQ
   : Joined<["-"], "fmacro-prefix-map=">, Group<f_Group>, Flags<[CC1Option]>,
-    HelpText<"remap file source paths in predefined preprocessor macros and __builtin_FILE()">;
+    HelpText<"remap file source paths in predefined preprocessor macros and "
+             "__builtin_FILE(). Implies -ffile-reproducible.">;
 defm force_dwarf_frame : BoolFOption<"force-dwarf-frame",
   CodeGenOpts<"ForceDwarfFrameSection">, DefaultFalse,
   PosFlag<SetTrue, [CC1Option], "Always emit a debug frame section">, NegFlag<SetFalse>>;
 def g_Flag : Flag<["-"], "g">, Group<g_Group>,
   HelpText<"Generate source-level debug information">;
 def gline_tables_only : Flag<["-"], "gline-tables-only">, Group<gN_Group>,
   Flags<[CoreOption]>, HelpText<"Emit debug line number tables only">;
 def gline_directives_only : Flag<["-"], "gline-directives-only">, Group<gN_Group>,

clang/include/clang/Lex/Preprocessor.h

void emitMacroExpansionWarnings(const Token &Identifier) const {
     if (Identifier.getIdentifierInfo()->isDeprecatedMacro())
       emitMacroDeprecationWarning(Identifier);
 
     if (Identifier.getIdentifierInfo()->isRestrictExpansion() &&
         !SourceMgr.isInMainFile(Identifier.getLocation()))
       emitRestrictExpansionWarning(Identifier);
   }
 
+  static void processPathForFileMacro(SmallVectorImpl<char> &Path,
+                                      const LangOptions &LangOpts,
+                                      const TargetInfo &TI);
+
 private:
   void emitMacroDeprecationWarning(const Token &Identifier) const;
   void emitRestrictExpansionWarning(const Token &Identifier) const;
   void emitFinalMacroWarning(const Token &Identifier, bool IsUndef) const;
 
   Optional<unsigned>
   getSkippedRangeForExcludedConditionalBlock(SourceLocation HashLoc);
 

clang/lib/AST/Expr.cpp

 #include "clang/AST/RecordLayout.h"
 #include "clang/AST/StmtVisitor.h"
 #include "clang/Basic/Builtins.h"
 #include "clang/Basic/CharInfo.h"
 #include "clang/Basic/SourceManager.h"
 #include "clang/Basic/TargetInfo.h"
 #include "clang/Lex/Lexer.h"
 #include "clang/Lex/LiteralSupport.h"
+#include "clang/Lex/Preprocessor.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/Format.h"
 #include "llvm/Support/raw_ostream.h"
 #include <algorithm>
 #include <cstring>
 using namespace clang;
 
 const Expr *Expr::getBestDynamicClassTypeExpr() const {
auto MakeStringLiteral = [&](StringRef Tmp) {
     // Decay the string to a pointer to the first character.
     LValuePathEntry Path[1] = {LValuePathEntry::ArrayIndex(0)};
     return APValue(Res, CharUnits::Zero(), Path, /*OnePastTheEnd=*/false);
   };
 
   switch (getIdentKind()) {
   case SourceLocExpr::File: {
     SmallString<256> Path(PLoc.getFilename());
-    Ctx.getLangOpts().remapPathPrefix(Path);
+    clang::Preprocessor::processPathForFileMacro(Path, Ctx.getLangOpts(),
+                                                 Ctx.getTargetInfo());
     return MakeStringLiteral(Path);
   }
   case SourceLocExpr::Function: {
     const auto *CurDecl = dyn_cast<Decl>(Context);
     return MakeStringLiteral(
         CurDecl ? PredefinedExpr::ComputeName(PredefinedExpr::Function, CurDecl)
                 : std::string(""));
   }
case SourceLocExpr::SourceLocStruct: {
     // corresponding to that. Note that we've already verified that the shape of
     // the ImplDecl type is as expected.
 
     APValue Value(APValue::UninitStruct(), 0, 4);
     for (FieldDecl *F : ImplDecl->fields()) {
       StringRef Name = F->getName();
       if (Name == "_M_file_name") {
         SmallString<256> Path(PLoc.getFilename());
-        Ctx.getLangOpts().remapPathPrefix(Path);
+        clang::Preprocessor::processPathForFileMacro(Path, Ctx.getLangOpts(),
+                                                     Ctx.getTargetInfo());
         Value.getStructField(F->getFieldIndex()) = MakeStringLiteral(Path);
       } else if (Name == "_M_function_name") {
         // Note: this emits the PrettyFunction name -- different than what
         // __builtin_FUNCTION() above returns!
         const auto *CurDecl = dyn_cast<Decl>(Context);
         Value.getStructField(F->getFieldIndex()) = MakeStringLiteral(
             CurDecl && !isa<TranslationUnitDecl>(CurDecl)
                 ? StringRef(PredefinedExpr::ComputeName(

clang/lib/Basic/LangOptions.cpp

if (!OpenCLCPlusPlus)
     return OpenCLVersion;
   if (OpenCLCPlusPlusVersion == 100)
     return 200;
   if (OpenCLCPlusPlusVersion == 202100)
     return 300;
   llvm_unreachable("Unknown OpenCL version");
 }
 
-void LangOptions::remapPathPrefix(SmallString<256> &Path) const {
+void LangOptions::remapPathPrefix(SmallVectorImpl<char> &Path) const {
   for (const auto &Entry : MacroPrefixMap)
     if (llvm::sys::path::replace_path_prefix(Path, Entry.first, Entry.second))
       break;
 }
 
 std::string LangOptions::getOpenCLVersionString() const {
   std::string Result;
   {

clang/lib/Driver/ToolChains/Clang.cpp

forAllAssociatedToolChains(C, JA, getToolChain(),
                                });
   else {
     // For IAMCU add special include arguments.
     getToolChain().AddIAMCUIncludeArgs(Args, CmdArgs);
   }
 
   addMacroPrefixMapArg(D, Args, CmdArgs);
   addCoveragePrefixMapArg(D, Args, CmdArgs);
+
+  Args.AddLastArg(CmdArgs, options::OPT_ffile_reproducible,
+                  options::OPT_fno_file_reproducible);
 }
 
 // FIXME: Move to target hook.
 static bool isSignedCharDefault(const llvm::Triple &Triple) {
   switch (Triple.getArch()) {
   default:
     return true;
 

clang/lib/Frontend/CompilerInvocation.cpp

if (Opts.CXXABI)
     GenerateArg(Args, OPT_fcxx_abi_EQ, TargetCXXABI::getSpelling(*Opts.CXXABI),
                 SA);
 
   if (Opts.RelativeCXXABIVTables)
     GenerateArg(Args, OPT_fexperimental_relative_cxx_abi_vtables, SA);
   else
     GenerateArg(Args, OPT_fno_experimental_relative_cxx_abi_vtables, SA);
 
+  if (Opts.UseTargetPathSeparator)
+    GenerateArg(Args, OPT_ffile_reproducible, SA);
+  else
+    GenerateArg(Args, OPT_fno_file_reproducible, SA);
+
   for (const auto &MP : Opts.MacroPrefixMap)
     GenerateArg(Args, OPT_fmacro_prefix_map_EQ, MP.first + "=" + MP.second, SA);
 
   if (!Opts.RandstructSeed.empty())
     GenerateArg(Args, OPT_frandomize_layout_seed_EQ, Opts.RandstructSeed, SA);
 }
 
 bool CompilerInvocation::ParseLangArgs(LangOptions &Opts, ArgList &Args,
Opts.RelativeCXXABIVTables =
                    TargetCXXABI::usesRelativeVTables(T));
 
   for (const auto &A : Args.getAllArgValues(OPT_fmacro_prefix_map_EQ)) {
     auto Split = StringRef(A).split('=');
     Opts.MacroPrefixMap.insert(
         {std::string(Split.first), std::string(Split.second)});
   }
 
+  Opts.UseTargetPathSeparator =
+      !Args.getLastArg(OPT_fno_file_reproducible) &&
+      (Args.getLastArg(OPT_ffile_compilation_dir_EQ) ||
+       Args.getLastArg(OPT_fmacro_prefix_map_EQ) ||
+       Args.getLastArg(OPT_ffile_reproducible));
+
   // Error if -mvscale-min is unbounded.
   if (Arg *A = Args.getLastArg(options::OPT_mvscale_min_EQ)) {
     unsigned VScaleMin;
     if (StringRef(A->getValue()).getAsInteger(10, VScaleMin) || VScaleMin == 0)
       Diags.Report(diag::err_cc1_unbounded_vscale_min);
   }
 
   if (const Arg *A = Args.getLastArg(OPT_frandomize_layout_seed_file_EQ)) {

clang/lib/Lex/PPMacroExpansion.cpp

if (PLoc.isValid()) {
         StringRef PLFileName = llvm::sys::path::filename(PLoc.getFilename());
         if (PLFileName != "")
           FN += PLFileName;
         else
           FN += PLoc.getFilename();
       } else {
         FN += PLoc.getFilename();
       }
-      getLangOpts().remapPathPrefix(FN);
+      processPathForFileMacro(FN, getLangOpts(), getTargetInfo());
       Lexer::Stringify(FN);
       OS << '"' << FN << '"';
     }
     Tok.setKind(tok::string_literal);
   } else if (II == Ident__DATE__) {
     Diag(Tok.getLocation(), diag::warn_pp_date_time);
     if (!DATELoc.isValid())
       ComputeDATE_TIME(DATELoc, TIMELoc, *this);

 
 void Preprocessor::markMacroAsUsed(MacroInfo *MI) {
   // If the 'used' status changed, and the macro requires 'unused' warning,
   // remove its SourceLocation from the warn-for-unused-macro locations.
   if (MI->isWarnIfUnused() && !MI->isUsed())
     WarnUnusedMacroLocs.erase(MI->getDefinitionLoc());
   MI->setIsUsed(true);
 }
+
+void Preprocessor::processPathForFileMacro(SmallVectorImpl<char> &Path,
+                                           const LangOptions &LangOpts,
+                                           const TargetInfo &TI) {
+  LangOpts.remapPathPrefix(Path);
+  if (LangOpts.UseTargetPathSeparator) {
+    if (TI.getTriple().isOSWindows())
+      llvm::sys::path::make_preferred(
+          Path, llvm::sys::path::Style::windows_backslash);
+    else
+      llvm::sys::path::make_preferred(Path, llvm::sys::path::Style::posix);
+  }
+}

clang/test/CodeGenCXX/builtin-source-location.cpp

 // RUN: %clang_cc1 -no-opaque-pointers -std=c++2a -fblocks %s -triple x86_64-unknown-unknown -emit-llvm -o %t.ll
 
 // This needs to be performed before #line directives which alter filename
-// RUN: %clang_cc1 -no-opaque-pointers -fmacro-prefix-map=%p=/UNLIKELY/PATH -emit-llvm -o - %s | FileCheck %s --check-prefix=CHECK-PREFIX-MAP
+// RUN: %clang_cc1 -no-opaque-pointers -fno-file-reproducible -fmacro-prefix-map=%p=/UNLIKELY/PATH -emit-llvm -o - %s | FileCheck %s --check-prefix=CHECK-PREFIX-MAP
 //
 // CHECK-PREFIX-MAP: /UNLIKELY/PATH{{/|\\\\}}builtin-source-location.cpp
 void testRemap() {
   const char *file = __builtin_FILE();
 }
 
 #line 8 "builtin-source-location.cpp"
 

clang/test/Preprocessor/file_test.c

 // UNSUPPORTED: system-windows
 // RUN: %clang -E -ffile-prefix-map=%p=/UNLIKELY_PATH/empty -c -o - %s | FileCheck %s
 // RUN: %clang -E -fmacro-prefix-map=%p=/UNLIKELY_PATH/empty -c -o - %s | FileCheck %s
 // RUN: %clang -E -fmacro-prefix-map=%p=/UNLIKELY_PATH=empty -c -o - %s | FileCheck %s -check-prefix CHECK-EVIL
 // RUN: %clang -E -fmacro-prefix-map=%p/= -c -o - %s | FileCheck %s --check-prefix CHECK-REMOVE
+// RUN: %clang -E -fno-file-reproducible -fmacro-prefix-map=%p/= -target x86_64-pc-win32 -c -o - %s | FileCheck %s --check-prefix CHECK-REMOVE
+// RUN: %clang -E -fno-file-reproducible -ffile-prefix-map=%p/= -target x86_64-pc-win32 -c -o - %s | FileCheck %s --check-prefix CHECK-REMOVE
+// RUN: %clang -E -fmacro-prefix-map=%p/= -target x86_64-pc-win32 -c -o - %s | FileCheck %s --check-prefix CHECK-WINDOWS
+// RUN: %clang -E -ffile-prefix-map=%p/= -target x86_64-pc-win32 -c -o - %s | FileCheck %s --check-prefix CHECK-WINDOWS
+// RUN: %clang -E -ffile-reproducible -target x86_64-pc-win32 -c -o - %s | FileCheck %s --check-prefix CHECK-WINDOWS-REPRODUCIBLE
 
 filename: __FILE__
 #include "Inputs/include-file-test/file_test.h"
 
 // CHECK: filename: "/UNLIKELY_PATH/empty/file_test.c"
 // CHECK: filename: "/UNLIKELY_PATH/empty/Inputs/include-file-test/file_test.h"
 // CHECK: basefile: "/UNLIKELY_PATH/empty/file_test.c"
 // CHECK-NOT: filename:
 
 // CHECK-EVIL: filename: "/UNLIKELY_PATH=empty/file_test.c"
 // CHECK-EVIL: filename: "/UNLIKELY_PATH=empty/Inputs/include-file-test/file_test.h"
 // CHECK-EVIL: basefile: "/UNLIKELY_PATH=empty/file_test.c"
 // CHECK-EVIL-NOT: filename:
 
 // CHECK-REMOVE: filename: "file_test.c"
 // CHECK-REMOVE: filename: "Inputs/include-file-test/file_test.h"
 // CHECK-REMOVE: basefile: "file_test.c"
 // CHECK-REMOVE-NOT: filename:
+
+// CHECK-WINDOWS: filename: "file_test.c"
+// CHECK-WINDOWS: filename: "Inputs\\include-file-test\\file_test.h"
+// CHECK-WINDOWS: basefile: "file_test.c"
+// CHECK-WINDOWS-NOT: filename:
+
+// CHECK-WINDOWS-REPRODUCIBLE: filename: "{{[^/]*}}file_test.c"
+// CHECK-WINDOWS-REPRODUCIBLE: filename: "{{[^/]*}}Inputs\\include-file-test\\file_test.h"
+// CHECK-WINDOWS-REPRODUCIBLE: basefile: "{{[^/]*}}file_test.c"
+// CHECK-WINDOWS-REPRODUCIBLE-NOT: filename:

clang/test/Preprocessor/file_test_windows.c

 // REQUIRES: system-windows
-// RUN: %clang -E -ffile-prefix-map=%p=A:\UNLIKELY_PATH\empty -c -o - %s | FileCheck %s
-// RUN: %clang -E -fmacro-prefix-map=%p=A:\UNLIKELY_PATH\empty -c -o - %s | FileCheck %s
-// RUN: %clang -E -fmacro-prefix-map=%p=A:\UNLIKELY_PATH=empty -c -o - %s | FileCheck %s -check-prefix CHECK-EVIL
-// RUN: %clang -E -fmacro-prefix-map=%p/iNPUTS\=A:\UNLIKELY_PATH_INC\ -fmacro-prefix-map=%p/=A:\UNLIKELY_PATH_BASE\ -c -o - %s | FileCheck %s -check-prefix CHECK-CASE
-// RUN: %clang -E -fmacro-prefix-map=%p\= -c -o - %s | FileCheck %s --check-prefix CHECK-REMOVE
+// RUN: %clang -E -fno-file-reproducible -ffile-prefix-map=%p=A:\UNLIKELY_PATH\empty -c -o - %s | FileCheck %s
+// RUN: %clang -E -fno-file-reproducible -fmacro-prefix-map=%p=A:\UNLIKELY_PATH\empty -c -o - %s | FileCheck %s
+// RUN: %clang -E -fno-file-reproducible -fmacro-prefix-map=%p=A:\UNLIKELY_PATH=empty -c -o - %s | FileCheck %s -check-prefix CHECK-EVIL
+// RUN: %clang -E -fno-file-reproducible -fmacro-prefix-map=%p/iNPUTS\=A:\UNLIKELY_PATH_INC\ -fmacro-prefix-map=%p/=A:\UNLIKELY_PATH_BASE\ -c -o - %s | FileCheck %s -check-prefix CHECK-CASE
+// RUN: %clang -E -fno-file-reproducible -fmacro-prefix-map=%p\= -c -o - %s | FileCheck %s --check-prefix CHECK-REMOVE
+
+// RUN: %clang -E -ffile-prefix-map=%p=A:\UNLIKELY_PATH\empty -c -o - %s | FileCheck %s --check-prefix CHECK-REPRODUCIBLE
+// RUN: %clang -E -fmacro-prefix-map=%p=A:\UNLIKELY_PATH\empty -c -o - %s | FileCheck %s --check-prefix CHECK-REPRODUCIBLE
+// RUN: %clang -E -fmacro-prefix-map=%p=A:\UNLIKELY_PATH=empty -c -o - %s | FileCheck %s -check-prefix CHECK-EVIL-REPRODUCIBLE
+// RUN: %clang -E -fmacro-prefix-map=%p/iNPUTS\=A:\UNLIKELY_PATH_INC\ -fmacro-prefix-map=%p/=A:\UNLIKELY_PATH_BASE\ -c -o - %s | FileCheck %s -check-prefix CHECK-CASE-REPRODUCIBLE
+// RUN: %clang -E -fmacro-prefix-map=%p\= -c -o - %s | FileCheck %s --check-prefix CHECK-REMOVE-REPRODUCIBLE
+
+// RUN: %clang -E -target x86_64-pc-linux-gnu -fmacro-prefix-map=%p\= -c -o - %s | FileCheck %s --check-prefix CHECK-REMOVE
+// RUN: %clang -E -target x86_64-pc-linux-gnu -ffile-prefix-map=%p\= -c -o - %s | FileCheck %s --check-prefix CHECK-REMOVE
+
+// Clang defaults to forward slashes for the non-prefix portion of the path even if the build environment is Windows.
+// RUN: %clang -E -fno-file-reproducible -target x86_64-pc-linux-gnu -fmacro-prefix-map=%p\= -c -o - %s | FileCheck %s --check-prefix CHECK-REMOVE
+// RUN: %clang -E -fno-file-reproducible -target x86_64-pc-linux-gnu -ffile-prefix-map=%p\= -c -o - %s | FileCheck %s --check-prefix CHECK-REMOVE
+
+// RUN: %clang -E -ffile-reproducible -target x86_64-pc-win32 -c -o - %s | FileCheck %s --check-prefix CHECK-WINDOWS-FULL
+// RUN: %clang -E -ffile-reproducible -target x86_64-pc-linux-gnu -c -o - %s | FileCheck %s --check-prefix CHECK-LINUX-FULL
 
 filename: __FILE__
 #include "Inputs/include-file-test/file_test.h"
 
 // CHECK: filename: "A:\\UNLIKELY_PATH\\empty\\file_test_windows.c"
 // CHECK: filename: "A:\\UNLIKELY_PATH\\empty/Inputs/include-file-test/file_test.h"
 // CHECK: basefile: "A:\\UNLIKELY_PATH\\empty\\file_test_windows.c"
 // CHECK-NOT: filename:
 
 // CHECK-EVIL: filename: "A:\\UNLIKELY_PATH=empty\\file_test_windows.c"
 // CHECK-EVIL: filename: "A:\\UNLIKELY_PATH=empty/Inputs/include-file-test/file_test.h"
 // CHECK-EVIL: basefile: "A:\\UNLIKELY_PATH=empty\\file_test_windows.c"
 // CHECK-EVIL-NOT: filename:
 
 // CHECK-CASE: filename: "A:\\UNLIKELY_PATH_BASE\\file_test_windows.c"
 // CHECK-CASE: filename: "A:\\UNLIKELY_PATH_INC\\include-file-test/file_test.h"
 // CHECK-CASE: basefile: "A:\\UNLIKELY_PATH_BASE\\file_test_windows.c"
 // CHECK-CASE-NOT: filename:
 
 // CHECK-REMOVE: filename: "file_test_windows.c"
 // CHECK-REMOVE: filename: "Inputs/include-file-test/file_test.h"
 // CHECK-REMOVE: basefile: "file_test_windows.c"
 // CHECK-REMOVE-NOT: filename:
+
+// CHECK-REPRODUCIBLE: filename: "A:\\UNLIKELY_PATH\\empty\\file_test_windows.c"
+// CHECK-REPRODUCIBLE: filename: "A:\\UNLIKELY_PATH\\empty\\Inputs\\include-file-test\\file_test.h"
+// CHECK-REPRODUCIBLE: basefile: "A:\\UNLIKELY_PATH\\empty\\file_test_windows.c"
+// CHECK-REPRODUCIBLE-NOT: filename:
+
+// CHECK-EVIL-REPRODUCIBLE: filename: "A:\\UNLIKELY_PATH=empty\\file_test_windows.c"
+// CHECK-EVIL-REPRODUCIBLE: filename: "A:\\UNLIKELY_PATH=empty\\Inputs\\include-file-test\\file_test.h"
+// CHECK-EVIL-REPRODUCIBLE: basefile: "A:\\UNLIKELY_PATH=empty\\file_test_windows.c"
+// CHECK-EVIL-REPRODUCIBLE-NOT: filename:
+
+// CHECK-CASE-REPRODUCIBLE: filename: "A:\\UNLIKELY_PATH_BASE\\file_test_windows.c"
+// CHECK-CASE-REPRODUCIBLE: filename: "A:\\UNLIKELY_PATH_INC\\include-file-test\\file_test.h"
+// CHECK-CASE-REPRODUCIBLE: basefile: "A:\\UNLIKELY_PATH_BASE\\file_test_windows.c"
+// CHECK-CASE-REPRODUCIBLE-NOT: filename:
+
+// CHECK-REMOVE-REPRODUCIBLE: filename: "file_test_windows.c"
+// CHECK-REMOVE-REPRODUCIBLE: filename: "Inputs\\include-file-test\\file_test.h"
+// CHECK-REMOVE-REPRODUCIBLE: basefile: "file_test_windows.c"
+// CHECK-REMOVE-REPRODUCIBLE-NOT: filename:
+
+// CHECK-WINDOWS-FULL: filename: "{{[^/]*}}file_test_windows.c"
+// CHECK-WINDOWS-FULL: filename: "{{[^/]*}}Inputs\\include-file-test\\file_test.h"
+// CHECK-WINDOWS-FULL: basefile: "{{[^/]*}}file_test_windows.c"
+// CHECK-WINDOWS-FULL-NOT: filename:
+
+// Clang does not modify the prefix for POSIX style, so it may have backslashes.
+// CHECK-LINUX-FULL: filename: "{{.*}}file_test_windows.c"
+// CHECK-LINUX-FULL: filename: "{{.*}}Inputs/include-file-test/file_test.h"
+// CHECK-LINUX-FULL: basefile: "{{.*}}file_test_windows.c"
+// CHECK-LINUX-FULL-NOT: filename: