[Frontend] Recognize environment variable SOURCE_DATE_EPOCH

[MaskRay]

Summary

See https://reproducible-builds.org/docs/source-date-epoch/ . The environment
variable `SOURCE_DATE_EPOCH` been recognized by many compilers.

In GCC, if SOURCE_DATE_EPOCH is set, it specifies a UNIX timestamp to be used
in replacement of the current date and time in the __DATE__ and __TIME__
macros. Note: GCC as of today does not update __TIMESTAMP__ (the modification
time of the current source file) but
https://wiki.debian.org/ReproducibleBuilds/TimestampsFromCPPMacros expresses the
intention to update it.

This patches parses SOURCE_DATE_EPOCH and changes all the three macros.

In addition, in case gmtime/localtime returns null (e.g. on 64-bit Windows
gmtime returns null when the timestamp is larger than 32536850399
(3001-01-19T21:59:59Z)), use ??? ?? ???? as used by GCC.

[MaskRay]

https://wiki.debian.org/ReproducibleBuilds/TimestampsFromCPPMacros says

We missed out TIMESTAMP by mistake, but we will probably send in another patch to GCC to fix that.

Let Clang update __TIMESTAMP__ as well.

[ychen]

The Windows premerge failure looks relevant.

[MaskRay]

Use %if clang-target-64-bits

[ychen]

clang/lib/Frontend/CompilerInvocation.cpp

4313

The time_t value may be negative and its value range is platform-dependent. It is better to be as transparent as possible. gmtime/localtime should be able to do the check by returning nullptr.

getAsInteger could handle V as time_t directly.

clang/lib/Lex/PPMacroExpansion.cpp

1092

Why not use localtime as the else branch?

As mentioned above, diagnose null TM here or when parsing the user-specifed value. A small trade-off to make, up to you.

[MaskRay]

Make error message precise

[MaskRay]

clang/lib/Lex/PPMacroExpansion.cpp

1092

Can you elaborate? With the check in Frontend I think a check here is not necessary...

[ychen]

clang/lib/Lex/PPMacroExpansion.cpp

1092

Sorry I was not clear. The time_t value is not portable so it is hard to diagnose in a target-independent way. For example, 253402300799 is too big of a value for windows that gmtime returns null (causes crash at asctime in the similar case below). We could diagnose when gmtime/localtime returns null, which is portable.

[MaskRay]

clang/lib/Lex/PPMacroExpansion.cpp

1092

I am still confused. In practice, a 64-bit gmtime implementation may returns null when tm_year would overflow while 32-bit gmtime can't. The way (with a more restricted upper bound) we check SOURCE_DATE_EPOCH in Frontend, std::gmtime(&TT); cannot return null. localtime(&TT) for time(nullptr) may overflow as where but that is an pre-existing issue.

[ychen]

clang/lib/Lex/PPMacroExpansion.cpp

1092

std::gmtime(&TT); cannot return null

I couldn't find good references for this. But on windows, it may return null (as the premerge CI shows). https://godbolt.org/z/jfxdG3KrM

[MaskRay]

Use ??? ?? ???? or ??:??:?? is gmtime/localtime returns null

[MaskRay]

clang/lib/Lex/PPMacroExpansion.cpp

1092

Thanks:) This is interesting. So Windows gmtime returns null when the year number is slightly larger than 3000 (I do not find the exact date...)

[emaste]

The time_t value may be negative

True, but we do not need to allow a negative SOURCE_DATE_EPOCH though. The spec implies that it must be nonnegative, although doesn't say so explicitly. https://reproducible-builds.org/specs/source-date-epoch/

[ychen]

Looks good to me except for one nit. And the test still fails on Windows for some reason.

clang/lib/Lex/PPMacroExpansion.cpp

1606

To be consistent with existing code, it might be better to use std::localtime instead of std::gmtime?

[MaskRay]

clang/lib/Lex/PPMacroExpansion.cpp

1606

No. This must use std::gmtime. localtime would introduce an undesired UTC time offset.

[ychen]

clang/lib/Lex/PPMacroExpansion.cpp

1606

Are you sure? https://linux.die.net/man/3/localtime gmtime/localtime both accept a UTC time. I think the only difference is the time zone.

[MaskRay]

clang/lib/Lex/PPMacroExpansion.cpp

1606

I'm sure. If you change this gmtime to localtime, for a timestamp which would otherwise display as 00:00:00, it would display as 16:00:00 if your timezone is UTC-0800.

[MaskRay]

Don't test SOURCE_DATE_EPOCH=253402300799 on Windows (which only supports upto 32536850399)

[ychen]

LGTM.

clang/docs/ReleaseNotes.rst

- When loading default configuration files, clang now unconditionally uses
   also loaded when clang is called via an executable without a prefix (e.g.
   plain ``clang``).
 - Default configuration paths were partially changed. Clang now attempts to load
   ``<triple>-<driver>.cfg`` first, and falls back to loading both
   ``<driver>.cfg`` and ``<triple>.cfg`` if the former is not found. `Triple`
   is the target triple and `driver` first tries the canonical name
   for the driver (respecting ``--driver-mode=``), and then the name found
   in the executable.
+- If the environment variable ``SOURCE_DATE_EPOCH`` is set, it specifies a UNIX
+  timestamp to be used in replacement of the current date and time in
+  the ``__DATE__``, ``__TIME__``, and ``__TIMESTAMP__`` macros. See
+  `<https://reproducible-builds.org/docs/source-date-epoch/>`_.
 
 New Compiler Flags
 ------------------
 
 - Implemented `-fcoro-aligned-allocation` flag. This flag implements
   Option 2 of P2014R0 aligned allocation of coroutine frames
   (`P2014R0 <https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/p2014r0.pdf>`_).
   With this flag, the coroutines will try to lookup aligned allocation

clang/include/clang/Basic/DiagnosticFrontendKinds.td

def err_fe_action_not_available : Error<
     "action %0 not compiled in">;
 def err_fe_invalid_alignment : Error<
     "invalid value '%1' in '%0'; alignment must be a power of 2">;
 def err_fe_invalid_exception_model
    : Error<"invalid exception model '%select{none|sjlj|seh|dwarf|wasm}0' for target '%1'">;
 def warn_fe_concepts_ts_flag : Warning<
   "-fconcepts-ts is deprecated - use '-std=c++20' for Concepts support">,
   InGroup<Deprecated>;
+def err_fe_invalid_source_date_epoch : Error<
+    "environment variable 'SOURCE_DATE_EPOCH' ('%0') must be a non-negative integer <= 253402300799">;
 
 def err_fe_unable_to_load_basic_block_sections_file : Error<
     "unable to load basic block sections function list: '%0'">;
 
 def warn_fe_serialized_diag_merge_failure : Warning<
     "unable to merge a subprocess's serialized diagnostics">,
     InGroup<SerializedDiagnostics>;
 def warn_fe_serialized_diag_failure : Warning<

clang/include/clang/Lex/PreprocessorOptions.h

std::function<Optional<ArrayRef<dependency_directives_scan::Directive>>(
       DependencyDirectivesForFile;
 
   /// Set up preprocessor for RunAnalysis action.
   bool SetUpStaticAnalyzer = false;
 
   /// Prevents intended crashes when using #pragma clang __debug. For testing.
   bool DisablePragmaDebugCrash = false;
 
+  /// If set, the UNIX timestamp specified by SOURCE_DATE_EPOCH.
+  Optional<uint64_t> SourceDateEpoch;
+
 public:
   PreprocessorOptions() : PrecompiledPreambleBytes(0, false) {}
 
   void addMacroDef(StringRef Name) {
     Macros.emplace_back(std::string(Name), false);
   }
   void addMacroUndef(StringRef Name) {
     Macros.emplace_back(std::string(Name), true);

clang/lib/Frontend/CompilerInvocation.cpp

for (const auto *A : Args.filtered(OPT_remap_file)) {
     if (Split.second.empty()) {
       Diags.Report(diag::err_drv_invalid_remap_file) << A->getAsString(Args);
       continue;
     }
 
     Opts.addRemappedFile(Split.first, Split.second);
   }
 
+  if (const char *Epoch = std::getenv("SOURCE_DATE_EPOCH")) {
+    uint64_t V;
+    // 253402300799 is the UNIX timestamp of 9999-12-31T23:59:59Z.
+    if (StringRef(Epoch).getAsInteger(10, V) || V > 253402300799)
+      Diags.Report(diag::err_fe_invalid_source_date_epoch) << Epoch;
+    else
+      Opts.SourceDateEpoch = V;
+  }
+
   // Always avoid lexing editor placeholders when we're just running the
   // preprocessor as we never want to emit the
   // "editor placeholder in source file" error in PP only mode.
   if (isStrictlyPreprocessorAction(Action))
     Opts.LexEditorPlaceholders = false;
 
   return Diags.getNumErrors() == NumErrorsBefore;
 }

clang/lib/Lex/PPMacroExpansion.cpp

void Preprocessor::removeCachedMacroExpandedTokensOfLastLexer() {
   MacroExpandingLexersStack.pop_back();
 }
 
 /// ComputeDATE_TIME - Compute the current time, enter it into the specified
 /// scratch buffer, then return DATELoc/TIMELoc locations with the position of
 /// the identifier tokens inserted.
 static void ComputeDATE_TIME(SourceLocation &DATELoc, SourceLocation &TIMELoc,
                              Preprocessor &PP) {
-  time_t TT = time(nullptr);
-  struct tm *TM = localtime(&TT);
+  time_t TT;
+  std::tm *TM;
+  if (PP.getPreprocessorOpts().SourceDateEpoch) {
+    TT = *PP.getPreprocessorOpts().SourceDateEpoch;
+    TM = std::gmtime(&TT);
+  } else {
+    TT = std::time(nullptr);
+    TM = std::localtime(&TT);
+  }
 
   static const char * const Months[] = {
     "Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec"
   };
 
   {
     SmallString<32> TmpBuffer;
     llvm::raw_svector_ostream TmpStream(TmpBuffer);
if (II == Ident__LINE__) {
 
     // __INCLUDE_LEVEL__ expands to a simple numeric value.
     OS << Depth;
     Tok.setKind(tok::numeric_constant);
   } else if (II == Ident__TIMESTAMP__) {
     Diag(Tok.getLocation(), diag::warn_pp_date_time);
     // MSVC, ICC, GCC, VisualAge C++ extension.  The generated string should be
     // of the form "Ddd Mmm dd hh::mm::ss yyyy", which is returned by asctime.
-
+    const char *Result;
+    if (getPreprocessorOpts().SourceDateEpoch) {
+      time_t TT = *getPreprocessorOpts().SourceDateEpoch;
+      std::tm *TM = std::gmtime(&TT);
+      Result = asctime(TM);
+    } else {
       // Get the file that we are lexing out of.  If we're currently lexing from
       // a macro, dig into the include stack.
       const FileEntry *CurFile = nullptr;
-    PreprocessorLexer *TheLexer = getCurrentFileLexer();
-
-    if (TheLexer)
+      if (PreprocessorLexer *TheLexer = getCurrentFileLexer())
         CurFile = SourceMgr.getFileEntryForID(TheLexer->getFileID());
-
-    const char *Result;
       if (CurFile) {
         time_t TT = CurFile->getModificationTime();
         struct tm *TM = localtime(&TT);
         Result = asctime(TM);
       } else {
         Result = "??? ??? ?? ??:??:?? ????\n";
       }
+    }
     // Surround the string with " and strip the trailing newline.
     OS << '"' << StringRef(Result).drop_back() << '"';
     Tok.setKind(tok::string_literal);
   } else if (II == Ident__FLT_EVAL_METHOD__) {
     // __FLT_EVAL_METHOD__ is set to the default value.
     if (getTUFPEvalMethod() ==
         LangOptions::FPEvalMethodKind::FEM_Indeterminable) {
       // This is possible if `AllowFPReassoc` or `AllowReciprocal` is enabled.

clang/test/Preprocessor/SOURCE_DATE_EPOCH.c

+// RUN: env SOURCE_DATE_EPOCH=0 %clang_cc1 -E %s | FileCheck %s --check-prefix=19700101
+
+// 19700101:      const char date[] = "Jan  1 1970";
+// 19700101-NEXT: const char time[] = "00:00:00";
+// 19700101-NEXT: const char timestamp[] = "Thu Jan  1 00:00:00 1970";
+
+// RUN: env SOURCE_DATE_EPOCH=2147483647 %clang_cc1 -E -Wdate-time %s 2>&1 | FileCheck %s --check-prefix=Y2038
+
+// Y2038:      warning: expansion of date or time macro is not reproducible [-Wdate-time]
+// Y2038:      const char date[] = "Jan 19 2038";
+// Y2038-NEXT: const char time[] = "03:14:07";
+// Y2038-NEXT: const char timestamp[] = "Tue Jan 19 03:14:07 2038";
+
+/// Some 32-bits systems have time64 problems.
+// RUN: %if clang-target-64-bits %{ env SOURCE_DATE_EPOCH=253402300799 %clang_cc1 -E -Wdate-time %s 2>&1 | FileCheck %s --check-prefix=99991231 %}
+
+// 99991231:      warning: expansion of date or time macro is not reproducible [-Wdate-time]
+// 99991231:      const char date[] = "Dec 31 9999";
+// 99991231-NEXT: const char time[] = "23:59:59";
+// 99991231-NEXT: const char timestamp[] = "Fri Dec 31 23:59:59 9999";
+
+// RUN: env SOURCE_DATE_EPOCH=253402300800 not %clang_cc1 -E %s 2>&1 | FileCheck %s --check-prefix=TOOBIG
+
+// TOOBIG: error: environment variable 'SOURCE_DATE_EPOCH' ('253402300800') must be a non-negative integer <= 253402300799
+
+// RUN: env SOURCE_DATE_EPOCH=0x0 not %clang_cc1 -E %s 2>&1 | FileCheck %s --check-prefix=NOTDECIMAL
+
+// NOTDECIMAL: error: environment variable 'SOURCE_DATE_EPOCH' ('0x0') must be a non-negative integer <= 253402300799
+
+const char date[] = __DATE__;
+const char time[] = __TIME__;
+const char timestamp[] = __TIMESTAMP__;