Rebase D23934: Add a -ffixed-date-time= flag that sets the initial value of __DATE__, __TIME__, __TIMESTAMP__

[appcs]

Summary

[Preprocessor] Add a -ffixed-date-time= flag that sets the initial value of DATE, TIME, TIMESTAMP

Rebase D23934.

[appcs]

(From D121040) Please let me know about the following and I will incorporate your feedback and submit subsequent patches for addressing these questions:

a. Disallow the simultaneous use of -Wdate-time/-Wpch-date-time and -ffixed-date-time=<timestamp>?
b. Disallow -ffixed-date-time=<timestamp>:

0. if PCH's are used
1. in PCH compilation

Also, (just as -Wdate-time warns on TIMESTAMP use), do we (in a separate patch), extend the scope of -Wpch-date-time to cover the use of TIMESTAMP is in a PCH?

Thank you.

[MaskRay]

Bazel uses -Wno-builtin-macro-redefined -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted".
For reproducibility purposes, this is sufficient. I have two questions:

• Do we need the arbitrary time feature?
• Should an arbitrary string like "redacted" be supported?

[appcs]

In D122517#3409266, @MaskRay wrote:

Bazel uses -Wno-builtin-macro-redefined -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted".
For reproducibility purposes, this is sufficient. I have two questions:

• Do we need the arbitrary time feature?

If you are saying that defining DATE, TIMESTAMP, TIME to an arbitrary time value, then that is the definition of the problem that we are trying to solve; else, please clarify. Thanks.

• Should an arbitrary string like "redacted" be supported?

Via -ffixed-date-time the value would have to be formatted e.g. 1969-12-31T23:59:59 (RFC3339 without fractional seconds and timezone); else, please clarify. Thanks.

[aaron.ballman]

clang/lib/Frontend/CompilerInvocation.cpp

4442

This function doesn't exist on Windows.

[appcs]

clang/lib/Frontend/CompilerInvocation.cpp

4442

Thank you; I am re-spinning with std::get_time().

[appcs]

• Re-spun with std::get_time().
• Expanded regression test with multiple error scenarios.

[appcs]

In D122517#3412464, @appcs wrote:

In D122517#3409266, @MaskRay wrote:

Bazel uses -Wno-builtin-macro-redefined -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted".
For reproducibility purposes, this is sufficient. I have two questions:

• Do we need the arbitrary time feature?

If you are saying that defining DATE, TIMESTAMP, TIME to an arbitrary time value, then that is the definition of the problem that we are trying to solve; else, please clarify. Thanks.

• Should an arbitrary string like "redacted" be supported?

Via -ffixed-date-time the value would have to be formatted e.g. 1969-12-31T23:59:59 (RFC3339 without fractional seconds and timezone); else, please clarify. Thanks.

Please ignore. Thanks.

[appcs]

In D122517#3409248, @appcs wrote:

(From D121040) Please let me know about the following and I will incorporate your feedback and submit subsequent patches for addressing these questions:

a. Disallow the simultaneous use of -Wdate-time/-Wpch-date-time and -ffixed-date-time=<timestamp>?
b. Disallow -ffixed-date-time=<timestamp>:

0. if PCH's are used
1. in PCH compilation

Also, (just as -Wdate-time warns on TIMESTAMP use), do we (in a separate patch), extend the scope of -Wpch-date-time to cover the use of TIMESTAMP is in a PCH?

Thank you.

Please ignore. Thanks.

[appcs]

In D122517#3409266, @MaskRay wrote:

Bazel uses -Wno-builtin-macro-redefined -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted".
For reproducibility purposes, this is sufficient. I have two questions:

• Do we need the arbitrary time feature?
• Should an arbitrary string like "redacted" be supported?

(After a fair amount of reading, and contemplation), I think I understand. Basically, Clang takes the (rightfully correct) stand that the very presence of the DATE, TIME and TIMESTAMP macros in the source makes the resulting binary irreproducible. Period.
Please could someone kindly confirm if I have understood correctly. Thank you.

[MaskRay]

In D122517#3423851, @appcs wrote:

In D122517#3409266, @MaskRay wrote:

Bazel uses -Wno-builtin-macro-redefined -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted".
For reproducibility purposes, this is sufficient. I have two questions:

• Do we need the arbitrary time feature?
• Should an arbitrary string like "redacted" be supported?

(After a fair amount of reading, and contemplation), I think I understand. Basically, Clang takes the (rightfully correct) stand that the very presence of the DATE, TIME and TIMESTAMP macros in the source makes the resulting binary irreproducible. Period.
Please could someone kindly confirm if I have understood correctly. Thank you.

Yeah, my question is whether we need the ability to set time to an arbitrary point and adjust the output of the three macros. Is it needed by a user?
For reproducibility purposes, the user can just set the macros to an arbitrary string, in Bazel's case, "redacted". The user doesn't care about the format with "redacted".

[MaskRay]

Created D135045 [Frontend] Recognize environment variable SOURCE_DATE_EPOCH. I think that's more likely what users want to use.

clang/include/clang/Driver/Options.td

def finstrument_function_entry_bare : Flag<["-"], "finstrument-function-entry-bare">, Group<f_Group>, Flags<[CC1Option]>,
   MarshallingInfoFlag<CodeGenOpts<"InstrumentFunctionEntryBare">>;
 def fcf_protection_EQ : Joined<["-"], "fcf-protection=">, Flags<[CoreOption, CC1Option]>, Group<f_Group>,
   HelpText<"Instrument control-flow architecture protection">, Values<"return,branch,full,none">;
 def fcf_protection : Flag<["-"], "fcf-protection">, Group<f_Group>, Flags<[CoreOption, CC1Option]>,
   Alias<fcf_protection_EQ>, AliasArgs<["full"]>,
   HelpText<"Enable cf-protection in 'full' mode">;
 def mibt_seal : Flag<["-"], "mibt-seal">, Group<m_Group>, Flags<[CoreOption, CC1Option]>,
   HelpText<"Optimize fcf-protection=branch/full (requires LTO).">;
+// Note: No MarshallingInfo for ffixed_date_time_EQ as the value needs to be
+// manually validated to be conforming to RFC3339 (without fractional seconds
+// and timezone), although, it is stored as an std::string
+def ffixed_date_time_EQ : Joined<["-"], "ffixed-date-time=">, Group<f_Group>,
+  Flags<[CC1Option, CoreOption]>,
+  HelpText<"Set a fixed value for __DATE__, __TIME__, __TIMESTAMP__, formatted as 1969-12-31T23:59:59 (RFC3339 without fractional seconds and timezone)">;
 
 defm xray_instrument : BoolFOption<"xray-instrument",
   LangOpts<"XRayInstrument">, DefaultFalse,
   PosFlag<SetTrue, [CC1Option], "Generate XRay instrumentation sleds on function entry and exit">,
   NegFlag<SetFalse>>;
 
 def fxray_instruction_threshold_EQ :
   JoinedOrSeparate<["-"], "fxray-instruction-threshold=">,

clang/include/clang/Lex/PreprocessorOptions.h

 //===- PreprocessorOptions.h ------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_CLANG_LEX_PREPROCESSOROPTIONS_H_
 #define LLVM_CLANG_LEX_PREPROCESSOROPTIONS_H_
 
 #include "clang/Basic/BitmaskEnum.h"
 #include "clang/Basic/LLVM.h"
 #include "clang/Lex/PreprocessorExcludedConditionalDirectiveSkipMapping.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/ADT/StringSet.h"
+#include <ctime>
 #include <functional>
 #include <map>
 #include <memory>
 #include <set>
 #include <string>
 #include <utility>
 #include <vector>
 
ExcludedPreprocessorDirectiveSkipMapping
       *ExcludedConditionalDirectiveSkipMappings = nullptr;
 
   /// Set up preprocessor for RunAnalysis action.
   bool SetUpStaticAnalyzer = false;
 
   /// Prevents intended crashes when using #pragma clang __debug. For testing.
   bool DisablePragmaDebugCrash = false;
 
+  /// If set, a fixed time that __DATE__, __TIME__, __TIMESTAMP__ expand to.
+  /// Note: Manually validated, not automatically marshalled.
+  std::string FixedDateTime;
+
 public:
   PreprocessorOptions() : PrecompiledPreambleBytes(0, false) {}
 
   void addMacroDef(StringRef Name) {
     Macros.emplace_back(std::string(Name), false);
   }
   void addMacroUndef(StringRef Name) {
     Macros.emplace_back(std::string(Name), true);

clang/lib/Driver/ToolChains/Clang.cpp

   if (Arg *A = Args.getLastArg(options::OPT_traditional,
                                options::OPT_traditional_cpp)) {
     if (isa<PreprocessJobAction>(JA))
       CmdArgs.push_back("-traditional-cpp");
     else
       D.Diag(diag::err_drv_clang_unsupported) << A->getAsString(Args);
   }
 
+  if (Arg *A = Args.getLastArg(options::OPT_ffixed_date_time_EQ)) {
+    StringRef DateTime = A->getValue();
+    CmdArgs.push_back(Args.MakeArgString("-ffixed-date-time=" + DateTime));
+  }
+
   Args.AddLastArg(CmdArgs, options::OPT_dM);
   Args.AddLastArg(CmdArgs, options::OPT_dD);
   Args.AddLastArg(CmdArgs, options::OPT_dI);
 
   Args.AddLastArg(CmdArgs, options::OPT_fmax_tokens_EQ);
 
   // Handle serialized diagnostics.
   if (Arg *A = Args.getLastArg(options::OPT__serialize_diags)) {

clang/lib/Frontend/CompilerInvocation.cpp

 #include "llvm/Support/VirtualFileSystem.h"
 #include "llvm/Support/raw_ostream.h"
 #include "llvm/Target/TargetOptions.h"
 #include <algorithm>
 #include <atomic>
 #include <cassert>
 #include <cstddef>
 #include <cstring>
+#include <ctime>
+#include <iomanip>
+#include <locale>
 #include <memory>
+#include <sstream>
 #include <string>
 #include <tuple>
 #include <type_traits>
 #include <utility>
 #include <vector>
 
 using namespace clang;
 using namespace driver;
#undef PREPROCESSOR_OPTION_WITH_MARSHALLING
   }
 
   for (const auto &CI : Opts.ChainedIncludes)
     GenerateArg(Args, OPT_chain_include, CI, SA);
 
   for (const auto &RF : Opts.RemappedFiles)
     GenerateArg(Args, OPT_remap_file, RF.first + ";" + RF.second, SA);
 
+  // We manually forward -ffixed-date-time=<value> even though we keep <value>
+  // in std::string form (which could be automatically marshalled), as we need
+  // to ascertain that <value> conforms to the RFC3339 format (without
+  // fractional seconds and timezone).
+  if (!Opts.FixedDateTime.empty())
+    GenerateArg(Args, OPT_ffixed_date_time_EQ, Twine(Opts.FixedDateTime), SA);
+
   // Don't handle LexEditorPlaceholders. It is implied by the action that is
   // generated elsewhere.
 }
 
 static bool ParsePreprocessorArgs(PreprocessorOptions &Opts, ArgList &Args,
                                   DiagnosticsEngine &Diags,
                                   frontend::ActionKind Action,
                                   const FrontendOptions &FrontendOpts) {
#undef PREPROCESSOR_OPTION_WITH_MARSHALLING
   }
 
   // Always avoid lexing editor placeholders when we're just running the
   // preprocessor as we never want to emit the
   // "editor placeholder in source file" error in PP only mode.
   if (isStrictlyPreprocessorAction(Action))
     Opts.LexEditorPlaceholders = false;
 
+  // Validate -ffixed-date-time=<value>
+  if (Arg *A = Args.getLastArg(OPT_ffixed_date_time_EQ)) {
+    StringRef DateTime = A->getValue();
+    std::tm FixedDateTime = {};
+    // RFC3339Format: RFC3339 format without fractional seconds and timezone.
+    const char *RFC3339Format = "%Y-%m-%dT%T";
+    // We use std::get_time() only to validate if the DateTime value conforms to
+    // RFC3339Format, reporting an error here itself, if not; if validation is
+    // successful, we pass it forward as-is to the preprocessor for possible
+    // conversion into std::tm later on, when needed.
+    std::istringstream ss(DateTime.str());
+    ss >> std::get_time(&FixedDateTime, RFC3339Format);
+    if (ss.fail() || !ss.eof())
+      Diags.Report(diag::err_drv_invalid_value)
+          << A->getAsString(Args) << DateTime;
+    else
+      Opts.FixedDateTime = DateTime.str();
+  }
+
   return Diags.getNumErrors() == NumErrorsBefore;
 }
 
 static void GeneratePreprocessorOutputArgs(
     const PreprocessorOutputOptions &Opts, SmallVectorImpl<const char *> &Args,
     CompilerInvocation::StringAllocator SA, frontend::ActionKind Action) {
   const PreprocessorOutputOptions &PreprocessorOutputOpts = Opts;
 

clang/lib/Lex/PPMacroExpansion.cpp

 #include "llvm/Support/Format.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/raw_ostream.h"
 #include <algorithm>
 #include <cassert>
 #include <cstddef>
 #include <cstring>
 #include <ctime>
+#include <iomanip>
+#include <locale>
+#include <sstream>
 #include <string>
 #include <tuple>
 #include <utility>
 
 using namespace clang;
 
 MacroDirective *
 Preprocessor::getLocalMacroDirectiveHistory(const IdentifierInfo *II) const {
void Preprocessor::removeCachedMacroExpandedTokensOfLastLexer() {
   assert(!MacroExpandingLexersStack.empty());
   size_t tokIndex = MacroExpandingLexersStack.back().second;
   assert(tokIndex < MacroExpandedTokens.size());
   // Pop the cached macro expanded tokens from the end.
   MacroExpandedTokens.resize(tokIndex);
   MacroExpandingLexersStack.pop_back();
 }
 
+/// MakeTime - Convert the value in FDTV (assumed to be valid) into a std::tm
+/// Make no assumption about daylight saving time.
+static void MakeTime(const std::string &FDTV, std::tm &FDT) {
+  // RFC3339Format: RFC3339 Format without fractional seconds and timezone.
+  const char *RFC3339Format = "%Y-%m-%dT%T";
+  std::istringstream ss(FDTV);
+  ss >> std::get_time(&FDT, RFC3339Format);
+  // FDTV should already be validated as being well-formed; but, ascertain that.
+  assert(!ss.fail() && ss.eof() && "DateTime invalid value!");
+  // Per ctime(3), tm.tm_isdst "indicates whether daylight saving time is in
+  // effect ... and is negative if the information is not available."
+  FDT.tm_isdst = -1;
+  // Per mktime(3), (for tm.tm_isdst) "a negative value means that mktime()
+  // should (use timezone information and system  databases to) attempt to
+  // determine whether DST is in effect at the specified time." Hence, we
+  // call mktime(3): "Since the return value of mktime() is ignored here, its
+  // dependence on the local timezone should not matter. The main effect of this
+  // call is to fill in tm_isdst flag with the best guess. This may result in
+  // unexpected results if fixed-date-time is set inside the "spring forward"
+  // hour for the local timezone, but this issue is inherent in the ISO 8601
+  // format." (Note: The foregoing comment is from one of the reviewers of
+  // the original D23934 work, and is included because it serves to clarify).
+  mktime(&FDT);
+}
+
 /// ComputeDATE_TIME - Compute the current time, enter it into the specified
 /// scratch buffer, then return DATELoc/TIMELoc locations with the position of
 /// the identifier tokens inserted.
 static void ComputeDATE_TIME(SourceLocation &DATELoc, SourceLocation &TIMELoc,
                              Preprocessor &PP) {
-  time_t TT = time(nullptr);
-  struct tm *TM = localtime(&TT);
+  PreprocessorOptions &PO = PP.getPreprocessorOpts();
+  std::tm FDT = {};
+  std::tm *TM = &FDT;
+  if (!PO.FixedDateTime.empty())
+    MakeTime(PO.FixedDateTime, FDT);
+  else {
+    std::time_t TT = std::time(nullptr);
+    TM = std::localtime(&TT);
+  }
 
   static const char * const Months[] = {
     "Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec"
   };
 
   {
     SmallString<32> TmpBuffer;
     llvm::raw_svector_ostream TmpStream(TmpBuffer);
if (II == Ident__LINE__) {
     // __INCLUDE_LEVEL__ expands to a simple numeric value.
     OS << Depth;
     Tok.setKind(tok::numeric_constant);
   } else if (II == Ident__TIMESTAMP__) {
     Diag(Tok.getLocation(), diag::warn_pp_date_time);
     // MSVC, ICC, GCC, VisualAge C++ extension.  The generated string should be
     // of the form "Ddd Mmm dd hh::mm::ss yyyy", which is returned by asctime.
 
+    const char *Result;
+    PreprocessorOptions &PO = getPreprocessorOpts();
+    if (!PO.FixedDateTime.empty()) {
+      std::tm FDT;
+      std::tm *TM = &FDT;
+      MakeTime(PO.FixedDateTime, FDT);
+      Result = asctime(TM);
+    } else {
       // Get the file that we are lexing out of.  If we're currently lexing from
       // a macro, dig into the include stack.
       const FileEntry *CurFile = nullptr;
       PreprocessorLexer *TheLexer = getCurrentFileLexer();
 
       if (TheLexer)
         CurFile = SourceMgr.getFileEntryForID(TheLexer->getFileID());
 
-    const char *Result;
       if (CurFile) {
-      time_t TT = CurFile->getModificationTime();
-      struct tm *TM = localtime(&TT);
+        std::time_t TT = CurFile->getModificationTime();
+        std::tm *TM = std::localtime(&TT);
         Result = asctime(TM);
       } else {
         Result = "??? ??? ?? ??:??:?? ????\n";
       }
+    }
     // Surround the string with " and strip the trailing newline.
     OS << '"' << StringRef(Result).drop_back() << '"';
     Tok.setKind(tok::string_literal);
   } else if (II == Ident__FLT_EVAL_METHOD__) {
     // __FLT_EVAL_METHOD__ is set to the default value.
     if (getTUFPEvalMethod() ==
         LangOptions::FPEvalMethodKind::FEM_Indeterminable) {
       // This is possible if `AllowFPReassoc` or `AllowReciprocal` is enabled.

clang/test/Preprocessor/fixed-date-time.cpp

+// RUN: %clang_cc1 -ffixed-date-time=2008-09-02T14:30:27 -std=c++11 %s -verify
+// RUN: not %clang_cc1 -ffixed-date-time=2008-09-02T14:30:27asdf -std=c++11 %s 2>&1 | FileCheck %s --check-prefix=CHECK-SFX_ERR
+// RUN: not %clang_cc1 -ffixed-date-time=qwer2008-09-02T14:30:27 -std=c++11 %s 2>&1 | FileCheck %s --check-prefix=CHECK-PFX_ERR
+// RUN: not %clang_cc1 -ffixed-date-time=qwer2008-09-02T14:30:27asdf -std=c++11 %s 2>&1 | FileCheck %s --check-prefix=CHECK-CMB_ERR
+// RUN: not %clang_cc1 -ffixed-date-time=2008-09-02T14:30:2X -std=c++11 %s 2>&1 | FileCheck %s --check-prefix=CHECK-SEC_ERR
+// RUN: not %clang_cc1 -ffixed-date-time=2008-09-02T14:?0:27 -std=c++11 %s 2>&1 | FileCheck %s --check-prefix=CHECK-MIN_ERR
+// RUN: not %clang_cc1 -ffixed-date-time=2008-09-02T^:30:27 -std=c++11 %s 2>&1 | FileCheck %s --check-prefix=CHECK-HOU_ERR
+// RUN: not %clang_cc1 -ffixed-date-time=2008-09-LT14:30:27 -std=c++11 %s 2>&1 | FileCheck %s --check-prefix=CHECK-DAY_ERR
+// RUN: not %clang_cc1 -ffixed-date-time=2008-+-02T14:30:27 -std=c++11 %s 2>&1 | FileCheck %s --check-prefix=CHECK-MON_ERR
+// RUN: not %clang_cc1 -ffixed-date-time=-09-02T14:30:27 -std=c++11 %s 2>&1 | FileCheck %s --check-prefix=CHECK-YEA_ERR
+// RUN: not %clang_cc1 -ffixed-date-time=2008:09:02T14-30-27 -std=c++11 %s 2>&1 | FileCheck %s --check-prefix=CHECK-SWP_ERR
+// RUN: not %clang_cc1 -ffixed-date-time=junk -std=c++11 %s 2>&1 | FileCheck %s --check-prefix=CHECK-JNK_ERR
+// RUN: not %clang_cc1 -ffixed-date-time="2008 09 02T14 30 27" -std=c++11 %s 2>&1 | FileCheck %s --check-prefix=CHECK-BSP_ERR
+// RUN: not %clang_cc1 -ffixed-date-time=02-09-2008T14:30:27 -std=c++11 %s 2>&1 | FileCheck %s --check-prefix=CHECK-JMB_ERR
+// RUN: not %clang_cc1 -ffixed-date-time= -std=c++11 %s 2>&1 | FileCheck %s --check-prefix=CHECK-MIS_ERR
+// expected-no-diagnostics
+
+// CHECK-SFX_ERR: error: invalid value '2008-09-02T14:30:27asdf' in '-ffixed-date-time=2008-09-02T14:30:27asdf'
+// CHECK-PFX_ERR: error: invalid value 'qwer2008-09-02T14:30:27' in '-ffixed-date-time=qwer2008-09-02T14:30:27'
+// CHECK-CMB_ERR: error: invalid value 'qwer2008-09-02T14:30:27asdf' in '-ffixed-date-time=qwer2008-09-02T14:30:27asdf'
+// CHECK-SEC_ERR: error: invalid value '2008-09-02T14:30:2X' in '-ffixed-date-time=2008-09-02T14:30:2X'
+// CHECK-MIN_ERR: error: invalid value '2008-09-02T14:?0:27' in '-ffixed-date-time=2008-09-02T14:?0:27'
+// CHECK-HOU_ERR: error: invalid value '2008-09-02T^:30:27' in '-ffixed-date-time=2008-09-02T^:30:27'
+// CHECK-DAY_ERR: error: invalid value '2008-09-LT14:30:27' in '-ffixed-date-time=2008-09-LT14:30:27'
+// CHECK-MON_ERR: error: invalid value '2008-+-02T14:30:27' in '-ffixed-date-time=2008-+-02T14:30:27'
+// CHECK-YEA_ERR: error: invalid value '-09-02T14:30:27' in '-ffixed-date-time=-09-02T14:30:27'
+// CHECK-SWP_ERR: error: invalid value '2008:09:02T14-30-27' in '-ffixed-date-time=2008:09:02T14-30-27'
+// CHECK-JNK_ERR: error: invalid value 'junk' in '-ffixed-date-time=junk'
+// CHECK-BSP_ERR: error: invalid value '2008 09 02T14 30 27' in '-ffixed-date-time=2008 09 02T14 30 27'
+// CHECK-JMB_ERR: error: invalid value '02-09-2008T14:30:27' in '-ffixed-date-time=02-09-2008T14:30:27'
+// CHECK-MIS_ERR: error: invalid value '' in '-ffixed-date-time='
+
+constexpr int constexpr_strcmp(const char *p, const char *q) {
+  return *p != *q ? *p - *q : !*p ? 0
+                                  : constexpr_strcmp(p + 1, q + 1);
+}
+
+static_assert(!constexpr_strcmp(__DATE__, "Sep  2 2008"), "");
+static_assert(!constexpr_strcmp(__TIME__, "14:30:27"), "");
+static_assert(!constexpr_strcmp(__TIMESTAMP__, "Tue Sep  2 14:30:27 2008"), "");