[-Wunsafe-buffer-usage] Add unsafe buffer checking opt-out pragmas

[ziqingluo-90]

Summary

Add a pair of clang pragmas:
#pragma clang unsafe_buffer_usage begin
#pragma clang unsafe_buffer_usage end,
which specify the start and end of an (unsafe buffer checking) opt-out region.

Behaviors of opt-out regions conform to the following rules:

1. No nested nor overlapped opt-out regions are allowed. One cannot start an opt-out region with ... unsafe_buffer_usage begin but never close it with ... unsafe_buffer_usage end. Mis-use of the pragmas will be warned.
2. Warnings raised from unsafe buffer operations inside such an opt-out region will always be suppressed. This behavior CANNOT be changed by clang diagnostic pragmas or command-line flags.
3. Warnings raised from unsafe operations outside of such opt-out regions may be reported on declarations inside opt-out regions. These warnings are NOT suppressed.
4. An un-suppressed unsafe operation warning may be attached with notes. These notes are NOT suppressed as well regardless of whether they are in opt-out regions.

The implementation maintains a separate sequence of location pairs representing opt-out regions in Preprocessor.
The UnsafeBufferUsage analyzer reads the region sequence to check if an unsafe operation is in an opt-out region. If it is, discard the warning raised from the operation immediately.

Examples,

void f() {
  int * p = new int [10];
#pragma clang unsafe_buffer_usage begin
  p[5];
#pragma clang unsafe_buffer_usage end
}

Nothing will be warned in the code above since the only unsafe operation p[5] is in an opt-out region.

void f() {
#pragma clang unsafe_buffer_usage begin
   int * p = new int [10];  // expect a warning on `p` for the unsafe operation below
#pragma clang unsafe_buffer_usage end
   p[5]; // may have a note here which is an attachment of the warning above
}

In the example above, a warning triggered by p[5], which is not in an opt-out region, will be reported on the declaration. Although the declaration is in an opt-out region, the warning is NOT suppressed.

[NoQ]

clang/include/clang/Basic/DiagnosticLexKinds.td

949

IIUC ExtWarn means it's a warning of the form "warning: XXX is a language extension". It's not just a warning that has something to do with language extensions, it's a warning that tries to warn the user about the very fact that it's an extension. So we should just use Warning or Error.

Also about wording: we already have a somewhat similar #pragma assume_nonnull, can we use similar text? And probably prefer Error because that's what the other pragma uses too:

def err_pp_assume_nonnull_syntax : Error<"expected 'begin' or 'end'">;
def err_pp_double_begin_of_assume_nonnull : Error<
  "already inside '#pragma clang assume_nonnull'">;
def err_pp_unmatched_end_of_assume_nonnull : Error<
  "not currently inside '#pragma clang assume_nonnull'">;
def err_pp_include_in_assume_nonnull : Error<
  "cannot %select{#include files|import headers}0 "
  "inside '#pragma clang assume_nonnull'">;
def err_pp_eof_in_assume_nonnull : Error<
  "'#pragma clang assume_nonnull' was not ended within this file">;

clang/lib/Sema/AnalysisBasedWarnings.cpp

2209

I believe this check should be performed *much earlier*. It's not about how we display unsafe usages to the user; we can exclude variables from analysis entirely when all their unsafe uses are guarded by the pragma. I suspect that we can drop these gadgets as early as in findGadgets() phase (assuming that D140062 causes us to never rely on unsafe gadgets for fixits).

[ziqingluo-90]

Fixing bugs in my test.

[jkorous]

clang/lib/Sema/AnalysisBasedWarnings.cpp

2209

Let's addres this in a follow-up patch.

[ziqingluo-90]

Addressing comments.

[ziqingluo-90]

clang/include/clang/Basic/DiagnosticLexKinds.td

949

Fixed!

clang/lib/Sema/AnalysisBasedWarnings.cpp

2209

I agree to both of you. Moving the check into UnsafeBufferUsage.cpp may depends on this https://reviews.llvm.org/D140062 patch. So I will make it a follow-up patch.

[ziqingluo-90]

Rebase the patch.

Move the check of whether a node is in an opt-out region to an earlier stage---the AST matching stage.

[NoQ]

clang/include/clang/Basic/Diagnostic.h

1040–1043 ↗

(On Diff #489136)

Ok, now I no longer see why this data should live in DiagnosticEngine. It's mostly about analysis, right? The pragma simply makes our analysis produce different results, regardless of whether these results are used for producing diagnostics or something else. Maybe let's keep it all in Preprocessor?

clang/lib/Analysis/UnsafeBufferUsage.cpp

621

This prevents safe fixable gadgets from being found in the opt-out zone. I think this clause should only apply to warning gadgets.

626–627

I think we should match all DeclStmts as well, because otherwise we may be unable to find the variable to fix.

[ziqingluo-90]

clang/include/clang/Basic/Diagnostic.h

1040–1043 ↗

(On Diff #489136)

make sense to me!

clang/lib/Analysis/UnsafeBufferUsage.cpp

621

You are right! Fixables should be found regardless of whether they are in an opt-out zone. A Fixable could later be immediately discarded once we know that the variable declaration associated to the Fixable is in an opt-out zone.

626–627

In case we are unable to find the variable to fix, it means that the variable declaration is in an opt-out zone. So we don't fix the variable anyway, right?

Or do you mean that a variable may still get fixed even if its declaration is in an opt-out zone? I could imagine it is possible if the variable is involved in some assignments that we want to fix.

[NoQ]

clang/lib/Analysis/UnsafeBufferUsage.cpp

626–627

do you mean that a variable may still get fixed even if its declaration is in an opt-out zone?

Yes I think that's the case. We do have tests about this right?:

#pragma clang unsafe_buffer_usage begin
  ...
  int *p3 = new int[10]; // expected-warning{{'p3' is an unsafe pointer used for buffer access}}

#pragma clang unsafe_buffer_usage end
  ...
  p3[5]; //expected-note{{used in buffer access here}}

And I think it's safe to assume that every time we emit a warning, we also want to emit a fixit.

If only we had any fixits implemented, this code would have been much easier to refactor because we'd have some actual tests covering it 🤔

[ziqingluo-90]

clang/lib/Analysis/UnsafeBufferUsage.cpp

621

Oh wait, scratch what I said above.

FixableGadgets should be found regardless of whether they are in an opt-out zone. They will be attached to variable declarations. The emission of an Unsafe Buffer diagnostic of a variable declaration only depends on WarningGadgets. So FixableGadgets have nothing to do with opt-out regions.

[ziqingluo-90]

clang/lib/Analysis/UnsafeBufferUsage.cpp

626–627

sorry I didn't think it through when I reply the comments. You are right.
Fixables and variable declarations should be irrelevant to opt-out regions.

I'll rebase this patch w.r.t. https://reviews.llvm.org/D139737 so that we will have tests covering it.

[ziqingluo-90]

Put the implementation of the opt-out pragma completely in Preprocessor.
Add an opt-out pragma test for fix-its.

[NoQ]

Ok I think I'm happy with the patch! Let's give other folks a few days to comment and then land?

The commit message will need to be updated to remove mentions of DiagnosticsEngine.

I'm somewhat surprised that there is no centralized documentation for pragmas, like there is for attributes. But I hope we'll provide enough documentation as part of D136811, which would also supply release notes.

---

It's worth saying out loud that this patch doesn't implement the opt-in pragma (#pragma only_safe_buffers begin ... end) as proposed in D136811, but only the opt-out pragma. We're actually downprioritizing it (possibly dropping entirely) because we've discovered a complete lack of symmetry between these two pragmas. While the opt-out pragma is straightforward, the opt-in pragma is very hard to get right, as it implies the possibility to turn on compiler warnings that were disabled by all other means. Moreover, because warnings enabled this way don't necessarily live between begin and end, this makes it tricky to avoid running unsafe buffer usage analysis on the entire translation unit even when the warning is completely disabled and there are no opt-in pragmas anywhere in the translation unit. So there are a lot of potential scary interactions that we'll have to address before we consider implementing the opt-in pragma. They probably can be addressed, but it'll take us some time to come up with a good design.

[ziqingluo-90]

Change the fix-it test style

clang/include/clang/Analysis/Analyses/UnsafeBufferUsage.h

public:
   /// Returns the text indicating that the user needs to provide input there:
   virtual std::string
   getUserFillPlaceHolder(StringRef HintTextToUser = "placeholder") {
     std::string s = std::string("<# ");
     s += HintTextToUser;
     s += " #>";
     return s;
   }
+
+  /// Returns a reference to the `Preprocessor`:
+  virtual const Preprocessor & getPP() const;
 };
 
 // This function invokes the analysis and allows the caller to react to it
 // through the handler class.
 void checkUnsafeBufferUsage(const Decl *D, UnsafeBufferUsageHandler &Handler);
 
 namespace internal {
 // Tests if any two `FixItHint`s in `FixIts` conflict.  Two `FixItHint`s
 // conflict if they have overlapping source ranges.
 bool anyConflict(const llvm::SmallVectorImpl<FixItHint> &FixIts,
                  const SourceManager &SM);
 } // namespace internal
 } // end namespace clang
 
 #endif /* LLVM_CLANG_ANALYSIS_ANALYSES_UNSAFEBUFFERUSAGE_H */

clang/include/clang/Basic/DiagnosticLexKinds.td

 
 def err_dep_source_scanner_missing_semi_after_at_import : Error<
   "could not find ';' after @import">;
 def err_dep_source_scanner_unexpected_tokens_at_import : Error<
   "unexpected extra tokens at end of @import declaration">;
 
 }
 
+def err_pp_double_begin_pragma_unsafe_buffer_usage :
+Error<"already inside '#pragma unsafe_buffer_usage'">;
+
+def err_pp_unmatched_end_begin_pragma_unsafe_buffer_usage :
+Error<"not currently inside '#pragma unsafe_buffer_usage'">;
+
+def err_pp_unclosed_pragma_unsafe_buffer_usage :
+Error<"'#pragma unsafe_buffer_usage' was not ended">;
+
+def err_pp_pragma_unsafe_buffer_usage_syntax :
+Error<"Expected 'begin' or 'end'">;
 }

clang/include/clang/Lex/Preprocessor.h

public:
   static void processPathForFileMacro(SmallVectorImpl<char> &Path,
                                       const LangOptions &LangOpts,
                                       const TargetInfo &TI);
 
 private:
   void emitMacroDeprecationWarning(const Token &Identifier) const;
   void emitRestrictExpansionWarning(const Token &Identifier) const;
   void emitFinalMacroWarning(const Token &Identifier, bool IsUndef) const;
+
+  /// This boolean state keeps track if the current scanned token (by this PP)
+  /// is in an "-Wunsafe-buffer-usage" opt-out region. Assuming PP scans a
+  /// translation unit in a linear order.
+  bool InSafeBufferOptOutRegion = 0;
+
+  /// Hold the start location of the current "-Wunsafe-buffer-usage" opt-out
+  /// region if PP is currently in such a region.  Hold undefined value
+  /// otherwise.
+  SourceLocation CurrentSafeBufferOptOutStart; // It is used to report the start location of an never-closed region.
+
+  // An ordered sequence of "-Wunsafe-buffer-usage" opt-out regions in one
+  // translation unit. Each region is represented by a pair of start and end
+  // locations.  A region is "open" if its' start and end locations are
+  // identical.
+  SmallVector<std::pair<SourceLocation, SourceLocation>, 8> SafeBufferOptOutMap;
+
+public:
+  /// \return true iff the given `Loc` is in a "-Wunsafe-buffer-usage" opt-out
+  /// region.  This `Loc` must be a source location that has been pre-processed.
+  bool isSafeBufferOptOut(const SourceManager&SourceMgr, const SourceLocation &Loc) const;
+
+  /// Alter the state of whether this PP currently is in a
+  /// "-Wunsafe-buffer-usage" opt-out region.
+  ///
+  /// \param isEnter: true if this PP is entering a region; otherwise, this PP
+  /// is exiting a region
+  /// \param Loc: the location of the entry or exit of a
+  /// region
+  /// \return true iff it is INVALID to enter or exit a region, i.e.,
+  /// attempt to enter a region before exiting a previous region, or exiting a
+  /// region that PP is not currently in.
+  bool enterOrExitSafeBufferOptOutRegion(bool isEnter,
+                                         const SourceLocation &Loc);
+
+  /// \return true iff this PP is currently in a "-Wunsafe-buffer-usage"
+  ///          opt-out region
+  bool isPPInSafeBufferOptOutRegion();
+
+  /// \param StartLoc: output argument. It will be set to the start location of
+  /// the current "-Wunsafe-buffer-usage" opt-out region iff this function
+  /// returns true.
+  /// \return true iff this PP is currently in a "-Wunsafe-buffer-usage"
+  ///          opt-out region
+  bool isPPInSafeBufferOptOutRegion(SourceLocation &StartLoc);
 };
 
 /// Abstract base class that describes a handler that will receive
 /// source ranges for each of the comments encountered in the source file.
 class CommentHandler {
 public:
   virtual ~CommentHandler();
 

clang/lib/Analysis/UnsafeBufferUsage.cpp

 //===- UnsafeBufferUsage.cpp - Replace pointers with modern C++ -----------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "clang/Analysis/Analyses/UnsafeBufferUsage.h"
 #include "clang/AST/RecursiveASTVisitor.h"
 #include "clang/ASTMatchers/ASTMatchFinder.h"
+#include "clang/Lex/Preprocessor.h"
 #include "clang/Lex/Lexer.h"
 #include "llvm/ADT/SmallVector.h"
 #include <memory>
 #include <optional>
 
 using namespace llvm;
 using namespace clang;
 using namespace ast_matchers;

 
 AST_MATCHER_P(Stmt, forEveryDescendant, internal::Matcher<Stmt>, innerMatcher) {
   const DynTypedMatcher &DTM = static_cast<DynTypedMatcher>(innerMatcher);
   
   MatchDescendantVisitor Visitor(&DTM, Finder, Builder, ASTMatchFinder::BK_All);  
   return Visitor.findMatch(DynTypedNode::create(Node));
 }
 
+// Matches a `Stmt` node iff the node is in a safe-buffer opt-out region
+AST_MATCHER_P(Stmt, notInSafeBufferOptOut, const Preprocessor *, PP) {
+  const SourceManager &SM = Finder->getASTContext().getSourceManager();
+  return !PP->isSafeBufferOptOut(SM, Node.getBeginLoc());
+}
+
 AST_MATCHER_P(CastExpr, castSubExpr, internal::Matcher<Expr>, innerMatcher) {
   return innerMatcher.matches(*Node.getSubExpr(), Finder, Builder);
 }
 
 // Returns a matcher that matches any expression 'e' such that `innerMatcher`
 // matches 'e' and 'e' is in an Unspecified Lvalue Context.
 static internal::Matcher<Stmt>
 isInUnspecifiedLvalueContext(internal::Matcher<Expr> innerMatcher) {
if (I == Map.end())
       return Kind::Wontfix;
 
     return I->second;
   }
 };
 } // namespace
 
 /// Scan the function and return a list of gadgets found with provided kits.
-static std::tuple<FixableGadgetList, WarningGadgetList, DeclUseTracker> findGadgets(const Decl *D) {
+static std::tuple<FixableGadgetList, WarningGadgetList, DeclUseTracker>
+findGadgets(const Decl *D, const Preprocessor &PP) {
 
   struct GadgetFinderCallback : MatchFinder::MatchCallback {
     FixableGadgetList FixableGadgets;
     WarningGadgetList WarningGadgets;
     DeclUseTracker Tracker;
 
     void run(const MatchFinder::MatchResult &Result) override {
       // In debug mode, assert that we've found exactly one gadget.
stmt(forEveryDescendant(
       eachOf(
       // A `FixableGadget` matcher and a `WarningGadget` matcher should not disable
       // each other (they could if they were put in the same `anyOf` group).
       // We also should make sure no two `FixableGadget` (resp. `WarningGadget`) matchers
       // match for the same node, so that we can group them
       // in one `anyOf` group (for better performance via short-circuiting).
       stmt(anyOf(
 #define FIXABLE_GADGET(x)                                                              \
         x ## Gadget::matcher().bind(#x),
 #include "clang/Analysis/Analyses/UnsafeBufferUsageGadgets.def"
         // Also match DeclStmts because we'll need them when fixing
         // their underlying VarDecls that otherwise don't have
         // any backreferences to DeclStmts.
         declStmt().bind("any_ds")
       )),
       stmt(anyOf(
         // Add Gadget::matcher() for every gadget in the registry.
 #define WARNING_GADGET(x)                                                              \
-        x ## Gadget::matcher().bind(#x),
+        allOf(x ## Gadget::matcher().bind(#x), notInSafeBufferOptOut(&PP)),
 #include "clang/Analysis/Analyses/UnsafeBufferUsageGadgets.def"
         // In parallel, match all DeclRefExprs so that to find out
         // whether there are any uncovered by gadgets.
         declRefExpr(anyOf(hasPointerType(), hasArrayType()), to(varDecl())).bind("any_dre")
       )))
     )),
     &CB
   );
void clang::checkUnsafeBufferUsage(const Decl *D,
                                    UnsafeBufferUsageHandler &Handler) {
   assert(D && D->getBody());
 
   WarningGadgetSets UnsafeOps;
   FixableGadgetSets FixablesForUnsafeVars;
   DeclUseTracker Tracker;
 
   {
-    auto [FixableGadgets, WarningGadgets, TrackerRes] = findGadgets(D);
+    auto [FixableGadgets, WarningGadgets, TrackerRes] = findGadgets(D, Handler.getPP());
     UnsafeOps = groupWarningGadgetsByVar(std::move(WarningGadgets));
     FixablesForUnsafeVars = groupFixablesByVar(std::move(FixableGadgets));
     Tracker = std::move(TrackerRes);
   }
 
   // Filter out non-local vars and vars with unclaimed DeclRefExpr-s.
   for (auto it = FixablesForUnsafeVars.byVar.cbegin();
        it != FixablesForUnsafeVars.byVar.cend();) {

clang/lib/Lex/PPLexerChange.cpp

 
 /// HandleEndOfFile - This callback is invoked when the lexer hits the end of
 /// the current file.  This either returns the EOF token or pops a level off
 /// the include stack and keeps going.
 bool Preprocessor::HandleEndOfFile(Token &Result, bool isEndOfMacro) {
   assert(!CurTokenLexer &&
          "Ending a file when currently in a macro!");
 
+  SourceLocation UnclosedSafeBufferOptOutLoc;
+
+  if (IncludeMacroStack.empty() &&
+      isPPInSafeBufferOptOutRegion(UnclosedSafeBufferOptOutLoc)) {
+    // To warn if a "-Wunsafe-buffer-usage" opt-out region is still open by the
+    // end of a file.
+    Diag(UnclosedSafeBufferOptOutLoc,
+         diag::err_pp_unclosed_pragma_unsafe_buffer_usage);
+  }
   // If we have an unclosed module region from a pragma at the end of a
   // module, complain and close it now.
   const bool LeavingSubmodule = CurLexer && CurLexerSubmodule;
   if ((LeavingSubmodule || IncludeMacroStack.empty()) &&
       !BuildingSubmoduleStack.empty() &&
       BuildingSubmoduleStack.back().IsPragma) {
     Diag(BuildingSubmoduleStack.back().ImportLoc,
          diag::err_pp_module_begin_without_module_end);

clang/lib/Lex/Pragma.cpp

static void DebugOverflowStack(void (*P)() = nullptr) {
     void (*volatile Self)(void(*P)()) = DebugOverflowStack;
     Self(reinterpret_cast<void(*)()>(Self));
   }
 #ifdef _MSC_VER
     #pragma warning(default : 4717)
 #endif
 };
 
+struct PragmaUnsafeBufferUsageHandler : public PragmaHandler {
+  PragmaUnsafeBufferUsageHandler() : PragmaHandler("unsafe_buffer_usage") {}
+  void HandlePragma(Preprocessor &PP, PragmaIntroducer Introducer,
+                    Token &FirstToken) override {
+    Token Tok;
+
+    PP.LexUnexpandedToken(Tok);
+    if (Tok.isNot(tok::identifier)) {
+      PP.Diag(Tok, diag::err_pp_pragma_unsafe_buffer_usage_syntax);
+      return;
+    }
+
+    IdentifierInfo *II = Tok.getIdentifierInfo();
+    SourceLocation Loc = Tok.getLocation();
+
+    if (II->isStr("begin")) {
+      if (PP.enterOrExitSafeBufferOptOutRegion(true, Loc))
+        PP.Diag(Loc, diag::err_pp_double_begin_pragma_unsafe_buffer_usage);
+    } else if (II->isStr("end")) {
+      if (PP.enterOrExitSafeBufferOptOutRegion(false, Loc))
+        PP.Diag(Loc, diag::err_pp_unmatched_end_begin_pragma_unsafe_buffer_usage);
+    } else
+      PP.Diag(Tok, diag::err_pp_pragma_unsafe_buffer_usage_syntax);
+  }
+};
+
 /// PragmaDiagnosticHandler - e.g. '\#pragma GCC diagnostic ignored "-Wformat"'
 struct PragmaDiagnosticHandler : public PragmaHandler {
 private:
   const char *Namespace;
 
 public:
   explicit PragmaDiagnosticHandler(const char *NS)
       : PragmaHandler("diagnostic"), Namespace(NS) {}
void Preprocessor::RegisterBuiltinPragmas() {
   auto *ModuleHandler = new PragmaNamespace("module");
   AddPragmaHandler("clang", ModuleHandler);
   ModuleHandler->AddPragma(new PragmaModuleImportHandler());
   ModuleHandler->AddPragma(new PragmaModuleBeginHandler());
   ModuleHandler->AddPragma(new PragmaModuleEndHandler());
   ModuleHandler->AddPragma(new PragmaModuleBuildHandler());
   ModuleHandler->AddPragma(new PragmaModuleLoadHandler());
 
+  // Safe Buffers pragmas
+  AddPragmaHandler("clang", new PragmaUnsafeBufferUsageHandler);
+
   // Add region pragmas.
   AddPragmaHandler(new PragmaRegionHandler("region"));
   AddPragmaHandler(new PragmaRegionHandler("endregion"));
 
   // MS extensions.
   if (LangOpts.MicrosoftExt) {
     AddPragmaHandler(new PragmaWarningHandler());
     AddPragmaHandler(new PragmaExecCharsetHandler());

clang/lib/Lex/Preprocessor.cpp

void Preprocessor::emitFinalMacroWarning(const Token &Identifier,
   assert(A.FinalAnnotationLoc &&
          "Final macro warning without recorded annotation!");
 
   Diag(Identifier, diag::warn_pragma_final_macro)
       << Identifier.getIdentifierInfo() << (IsUndef ? 0 : 1);
   Diag(*A.FinalAnnotationLoc, diag::note_pp_macro_annotation) << 2;
 }
 
+bool Preprocessor::isSafeBufferOptOut(const SourceManager &SourceMgr,
+                                           const SourceLocation &Loc) const {
+  // Try to find a region in `SafeBufferOptOutMap` where `Loc` is in:
+  auto FirstRegionEndingAfterLoc = llvm::partition_point(
+      SafeBufferOptOutMap,
+      [&SourceMgr,
+       &Loc](const std::pair<SourceLocation, SourceLocation> &Region) {
+        return SourceMgr.isBeforeInTranslationUnit(Region.second, Loc);
+      });
+
+  if (FirstRegionEndingAfterLoc != SafeBufferOptOutMap.end()) {
+    // To test if the start location of the found region precedes `Loc`:
+    return SourceMgr.isBeforeInTranslationUnit(FirstRegionEndingAfterLoc->first,
+                                               Loc);
+  }
+  // If we do not find a region whose end location passes `Loc`, we want to
+  // check if the current region is still open:
+  if (!SafeBufferOptOutMap.empty() &&
+      SafeBufferOptOutMap.back().first == SafeBufferOptOutMap.back().second)
+    return SourceMgr.isBeforeInTranslationUnit(SafeBufferOptOutMap.back().first,
+                                               Loc);
+  return false;
+}
+
+bool Preprocessor::enterOrExitSafeBufferOptOutRegion(
+    bool isEnter, const SourceLocation &Loc) {
+  if (isEnter) {
+    if (isPPInSafeBufferOptOutRegion())
+      return true; // invalid enter action
+    InSafeBufferOptOutRegion = true;
+    CurrentSafeBufferOptOutStart = Loc;
+
+    // To set the start location of a new region:
+
+    if (!SafeBufferOptOutMap.empty()) {
+      auto *PrevRegion = &SafeBufferOptOutMap.back();
+      assert(PrevRegion->first != PrevRegion->second &&
+             "Shall not begin a safe buffer opt-out region before closing the "
+             "previous one.");
+    }
+    // If the start location equals to the end location, we call the region a
+    // open region or a unclosed region (i.e., end location has not been set
+    // yet).
+    SafeBufferOptOutMap.emplace_back(Loc, Loc);
+  } else {
+    if (!isPPInSafeBufferOptOutRegion())
+      return true; // invalid enter action
+    InSafeBufferOptOutRegion = false;
+
+    // To set the end location of the current open region:
+
+    assert(!SafeBufferOptOutMap.empty() &&
+           "Misordered safe buffer opt-out regions");
+    auto *CurrRegion = &SafeBufferOptOutMap.back();
+    assert(CurrRegion->first == CurrRegion->second &&
+           "Set end location to a closed safe buffer opt-out region");
+    CurrRegion->second = Loc;
+  }
+  return false;
+}
+
+bool Preprocessor::isPPInSafeBufferOptOutRegion() {
+  return InSafeBufferOptOutRegion;
+}
+bool Preprocessor::isPPInSafeBufferOptOutRegion(SourceLocation &StartLoc) {
+  StartLoc = CurrentSafeBufferOptOutStart;
+  return InSafeBufferOptOutRegion;
+}
+
 ModuleLoader::~ModuleLoader() = default;
 
 CommentHandler::~CommentHandler() = default;
 
 EmptylineHandler::~EmptylineHandler() = default;
 
 CodeCompletionHandler::~CodeCompletionHandler() = default;
 
 void Preprocessor::createPreprocessingRecord() {
   if (Record)
     return;
 
   Record = new PreprocessingRecord(getSourceManager());
   addPPCallbacks(std::unique_ptr<PPCallbacks>(Record));
 }

clang/lib/Sema/AnalysisBasedWarnings.cpp

public:
   void handleFixableVariable(const VarDecl *Variable,
                              FixItList &&Fixes) override {
     S.Diag(Variable->getLocation(), diag::warn_unsafe_buffer_variable)
         << Variable << (Variable->getType()->isPointerType() ? 0 : 1)
         << Variable->getSourceRange();
     if (!Fixes.empty()) {
       unsigned FixItStrategy = 0; // For now we only has 'std::span' strategy
       const auto &FD = S.Diag(Variable->getLocation(),
                               diag::note_unsafe_buffer_variable_fixit);
 
       FD << Variable->getName() << FixItStrategy;
       for (const auto &F : Fixes)
         FD << F;
     }
   }
+
+  const clang::Preprocessor & getPP() const override {
+    return S.getPreprocessor();
+  }  
 };
 } // namespace
 
 //===----------------------------------------------------------------------===//
 // AnalysisBasedWarnings - Worker object used by Sema to execute analysis-based
 //  warnings on a function, method, or block.
 //===----------------------------------------------------------------------===//
 

clang/test/SemaCXX/warn-unsafe-buffer-usage-pragma-fixit.cpp

+// RUN: %clang_cc1 -std=c++20 -Wunsafe-buffer-usage -fdiagnostics-parseable-fixits %s 2>&1 | FileCheck %s
+
+void basic(int * x) {
+  int tmp;
+  int *p1 = new int[10];  // no fix
+  // CHECK-NOT: fix-it:"{{.*}}":{[[@LINE-1]]:
+  int *p2 = new int[10];
+  // CHECK-DAG: fix-it:"{{.*}}":{[[@LINE-1]]:3-[[@LINE-1]]:12}:"std::span<int> p2"
+  // CHECK-DAG: fix-it:"{{.*}}":{[[@LINE-2]]:13-[[@LINE-2]]:13}:"{"
+  // CHECK-DAG: fix-it:"{{.*}}":{[[@LINE-3]]:24-[[@LINE-3]]:24}:", 10}"
+#pragma clang unsafe_buffer_usage begin
+  tmp = p1[5];
+#pragma clang unsafe_buffer_usage end
+  tmp = p2[5];
+}
+
+void withDiagnosticWarning() {
+  int tmp;
+  int *p1 = new int[10]; // no fix
+  // CHECK-NOT: fix-it:"{{.*}}":{[[@LINE-1]]:
+  int *p2 = new int[10];
+  // CHECK-DAG: fix-it:"{{.*}}":{[[@LINE-1]]:3-[[@LINE-1]]:12}:"std::span<int> p2"
+  // CHECK-DAG: fix-it:"{{.*}}":{[[@LINE-2]]:13-[[@LINE-2]]:13}:"{"
+  // CHECK-DAG: fix-it:"{{.*}}":{[[@LINE-3]]:24-[[@LINE-3]]:24}:", 10}"
+
+  // diagnostics in opt-out region
+#pragma clang unsafe_buffer_usage begin
+  tmp = p1[5];  // not to warn
+  tmp = p2[5];  // not to warn
+#pragma clang diagnostic push
+#pragma clang diagnostic warning "-Wunsafe-buffer-usage"
+  tmp = p1[5];  // not to warn
+  tmp = p2[5];  // not to warn
+#pragma clang diagnostic warning "-Weverything"
+  tmp = p1[5];  // not to warn
+  tmp = p2[5];  // not to warn
+#pragma clang diagnostic pop
+#pragma clang unsafe_buffer_usage end
+
+  // opt-out region under diagnostic warning
+#pragma clang diagnostic push
+#pragma clang diagnostic warning "-Wunsafe-buffer-usage"
+#pragma clang unsafe_buffer_usage begin
+  tmp = p1[5];  // not to warn
+  tmp = p2[5];  // not to warn
+#pragma clang unsafe_buffer_usage end
+#pragma clang diagnostic pop
+
+  tmp = p2[5];
+}
+
+
+void withDiagnosticIgnore() {
+  int tmp;
+  int *p1 = new int[10];
+  // CHECK-NOT: fix-it:"{{.*}}":{[[@LINE-1]]:
+  int *p2 = new int[10];
+  // CHECK-DAG: fix-it:"{{.*}}":{[[@LINE-1]]:3-[[@LINE-1]]:12}:"std::span<int> p2"
+  // CHECK-DAG: fix-it:"{{.*}}":{[[@LINE-2]]:13-[[@LINE-2]]:13}:"{"
+  // CHECK-DAG: fix-it:"{{.*}}":{[[@LINE-3]]:24-[[@LINE-3]]:24}:", 10}"
+  int *p3 = new int[10];
+  // CHECK-DAG: fix-it:"{{.*}}":{[[@LINE-1]]:3-[[@LINE-1]]:12}:"std::span<int> p3"
+  // CHECK-DAG: fix-it:"{{.*}}":{[[@LINE-2]]:13-[[@LINE-2]]:13}:"{"
+  // CHECK-DAG: fix-it:"{{.*}}":{[[@LINE-3]]:24-[[@LINE-3]]:24}:", 10}"
+
+#pragma clang unsafe_buffer_usage begin
+  tmp = p1[5];  // not to warn
+  tmp = p2[5];  // not to warn
+#pragma clang diagnostic push
+#pragma clang diagnostic ignored "-Wunsafe-buffer-usage"
+  tmp = p1[5];  // not to warn
+  tmp = p2[5];  // not to warn
+#pragma clang diagnostic ignored "-Weverything"
+  tmp = p1[5];  // not to warn
+  tmp = p2[5];  // not to warn
+#pragma clang diagnostic pop
+#pragma clang unsafe_buffer_usage end
+
+#pragma clang diagnostic push
+#pragma clang diagnostic ignored "-Wunsafe-buffer-usage"
+#pragma clang unsafe_buffer_usage begin
+  tmp = p1[5];  // not to warn
+  tmp = p2[5];  // not to warn
+#pragma clang unsafe_buffer_usage end
+#pragma clang diagnostic pop
+
+  tmp = p2[5];
+
+#pragma clang diagnostic push
+#pragma clang diagnostic ignored "-Wunsafe-buffer-usage"
+#pragma clang unsafe_buffer_usage begin
+  tmp = p1[5];  // not to warn
+  tmp = p2[5];  // not to warn
+#pragma clang unsafe_buffer_usage end
+  tmp = p3[5];  // expected-note{{used in buffer access here}}
+#pragma clang diagnostic pop
+}
+
+void noteGoesWithVarDeclWarning() {
+#pragma clang diagnostic push
+#pragma clang diagnostic ignored "-Wunsafe-buffer-usage"
+  int *p = new int[10]; // not to warn
+  // CHECK-NOT: fix-it:"{{.*}}":{[[@LINE-1]]:
+#pragma clang diagnostic pop
+
+  p[5]; // not to note since the associated warning is suppressed
+}

clang/test/SemaCXX/warn-unsafe-buffer-usage-pragma-misuse.cpp

+// RUN: %clang_cc1 -std=c++20 -Wunsafe-buffer-usage -verify %s
+
+void beginUnclosed(int * x) {
+#pragma clang unsafe_buffer_usage begin
+
+#pragma clang unsafe_buffer_usage begin  // expected-error{{already inside '#pragma unsafe_buffer_usage'}}
+  x++;
+#pragma clang unsafe_buffer_usage end
+}
+
+void endUnopened(int *x) {
+#pragma clang unsafe_buffer_usage end    // expected-error{{not currently inside '#pragma unsafe_buffer_usage'}}
+
+#pragma clang unsafe_buffer_usage begin
+  x++;
+#pragma clang unsafe_buffer_usage end
+}
+
+void wrongOption() {
+#pragma clang unsafe_buffer_usage start // expected-error{{Expected 'begin' or 'end'}}
+#pragma clang unsafe_buffer_usage close // expected-error{{Expected 'begin' or 'end'}}
+}
+
+void unclosed(int * p1) {
+#pragma clang unsafe_buffer_usage begin
+// End of the included file will not raise the unclosed region warning:
+#define _INCLUDE_NO_WARN
+#include "warn-unsafe-buffer-usage-pragma.h"
+#pragma clang unsafe_buffer_usage end
+
+// End of this file raises the warning:
+#pragma clang unsafe_buffer_usage begin  // expected-error{{'#pragma unsafe_buffer_usage' was not ended}}
+}

clang/test/SemaCXX/warn-unsafe-buffer-usage-pragma.h

+#ifdef _INCLUDE_NO_WARN
+// the snippet will be included in an opt-out region
+p1++;
+
+#undef _INCLUDE_NO_WARN
+
+#elif defined(_INCLUDE_WARN)
+// the snippet will be included in a location where warnings are expected
+p2++; // expected-note{{used in pointer arithmetic here}}
+#undef _INCLUDE_WARN
+
+#else
+
+#endif

clang/test/SemaCXX/warn-unsafe-buffer-usage-pragma.cpp

+// RUN: %clang_cc1 -std=c++20 -Wunsafe-buffer-usage -Wno-unused-value -verify %s
+
+void basic(int * x) {    // expected-warning{{'x' is an unsafe pointer used for buffer access}}
+  int *p1 = new int[10]; // not to warn
+  int *p2 = new int[10]; // expected-warning{{'p2' is an unsafe pointer used for buffer access}}
+
+#pragma clang unsafe_buffer_usage begin
+  p1[5];  // not to warn
+
+#define _INCLUDE_NO_WARN
+#include "warn-unsafe-buffer-usage-pragma.h" // increment p1 in header
+
+  int *p3 = new int[10]; // expected-warning{{'p3' is an unsafe pointer used for buffer access}}
+
+#pragma clang unsafe_buffer_usage end
+  p2[5]; //expected-note{{used in buffer access here}}
+  p3[5]; //expected-note{{used in buffer access here}}
+  x++;   //expected-note{{used in pointer arithmetic here}}
+#define _INCLUDE_WARN
+#include "warn-unsafe-buffer-usage-pragma.h" // increment p2 in header
+}
+
+
+void withDiagnosticWarning() {
+  int *p1 = new int[10]; // not to warn
+  int *p2 = new int[10]; // expected-warning{{'p2' is an unsafe pointer used for buffer access}}
+
+  // diagnostics in opt-out region
+#pragma clang unsafe_buffer_usage begin
+  p1[5];  // not to warn
+  p2[5];  // not to warn
+#pragma clang diagnostic push
+#pragma clang diagnostic warning "-Wunsafe-buffer-usage"
+  p1[5];  // not to warn
+  p2[5];  // not to warn
+#pragma clang diagnostic warning "-Weverything"
+  p1[5];  // not to warn expected-warning{{expression result unused}}
+  p2[5];  // not to warn expected-warning{{expression result unused}}
+#pragma clang diagnostic pop
+#pragma clang unsafe_buffer_usage end
+
+  // opt-out region under diagnostic warning
+#pragma clang diagnostic push
+#pragma clang diagnostic warning "-Wunsafe-buffer-usage"
+#pragma clang unsafe_buffer_usage begin
+  p1[5];  // not to warn
+  p2[5];  // not to warn
+#pragma clang unsafe_buffer_usage end
+#pragma clang diagnostic pop
+
+  p2[5]; // expected-note{{used in buffer access here}}
+}
+
+
+void withDiagnosticIgnore() {
+  int *p1 = new int[10]; // not to warn
+  int *p2 = new int[10]; // expected-warning{{'p2' is an unsafe pointer used for buffer access}}
+  int *p3 = new int[10]; // expected-warning{{'p3' is an unsafe pointer used for buffer access}}
+
+#pragma clang unsafe_buffer_usage begin
+  p1[5];  // not to warn
+  p2[5];  // not to warn
+#pragma clang diagnostic push
+#pragma clang diagnostic ignored "-Wunsafe-buffer-usage"
+  p1[5];  // not to warn
+  p2[5];  // not to warn
+#pragma clang diagnostic ignored "-Weverything"
+  p1[5];  // not to warn
+  p2[5];  // not to warn
+#pragma clang diagnostic pop
+#pragma clang unsafe_buffer_usage end
+
+#pragma clang diagnostic push
+#pragma clang diagnostic ignored "-Wunsafe-buffer-usage"
+#pragma clang unsafe_buffer_usage begin
+  p1[5];  // not to warn
+  p2[5];  // not to warn
+#pragma clang unsafe_buffer_usage end
+#pragma clang diagnostic pop
+
+  p2[5]; // expected-note{{used in buffer access here}}
+
+#pragma clang diagnostic push
+#pragma clang diagnostic ignored "-Wunsafe-buffer-usage"
+#pragma clang unsafe_buffer_usage begin
+  p1[5];  // not to warn
+  p2[5];  // not to warn
+#pragma clang unsafe_buffer_usage end
+  p3[5];  // expected-note{{used in buffer access here}}
+#pragma clang diagnostic pop
+}
+
+void noteGoesWithVarDeclWarning() {
+#pragma clang diagnostic push
+#pragma clang diagnostic ignored "-Wunsafe-buffer-usage"
+  int *p = new int[10]; // not to warn
+#pragma clang diagnostic pop
+
+  p[5]; // not to note since the associated warning is suppressed
+}