[lldb] Print better diagnostics for user expressions and modules

[teemperor]

Summary

Currently our expression evaluators only prints very basic errors that are not very useful when writing complex expressions.

For example, in the expression below the user made a type error, but it's not clear from the diagnostic what went wrong:

(lldb) expr printf("Modulos are:", foobar%mo1, foobar%mo2, foobar%mo3)
error: invalid operands to binary expression ('int' and 'double')

This patch enables full Clang diagnostics in our expression evaluator. After this patch the diagnostics for the expression look like this:

(lldb) expr printf("Modulos are:", foobar%mo1, foobar%mo2, foobar%mo3)
error: <user expression 1>:1:54: invalid operands to binary expression ('int' and 'float')
printf("Modulos are:", foobar%mo1, foobar%mo2, foobar%mo3)
                                               ~~~~~~^~~~

To make this possible, we now emulate a user expression file within our diagnostics. This prevents that the user is exposed to
our internal wrapper code we inject.

Note that the diagnostics that refer to declarations from the debug information (e.g. 'note' diagnostics pointing to a called function)
will not be improved by this as they don't have any source locations associated with them, so caret or line printing isn't possible.
We instead just suppress these diagnostics as we already do with warnings as they would otherwise just be a context message
without any context (and the original diagnostic in the user expression should be enough to explain the issue).

Fixes rdar://24306342

[aprantl]

I have a question based on my half-knowledge about the expression evaluator: I thought that we already wrote out a temporary file for the expression in order to support expr -g? Is this orthogonal or do we now have to ways of pretending there is a source file backing up the expression?

lldb/source/Plugins/ExpressionParser/Clang/ClangExpressionSourceCode.cpp

73 ↗

(On Diff #213018)

The comment seems asymmetric now.

[jingham]

In D65646#1612436, @aprantl wrote:

I have a question based on my half-knowledge about the expression evaluator: I thought that we already wrote out a temporary file for the expression in order to support expr -g? Is this orthogonal or do we now have to ways of pretending there is a source file backing up the expression?

The file creation happens a little above the change to pass in a memory file:

// We also want a real file on disk if we generate full debug info.
should_create_file |= m_compiler->getCodeGenOpts().getDebugInfo() ==
                      codegenoptions::FullDebugInfo;

around line 891. Raphael's change doesn't change whether we emit this source file or not.

But it seem weird that this will make every expression seem to come from the same file and line. It would be really nice to be able to set breakpoints in compiled expressions, but if all the files think they have the same source file & starting line, this will be harder to get working.

Note, however, the creation of the source file & tracing back from the debug_line that we insert when we JIT the expression to this source file seems broken, however. If I do:

(lldb) expr --top-level -- int $foo(int) { (int) printf("I am here.\n"); }
(lldb) b s -n printf
Breakpoint 3: where = libsystem_c.dylib`printf, address = 0x00007fff616588ec
(lldb) expr -i 0 -- $foo(10)
1 location added to breakpoint 2
error: Execution was interrupted, reason: breakpoint 2.1.
The process has been left at the point where it was interrupted, use "thread return -x" to return to the state before expression evaluation.
Process 3708 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 2.1
    frame #0: 0x0000000100104b1b $__lldb_expr1`$foo(int) at Parse:1
Target 0: (something) stopped.

I should have seen the source code for $foo, but we don't seem to have found the file. So something is broken in this process.

[teemperor]

Is this orthogonal or do we now have to ways of pretending there is a source file backing up the expression?

Jim already answered this, but this patch only changes the file names of our expression and pretends for the sake of diagnostics that we are in a separate file for the user code.

But it seem weird that this will make every expression seem to come from the same file and line.

What about enumerating the expression file names we print? E.g. <user expression 1>, <user expression 2>?

[jingham]

That would be fine. In the swift REPL, we increment the line number so that it appears that all the expressions are one contiguous source file. But I think for the expression parser treating each expression as a separate file is a better fiction.

[teemperor]

• We now enumerate the fake expression files. E.g. <user expression 3>.

[JDevlieghere]

lldb/source/Plugins/ExpressionParser/Clang/ClangExpressionParser.cpp

267 ↗

(On Diff #213587)

std::move(filename)

lldb/source/Plugins/ExpressionParser/Clang/ClangExpressionSourceCode.h

29 ↗

(On Diff #213587)

Can we use StringRefs for prefix and body?

31 ↗

(On Diff #213587)

Can you use an enum to name the boolean parameter?

enum Wrapping : bool {
    Wrap = true,
    NoWrap = false,
  };

lldb/source/Plugins/ExpressionParser/Clang/ClangPersistentVariables.h

88 ↗

(On Diff #213587)

///

[teemperor]

• Addressed feedback.

[teemperor]

• Addressed some feedback I forgot.

[teemperor]

Fixed some of the unrelated comments in a separate commit (that is already upstreamed).

[JDevlieghere]

LGTM, but you might want to have someone with more functional knowledge give the go-ahead.

[teemperor]

ping :)

[teemperor]

• Rebased test.
• Also testing Objective-C modules now (as the ClangModulesDeclVendor provides valid SourceLocations, so this patch also improves diagnostics from modules).

cfe/trunk/include/clang/Basic/DiagnosticOptions.def

 #endif
 
 SEMANTIC_DIAGOPT(IgnoreWarnings, 1, 0)   /// -w
 DIAGOPT(NoRewriteMacros, 1, 0)  /// -Wno-rewrite-macros
 DIAGOPT(Pedantic, 1, 0)         /// -pedantic
 DIAGOPT(PedanticErrors, 1, 0)   /// -pedantic-errors
 DIAGOPT(ShowColumn, 1, 1)       /// Show column number on diagnostics.
 DIAGOPT(ShowLocation, 1, 1)     /// Show source location information.
+DIAGOPT(ShowLevel, 1, 1)        /// Show diagnostic level.
 DIAGOPT(AbsolutePath, 1, 0)     /// Use absolute paths.
 DIAGOPT(ShowCarets, 1, 1)       /// Show carets in diagnostics.
 DIAGOPT(ShowFixits, 1, 1)       /// Show fixit information.
 DIAGOPT(ShowSourceRanges, 1, 0) /// Show source ranges in numeric form.
 DIAGOPT(ShowParseableFixits, 1, 0) /// Show machine parseable fix-its.
 DIAGOPT(ShowPresumedLoc, 1, 0)  /// Show presumed location for diagnostics.
 DIAGOPT(ShowOptionNames, 1, 0)  /// Show the option name for mappable
                                 /// diagnostics.

cfe/trunk/lib/Frontend/TextDiagnostic.cpp

void TextDiagnostic::emitDiagnosticMessage(
 
   // Emit the location of this particular diagnostic.
   if (Loc.isValid())
     emitDiagnosticLoc(Loc, PLoc, Level, Ranges);
 
   if (DiagOpts->ShowColors)
     OS.resetColor();
 
+  if (DiagOpts->ShowLevel)
     printDiagnosticLevel(OS, Level, DiagOpts->ShowColors,
                          DiagOpts->CLFallbackMode);
   printDiagnosticMessage(OS,
                          /*IsSupplemental*/ Level == DiagnosticsEngine::Note,
                          Message, OS.tell() - StartOfLocationInfo,
                          DiagOpts->MessageLength, DiagOpts->ShowColors);
 }
 
 /*static*/ void
 TextDiagnostic::printDiagnosticLevel(raw_ostream &OS,

lldb/trunk/packages/Python/lldbsuite/test/commands/expression/diagnostics/Makefile

+CXX_SOURCES := main.cpp
+
+include Makefile.rules

lldb/trunk/packages/Python/lldbsuite/test/commands/expression/diagnostics/TestExprDiagnostics.py

+"""
+Test the diagnostics emitted by our embeded Clang instance that parses expressions.
+"""
+
+import lldb
+from lldbsuite.test.lldbtest import *
+from lldbsuite.test import lldbutil
+from lldbsuite.test.decorators import *
+
+class ExprDiagnosticsTestCase(TestBase):
+
+    mydir = TestBase.compute_mydir(__file__)
+
+    def setUp(self):
+        # Call super's setUp().
+        TestBase.setUp(self)
+
+        self.main_source = "main.cpp"
+        self.main_source_spec = lldb.SBFileSpec(self.main_source)
+
+    def test_source_and_caret_printing(self):
+        """Test that the source and caret positions LLDB prints are correct"""
+        self.build()
+
+        (target, process, thread, bkpt) = lldbutil.run_to_source_breakpoint(self,
+                                          '// Break here', self.main_source_spec)
+        frame = thread.GetFrameAtIndex(0)
+
+        # Test that source/caret are at the right position.
+        value = frame.EvaluateExpression("unknown_identifier")
+        self.assertFalse(value.GetError().Success())
+        # We should get a nice diagnostic with a caret pointing at the start of
+        # the identifier.
+        self.assertIn("\nunknown_identifier\n^\n", value.GetError().GetCString())
+        self.assertIn("<user expression 0>:1:1", value.GetError().GetCString())
+
+        # Same as above but with the identifier in the middle.
+        value = frame.EvaluateExpression("1 + unknown_identifier  ")
+        self.assertFalse(value.GetError().Success())
+        self.assertIn("\n1 + unknown_identifier", value.GetError().GetCString())
+        self.assertIn("\n    ^\n", value.GetError().GetCString())
+
+        # Multiline expressions.
+        value = frame.EvaluateExpression("int a = 0;\nfoobar +=1;\na")
+        self.assertFalse(value.GetError().Success())
+        # We should still get the right line information and caret position.
+        self.assertIn("\nfoobar +=1;\n^\n", value.GetError().GetCString())
+        # It's the second line of the user expression.
+        self.assertIn("<user expression 2>:2:1", value.GetError().GetCString())
+
+        # Top-level expressions.
+        top_level_opts = lldb.SBExpressionOptions();
+        top_level_opts.SetTopLevel(True)
+
+        value = frame.EvaluateExpression("void foo(unknown_type x) {}", top_level_opts)
+        self.assertFalse(value.GetError().Success())
+        self.assertIn("\nvoid foo(unknown_type x) {}\n         ^\n", value.GetError().GetCString())
+        # Top-level expressions might use a different wrapper code, but the file name should still
+        # be the same.
+        self.assertIn("<user expression 3>:1:10", value.GetError().GetCString())
+
+        # Multiline top-level expressions.
+        value = frame.EvaluateExpression("void x() {}\nvoid foo(unknown_type x) {}", top_level_opts)
+        self.assertFalse(value.GetError().Success())
+        self.assertIn("\nvoid foo(unknown_type x) {}\n         ^\n", value.GetError().GetCString())
+        self.assertIn("<user expression 4>:2:10", value.GetError().GetCString())
+
+        # Test that we render Clang's 'notes' correctly.
+        value = frame.EvaluateExpression("struct SFoo{}; struct SFoo { int x; };", top_level_opts)
+        self.assertFalse(value.GetError().Success())
+        self.assertIn("<user expression 5>:1:8: previous definition is here\nstruct SFoo{}; struct SFoo { int x; };\n       ^\n", value.GetError().GetCString())
+
+        # Declarations from the debug information currently have no debug information. It's not clear what
+        # we should do in this case, but we should at least not print anything that's wrong.
+        # In the future our declarations should have valid source locations.
+        value = frame.EvaluateExpression("struct FooBar { double x };", top_level_opts)
+        self.assertFalse(value.GetError().Success())
+        self.assertEqual("error: <user expression 6>:1:8: redefinition of 'FooBar'\nstruct FooBar { double x };\n       ^\n", value.GetError().GetCString())
+
+        value = frame.EvaluateExpression("foo(1, 2)")
+        self.assertFalse(value.GetError().Success())
+        self.assertEqual("error: <user expression 7>:1:1: no matching function for call to 'foo'\nfoo(1, 2)\n^~~\nnote: candidate function not viable: requires single argument 'x', but 2 arguments were provided\n\n", value.GetError().GetCString())
+
+        # Redefine something that we defined in a user-expression. We should use the previous expression file name
+        # for the original decl.
+        value = frame.EvaluateExpression("struct Redef { double x; };", top_level_opts)
+        value = frame.EvaluateExpression("struct Redef { float y; };", top_level_opts)
+        self.assertFalse(value.GetError().Success())
+        self.assertIn("error: <user expression 9>:1:8: redefinition of 'Redef'\nstruct Redef { float y; };\n       ^\n<user expression 8>:1:8: previous definition is here\nstruct Redef { double x; };\n       ^", value.GetError().GetCString())
+
+    @skipUnlessDarwin
+    def test_source_locations_from_objc_modules(self):
+        self.build()
+
+        (target, process, thread, bkpt) = lldbutil.run_to_source_breakpoint(self,
+                                          '// Break here', self.main_source_spec)
+        frame = thread.GetFrameAtIndex(0)
+
+        # Import foundation so that the Obj-C module is loaded (which contains source locations
+        # that can be used by LLDB).
+        self.runCmd("expr @import Foundation")
+        value = frame.EvaluateExpression("NSLog(1);")
+        self.assertFalse(value.GetError().Success())
+        print(value.GetError().GetCString())
+        # LLDB should print the source line that defines NSLog. To not rely on any
+        # header paths/line numbers or the actual formatting of the Foundation headers, only look
+        # for a few tokens in the output.
+        # File path should come from Foundation framework.
+        self.assertIn("/Foundation.framework/", value.GetError().GetCString())
+        # The NSLog definition source line should be printed. Return value and
+        # the first argument are probably stable enough that this test can check for them.
+        self.assertIn("void NSLog(NSString *format", value.GetError().GetCString())
+

lldb/trunk/packages/Python/lldbsuite/test/commands/expression/diagnostics/main.cpp

+void foo(int x) {}
+
+struct FooBar {
+  int i;
+};
+
+int main() {
+  FooBar f;
+  foo(1);
+  return 0; // Break here
+}

lldb/trunk/packages/Python/lldbsuite/test/lang/objc/foundation/TestObjCMethods.py

# locations = 1")
         self.expect("expression self->isa", VARIABLES_DISPLAYED_CORRECTLY,
                     substrs=["(Class)"])
 
         # This should fail expectedly.
         self.expect(
             "expression self->non_existent_member",
             COMMAND_FAILED_AS_EXPECTED,
             error=True,
-            startstr="error: 'MyString' does not have a member named 'non_existent_member'")
+            substrs=["error:", "'MyString' does not have a member named 'non_existent_member'"])
 
         # Use expression parser.
         self.runCmd("expression self->str")
         self.runCmd("expression self->date")
 
         # (lldb) expression self->str
         # error: instance variable 'str' is protected
         # error: 1 errors parsing expression

lldb/trunk/source/Plugins/ExpressionParser/Clang/ClangDiagnostic.h

 public:
   typedef std::vector<clang::FixItHint> FixItList;
 
   static inline bool classof(const ClangDiagnostic *) { return true; }
   static inline bool classof(const Diagnostic *diag) {
     return diag->getKind() == eDiagnosticOriginClang;
   }
 
-  ClangDiagnostic(const char *message, DiagnosticSeverity severity,
+  ClangDiagnostic(llvm::StringRef message, DiagnosticSeverity severity,
                   uint32_t compiler_id)
       : Diagnostic(message, severity, eDiagnosticOriginClang, compiler_id) {}
 
   ~ClangDiagnostic() override = default;
 
   bool HasFixIts() const override { return !m_fixit_vec.empty(); }
 
   void AddFixitHint(const clang::FixItHint &fixit) {

lldb/trunk/source/Plugins/ExpressionParser/Clang/ClangExpressionParser.h

public:
   ///     optional tuning Objective-C runtime support. Can be NULL.
   ///
   /// \param[in] expr
   ///     The expression to be parsed.
   ///
   /// @param[in] include_directories
   ///     List of include directories that should be used when parsing the
   ///     expression.
+  ///
+  /// @param[in] filename
+  ///     Name of the source file that should be used when rendering
+  ///     diagnostics (i.e. errors, warnings or notes from Clang).
   ClangExpressionParser(ExecutionContextScope *exe_scope, Expression &expr,
                         bool generate_debug_info,
-                        std::vector<std::string> include_directories = {});
+                        std::vector<std::string> include_directories = {},
+                        std::string filename = "<clang expression>");
 
   /// Destructor
   ~ClangExpressionParser() override;
 
   bool Complete(CompletionRequest &request, unsigned line, unsigned pos,
                 unsigned typed_pos) override;
 
   /// Parse a single expression and convert it to IR using Clang.  Don't wrap
std::unique_ptr<clang::CodeGenerator>
       m_code_generator; ///< The Clang object that generates IR
 
   class LLDBPreprocessorCallbacks;
   LLDBPreprocessorCallbacks *m_pp_callbacks; ///< Called when the preprocessor
                                              ///encounters module imports
   std::unique_ptr<ClangASTContext> m_ast_context;
 
   std::vector<std::string> m_include_directories;
+  /// File name used for the user expression.
+  std::string m_filename;
 };
 }
 
 #endif // liblldb_ClangExpressionParser_h_

lldb/trunk/source/Plugins/ExpressionParser/Clang/ClangExpressionParser.cpp

public:
 
   bool hasErrors() { return m_has_errors; }
 
   llvm::StringRef getErrorString() { return m_error_stream.GetString(); }
 };
 
 class ClangDiagnosticManagerAdapter : public clang::DiagnosticConsumer {
 public:
-  ClangDiagnosticManagerAdapter()
-      : m_passthrough(new clang::TextDiagnosticBuffer) {}
-
-  ClangDiagnosticManagerAdapter(
-      const std::shared_ptr<clang::TextDiagnosticBuffer> &passthrough)
-      : m_passthrough(passthrough) {}
+  ClangDiagnosticManagerAdapter(DiagnosticOptions &opts) {
+    DiagnosticOptions *m_options = new DiagnosticOptions(opts);
+    m_options->ShowPresumedLoc = true;
+    m_options->ShowLevel = false;
+    m_os.reset(new llvm::raw_string_ostream(m_output));
+    m_passthrough.reset(
+        new clang::TextDiagnosticPrinter(*m_os, m_options, false));
+  }
 
   void ResetManager(DiagnosticManager *manager = nullptr) {
     m_manager = manager;
   }
 
   void HandleDiagnostic(DiagnosticsEngine::Level DiagLevel,
                         const clang::Diagnostic &Info) override {
-    if (m_manager) {
-      llvm::SmallVector<char, 32> diag_str;
-      Info.FormatDiagnostic(diag_str);
-      diag_str.push_back('\0');
-      const char *data = diag_str.data();
+    // Render diagnostic message to m_output.
+    m_output.clear();
+    m_passthrough->HandleDiagnostic(DiagLevel, Info);
+    m_os->flush();
 
+    if (m_manager) {
       lldb_private::DiagnosticSeverity severity;
       bool make_new_diagnostic = true;
 
       switch (DiagLevel) {
       case DiagnosticsEngine::Level::Fatal:
       case DiagnosticsEngine::Level::Error:
         severity = eDiagnosticSeverityError;
         break;
       case DiagnosticsEngine::Level::Warning:
         severity = eDiagnosticSeverityWarning;
         break;
       case DiagnosticsEngine::Level::Remark:
       case DiagnosticsEngine::Level::Ignored:
         severity = eDiagnosticSeverityRemark;
         break;
       case DiagnosticsEngine::Level::Note:
-        m_manager->AppendMessageToDiagnostic(data);
+        m_manager->AppendMessageToDiagnostic(m_output);
         make_new_diagnostic = false;
       }
       if (make_new_diagnostic) {
+        // ClangDiagnostic messages are expected to have no whitespace/newlines
+        // around them.
+        std::string stripped_output = llvm::StringRef(m_output).trim();
+
         ClangDiagnostic *new_diagnostic =
-            new ClangDiagnostic(data, severity, Info.getID());
+            new ClangDiagnostic(stripped_output, severity, Info.getID());
         m_manager->AddDiagnostic(new_diagnostic);
 
         // Don't store away warning fixits, since the compiler doesn't have
         // enough context in an expression for the warning to be useful.
         // FIXME: Should we try to filter out FixIts that apply to our generated
         // code, and not the user's expression?
         if (severity == eDiagnosticSeverityError) {
           size_t num_fixit_hints = Info.getNumFixItHints();
           for (size_t i = 0; i < num_fixit_hints; i++) {
             const clang::FixItHint &fixit = Info.getFixItHint(i);
             if (!fixit.isNull())
               new_diagnostic->AddFixitHint(fixit);
           }
         }
       }
     }
-
-    m_passthrough->HandleDiagnostic(DiagLevel, Info);
-  }
-
-  void FlushDiagnostics(DiagnosticsEngine &Diags) {
-    m_passthrough->FlushDiagnostics(Diags);
-  }
-
-  DiagnosticConsumer *clone(DiagnosticsEngine &Diags) const {
-    return new ClangDiagnosticManagerAdapter(m_passthrough);
   }
 
-  clang::TextDiagnosticBuffer *GetPassthrough() { return m_passthrough.get(); }
+  clang::TextDiagnosticPrinter *GetPassthrough() { return m_passthrough.get(); }
 
 private:
   DiagnosticManager *m_manager = nullptr;
-  std::shared_ptr<clang::TextDiagnosticBuffer> m_passthrough;
+  std::shared_ptr<clang::TextDiagnosticPrinter> m_passthrough;
+  /// Output stream of m_passthrough.
+  std::shared_ptr<llvm::raw_string_ostream> m_os;
+  /// Output string filled by m_os.
+  std::string m_output;
 };
 
 static void SetupModuleHeaderPaths(CompilerInstance *compiler,
                                    std::vector<std::string> include_directories,
                                    lldb::TargetSP target_sp) {
   Log *log(lldb_private::GetLogIfAllCategoriesSet(LIBLLDB_LOG_EXPRESSIONS));
 
   HeaderSearchOptions &search_opts = compiler->getHeaderSearchOpts();
for (const std::string &include_dir : system_include_directories) {
     LLDB_LOG(log, "Added system include dir: {0}", include_dir);
   }
 }
 
 //===----------------------------------------------------------------------===//
 // Implementation of ClangExpressionParser
 //===----------------------------------------------------------------------===//
 
-ClangExpressionParser::ClangExpressionParser(
-    ExecutionContextScope *exe_scope, Expression &expr,
-    bool generate_debug_info, std::vector<std::string> include_directories)
+ClangExpressionParser::ClangExpressionParser(ExecutionContextScope *exe_scope, Expression &expr,
+    bool generate_debug_info, std::vector<std::string> include_directories, std::string filename)
     : ExpressionParser(exe_scope, expr, generate_debug_info), m_compiler(),
       m_pp_callbacks(nullptr),
-      m_include_directories(std::move(include_directories)) {
+      m_include_directories(std::move(include_directories)), m_filename(std::move(filename)) {
   Log *log(lldb_private::GetLogIfAllCategoriesSet(LIBLLDB_LOG_EXPRESSIONS));
 
   // We can't compile expressions without a target.  So if the exe_scope is
   // null or doesn't have a target, then we just need to get out of here.  I'll
   // lldb_assert and not make any of the compiler objects since
   // I can't return errors directly from the constructor.  Further calls will
   // check if the compiler was made and
   // bag out if it wasn't.
ClangExpressionParser::ClangExpressionParser(ExecutionContextScope *exe_scope, Expression &expr,
   // Inform the target of the language options
   //
   // FIXME: We shouldn't need to do this, the target should be immutable once
   // created. This complexity should be lifted elsewhere.
   m_compiler->getTarget().adjust(m_compiler->getLangOpts());
 
   // 6. Set up the diagnostic buffer for reporting errors
 
-  m_compiler->getDiagnostics().setClient(new ClangDiagnosticManagerAdapter);
+  auto diag_mgr = new ClangDiagnosticManagerAdapter(
+      m_compiler->getDiagnostics().getDiagnosticOptions());
+  m_compiler->getDiagnostics().setClient(diag_mgr);
 
   // 7. Set up the source management objects inside the compiler
   m_compiler->createFileManager();
   if (!m_compiler->hasSourceManager())
     m_compiler->createSourceManager(m_compiler->getFileManager());
   m_compiler->createPreprocessor(TU_Complete);
 
   if (ClangModulesDeclVendor *decl_vendor =

 unsigned
 ClangExpressionParser::ParseInternal(DiagnosticManager &diagnostic_manager,
                                      CodeCompleteConsumer *completion_consumer,
                                      unsigned completion_line,
                                      unsigned completion_column) {
   ClangDiagnosticManagerAdapter *adapter =
       static_cast<ClangDiagnosticManagerAdapter *>(
           m_compiler->getDiagnostics().getClient());
-  clang::TextDiagnosticBuffer *diag_buf = adapter->GetPassthrough();
-  diag_buf->FlushDiagnostics(m_compiler->getDiagnostics());
+  auto diag_buf = adapter->GetPassthrough();
 
   adapter->ResetManager(&diagnostic_manager);
 
   const char *expr_text = m_expr.Text();
 
   clang::SourceManager &source_mgr = m_compiler->getSourceManager();
   bool created_main_file = false;
 
if (temp_fd != -1) {
           }
         }
       }
     }
   }
 
   if (!created_main_file) {
     std::unique_ptr<MemoryBuffer> memory_buffer =
-        MemoryBuffer::getMemBufferCopy(expr_text, "<lldb-expr>");
+        MemoryBuffer::getMemBufferCopy(expr_text, m_filename);
     source_mgr.setMainFileID(source_mgr.createFileID(std::move(memory_buffer)));
   }
 
   diag_buf->BeginSourceFile(m_compiler->getLangOpts(),
                             &m_compiler->getPreprocessor());
 
   ClangExpressionHelper *type_system_helper =
       dyn_cast<ClangExpressionHelper>(m_expr.GetTypeSystemHelper());

lldb/trunk/source/Plugins/ExpressionParser/Clang/ClangExpressionSourceCode.h

 namespace lldb_private {
 
 class ExecutionContext;
 
 class ClangExpressionSourceCode : public ExpressionSourceCode {
 public:
   static const char *g_expression_prefix;
 
-  static ClangExpressionSourceCode *CreateWrapped(llvm::StringRef prefix,
+  static ClangExpressionSourceCode *CreateWrapped(llvm::StringRef filename,
+                                                  llvm::StringRef prefix,
                                                   llvm::StringRef body) {
-    return new ClangExpressionSourceCode("$__lldb_expr", prefix, body, Wrap);
+    return new ClangExpressionSourceCode(filename, "$__lldb_expr", prefix, body,
+                                         Wrap);
   }
 
   /// Generates the source code that will evaluate the expression.
   ///
   /// \param text output parameter containing the source code string.
   /// \param wrapping_language If the expression is supossed to be wrapped,
   ///        then this is the language that should be used for that.
   /// \param static_method True iff the expression is valuated inside a static
public:
   bool GetText(std::string &text, lldb::LanguageType wrapping_language,
                bool static_method, ExecutionContext &exe_ctx, bool add_locals,
                bool force_add_all_locals,
                llvm::ArrayRef<std::string> modules) const;
 
   // Given a string returned by GetText, find the beginning and end of the body
   // passed to CreateWrapped. Return true if the bounds could be found.  This
   // will also work on text with FixItHints applied.
-  static bool GetOriginalBodyBounds(std::string transformed_text,
-                                    lldb::LanguageType wrapping_language,
-                                    size_t &start_loc, size_t &end_loc);
+  bool GetOriginalBodyBounds(std::string transformed_text,
 
 protected:
-  ClangExpressionSourceCode(llvm::StringRef name, llvm::StringRef prefix,
-                            llvm::StringRef body, Wrapping wrap)
-      : ExpressionSourceCode(name, prefix, body, wrap) {}
+  ClangExpressionSourceCode(llvm::StringRef filename, llvm::StringRef name,
+                            llvm::StringRef prefix, llvm::StringRef body,
+                            Wrapping wrap);
+
+private:
+  /// String marking the start of the user expression.
+  std::string m_start_marker;
+  /// String marking the end of the user expression.
+  std::string m_end_marker;
 };
 
 } // namespace lldb_private
 
 #endif

lldb/trunk/source/Plugins/ExpressionParser/Clang/ClangExpressionSourceCode.cpp

 #include "lldb/Target/Language.h"
 #include "lldb/Target/Platform.h"
 #include "lldb/Target/StackFrame.h"
 #include "lldb/Target/Target.h"
 #include "lldb/Utility/StreamString.h"
 
 using namespace lldb_private;
 
-const char *ClangExpressionSourceCode::g_expression_prefix = R"(
+const char *ClangExpressionSourceCode::g_expression_prefix =
+    R"(
+#line 1 "<lldb wrapper prefix>"
 #ifndef offsetof
 #define offsetof(t, d) __builtin_offsetof(t, d)
 #endif
 #ifndef NULL
 #define NULL (__null)
 #endif
 #ifndef Nil
 #define Nil (__null)

 typedef __PTRDIFF_TYPE__ ptrdiff_t;
 typedef unsigned short unichar;
 extern "C"
 {
     int printf(const char * __restrict, ...);
 }
 )";
 
-static const char *c_start_marker = "    /*LLDB_BODY_START*/\n    ";
-static const char *c_end_marker = ";\n    /*LLDB_BODY_END*/\n";
-
 namespace {
 
 class AddMacroState {
   enum State {
     CURRENT_FILE_NOT_YET_PUSHED,
     CURRENT_FILE_PUSHED,
     CURRENT_FILE_POPPED
   };
case DebugMacroEntry::INDIRECT:
       break;
     default:
       // This is an unknown/invalid entry. Ignore.
       break;
     }
   }
 }
 
+lldb_private::ClangExpressionSourceCode::ClangExpressionSourceCode(
+    llvm::StringRef filename, llvm::StringRef name, llvm::StringRef prefix,
+    llvm::StringRef body, Wrapping wrap)
+    : ExpressionSourceCode(name, prefix, body, wrap) {
+  // Use #line markers to pretend that we have a single-line source file
+  // containing only the user expression. This will hide our wrapper code
+  // from the user when we render diagnostics with Clang.
+  m_start_marker = "#line 1 \"" + filename.str() + "\"\n";
+  m_end_marker = "\n;\n#line 1 \"<lldb wrapper suffix>\"\n";
+}
+
 namespace {
 /// Allows checking if a token is contained in a given expression.
 class TokenVerifier {
   /// The tokens we found in the expression.
   llvm::StringSet<> m_tokens;
 
 public:
   TokenVerifier(std::string body);
if (m_wrap) {
     std::string tagged_body;
     switch (wrapping_language) {
     default:
       tagged_body = m_body;
       break;
     case lldb::eLanguageTypeC:
     case lldb::eLanguageTypeC_plus_plus:
     case lldb::eLanguageTypeObjC:
-      tagged_body.append(c_start_marker);
+      tagged_body.append(m_start_marker);
       tagged_body.append(m_body);
-      tagged_body.append(c_end_marker);
+      tagged_body.append(m_end_marker);
       break;
     }
     switch (wrapping_language) {
     default:
       break;
     case lldb::eLanguageTypeC:
       wrap_stream.Printf("%s"
                          "void                           \n"
bool ClangExpressionSourceCode::GetText(
   }
 
   return true;
 }
 
 bool ClangExpressionSourceCode::GetOriginalBodyBounds(
     std::string transformed_text, lldb::LanguageType wrapping_language,
     size_t &start_loc, size_t &end_loc) {
-  const char *start_marker;
-  const char *end_marker;
-
   switch (wrapping_language) {
   default:
     return false;
   case lldb::eLanguageTypeC:
   case lldb::eLanguageTypeC_plus_plus:
   case lldb::eLanguageTypeObjC:
-    start_marker = c_start_marker;
-    end_marker = c_end_marker;
     break;
   }
 
-  start_loc = transformed_text.find(start_marker);
+  start_loc = transformed_text.find(m_start_marker);
   if (start_loc == std::string::npos)
     return false;
-  start_loc += strlen(start_marker);
-  end_loc = transformed_text.find(end_marker);
+  start_loc += m_start_marker.size();
+  end_loc = transformed_text.find(m_end_marker);
   return end_loc != std::string::npos;
 }

lldb/trunk/source/Plugins/ExpressionParser/Clang/ClangPersistentVariables.h

lldb::ExpressionVariableSP CreatePersistentVariable(
       uint32_t addr_byte_size) override;
 
   void RemovePersistentVariable(lldb::ExpressionVariableSP variable) override;
 
   llvm::StringRef GetPersistentVariablePrefix(bool is_error) const override {
     return "$";
   }
 
+  /// Returns the next file name that should be used for user expressions.
+  std::string GetNextExprFileName() {
+    std::string name;
+    name.append("<user expression ");
+    name.append(std::to_string(m_next_user_file_id++));
+    name.append(">");
+    return name;
+  }
+
   llvm::Optional<CompilerType>
   GetCompilerTypeFromPersistentDecl(ConstString type_name) override;
 
   void RegisterPersistentDecl(ConstString name, clang::NamedDecl *decl);
 
   clang::NamedDecl *GetPersistentDecl(ConstString name);
 
   void AddHandLoadedClangModule(ClangModulesDeclVendor::ModuleID module) {
     m_hand_loaded_clang_modules.push_back(module);
   }
 
   const ClangModulesDeclVendor::ModuleVector &GetHandLoadedClangModules() {
     return m_hand_loaded_clang_modules;
   }
 
 private:
+  /// The counter used by GetNextExprFileName.
+  uint32_t m_next_user_file_id = 0;
   // The counter used by GetNextPersistentVariableName
   uint32_t m_next_persistent_variable_id = 0;
 
   typedef llvm::DenseMap<const char *, clang::NamedDecl *> PersistentDeclMap;
   PersistentDeclMap
       m_persistent_decls; ///< Persistent entities declared by the user.
 
   ClangModulesDeclVendor::ModuleVector
       m_hand_loaded_clang_modules; ///< These are Clang modules we hand-loaded;
                                    ///these are the highest-
                                    ///< priority source for macros.
 };
 
 } // namespace lldb_private
 
 #endif // liblldb_ClangPersistentVariables_h_

lldb/trunk/source/Plugins/ExpressionParser/Clang/ClangUserExpression.h

 #define liblldb_ClangUserExpression_h_
 
 #include <vector>
 
 #include "ASTResultSynthesizer.h"
 #include "ASTStructExtractor.h"
 #include "ClangExpressionDeclMap.h"
 #include "ClangExpressionHelper.h"
+#include "ClangExpressionSourceCode.h"
 #include "ClangExpressionVariable.h"
 #include "IRForTarget.h"
 
 #include "lldb/Core/Address.h"
 #include "lldb/Core/ClangForward.h"
 #include "lldb/Expression/LLVMUserExpression.h"
 #include "lldb/Expression/Materializer.h"
 #include "lldb/Target/ExecutionContext.h"
private:
   /// The include directories that should be used when parsing the expression.
   std::vector<std::string> m_include_directories;
 
   /// The absolute character position in the transformed source code where the
   /// user code (as typed by the user) starts. If the variable is empty, then we
   /// were not able to calculate this position.
   llvm::Optional<size_t> m_user_expression_start_pos;
   ResultDelegate m_result_delegate;
+  ClangPersistentVariables *m_clang_state;
+  std::unique_ptr<ClangExpressionSourceCode> m_source_code;
+  /// File name used for the expression.
+  std::string m_filename;
 
   /// The object (if any) in which context the expression is evaluated.
   /// See the comment to `UserExpression::Evaluate` for details.
   ValueObject *m_ctx_obj;
 
   /// True iff this expression explicitly imported C++ modules.
   bool m_imported_cpp_modules = false;
 

lldb/trunk/source/Plugins/ExpressionParser/Clang/ClangUserExpression.cpp

 #include <string>
 
 #include "ClangUserExpression.h"
 
 #include "ASTResultSynthesizer.h"
 #include "ClangDiagnostic.h"
 #include "ClangExpressionDeclMap.h"
 #include "ClangExpressionParser.h"
-#include "ClangExpressionSourceCode.h"
 #include "ClangModulesDeclVendor.h"
 #include "ClangPersistentVariables.h"
 
 #include "lldb/Core/Debugger.h"
 #include "lldb/Core/Module.h"
 #include "lldb/Core/StreamFile.h"
 #include "lldb/Core/ValueObjectConstResult.h"
 #include "lldb/Expression/ExpressionSourceCode.h"

 }
 
 bool ClangUserExpression::SetupPersistentState(DiagnosticManager &diagnostic_manager,
                                  ExecutionContext &exe_ctx) {
   if (Target *target = exe_ctx.GetTargetPtr()) {
     if (PersistentExpressionState *persistent_state =
             target->GetPersistentExpressionStateForLanguage(
                 lldb::eLanguageTypeC)) {
+      m_clang_state = llvm::cast<ClangPersistentVariables>(persistent_state);
       m_result_delegate.RegisterPersistentState(persistent_state);
     } else {
       diagnostic_manager.PutString(
           eDiagnosticSeverityError,
           "couldn't start parsing (no persistent data)");
       return false;
     }
   } else {
void ClangUserExpression::UpdateLanguageForExpr() {
   else
     m_expr_lang = lldb::eLanguageTypeC;
 }
 
 void ClangUserExpression::CreateSourceCode(
     DiagnosticManager &diagnostic_manager, ExecutionContext &exe_ctx,
     std::vector<std::string> modules_to_import, bool for_completion) {
 
+  m_filename = m_clang_state->GetNextExprFileName();
   std::string prefix = m_expr_prefix;
 
   if (m_options.GetExecutionPolicy() == eExecutionPolicyTopLevel) {
     m_transformed_text = m_expr_text;
   } else {
-    std::unique_ptr<ClangExpressionSourceCode> source_code(
-        ClangExpressionSourceCode::CreateWrapped(prefix.c_str(),
-                                                 m_expr_text.c_str()));
+    m_source_code.reset(ClangExpressionSourceCode::CreateWrapped(
+        m_filename, prefix.c_str(), m_expr_text.c_str()));
 
-    if (!source_code->GetText(m_transformed_text, m_expr_lang,
+    if (!m_source_code->GetText(m_transformed_text, m_expr_lang,
                                 m_in_static_method, exe_ctx, !m_ctx_obj,
                                 for_completion, modules_to_import)) {
       diagnostic_manager.PutString(eDiagnosticSeverityError,
                                    "couldn't construct expression body");
       return;
     }
 
     // Find and store the start position of the original code inside the
     // transformed code. We need this later for the code completion.
     std::size_t original_start;
     std::size_t original_end;
-    bool found_bounds = source_code->GetOriginalBodyBounds(
+    bool found_bounds = m_source_code->GetOriginalBodyBounds(
         m_transformed_text, m_expr_lang, original_start, original_end);
     if (found_bounds)
       m_user_expression_start_pos = original_start;
   }
 }
 
 static bool SupportsCxxModuleImport(lldb::LanguageType language) {
   switch (language) {
bool ClangUserExpression::Parse(DiagnosticManager &diagnostic_manager,
   if (!exe_scope)
     exe_scope = exe_ctx.GetTargetPtr();
 
   // We use a shared pointer here so we can use the original parser - if it
   // succeeds or the rewrite parser we might make if it fails.  But the
   // parser_sp will never be empty.
 
   ClangExpressionParser parser(exe_scope, *this, generate_debug_info,
-                               m_include_directories);
+                               m_include_directories, m_filename);
 
   unsigned num_errors = parser.Parse(diagnostic_manager);
 
   // Check here for FixItHints.  If there are any try to apply the fixits and
   // set the fixed text in m_fixed_text before returning an error.
   if (num_errors) {
     if (diagnostic_manager.HasFixIts()) {
       if (parser.RewriteExpression(diagnostic_manager)) {
         size_t fixed_start;
         size_t fixed_end;
         const std::string &fixed_expression =
             diagnostic_manager.GetFixedExpression();
-        if (ClangExpressionSourceCode::GetOriginalBodyBounds(
-                fixed_expression, m_expr_lang, fixed_start, fixed_end))
+        if (m_source_code->GetOriginalBodyBounds(fixed_expression, m_expr_lang,
+                                                 fixed_start, fixed_end))
           m_fixed_text =
               fixed_expression.substr(fixed_start, fixed_end - fixed_start);
       }
     }
     return false;
   }
 
   //////////////////////////////////////////////////////////////////////////////