Adding support to step into the callable wrapped by libc++ std::function

[shafik]

Summary

• Adding support to step into the callable wrapped by libc++ std::function
• Adding test to validate that functionality

[davide]

Some basic comments. Haven't looked at the implementation very closely, I'll do that probably tomorrow. Thanks for working on this!

include/lldb/Target/CPPLanguageRuntime.h

59–60 ↗

(On Diff #168196)

Please add a doxygen comment to this function. (I understand the surrounding ones are not commented, but we should start somewhere)/

packages/Python/lldbsuite/test/lang/cpp/std-function-step-into-callable/TestStdFunctionStepIntoCallable.py

1–3 ↗

(On Diff #168196)

This comment seems a little outdated.

8–9 ↗

(On Diff #168196)

I'm not sure you need these.

27–28 ↗

(On Diff #168196)

Is this affected by the debug info variant used? (i.e. dSYM/gmodules/DWARF/dwo).
If not, this could be a NO_DEBUG_INFO test

56–73 ↗

(On Diff #168196)

All these lines check hardcoded make me a little nervous, as they're really fragile. Can you make them parametric at least?
so that if somebody slides down the test case they're still valid (i.e. base + offset)

packages/Python/lldbsuite/test/lang/cpp/std-function-step-into-callable/main.cpp

1–9 ↗

(On Diff #168196)

You don't need a license for the test case, I think.

source/Target/ThreadPlanStepThrough.cpp

99–105 ↗

(On Diff #168196)

Can you add a comment here?

[shafik]

Addressing comments and fixing a bug in ThreadPlanStepThrough.cpp where I would overwrite the result of objc_runtime.

[jingham]

Just a couple of trivial requests, mostly about comments...

packages/Python/lldbsuite/test/lang/cpp/std-function-step-into-callable/TestStdFunctionStepIntoCallable.py

45–61 ↗

(On Diff #168389)

Most of this setup (excepting the line_number calls) can be done in one line using lldbutils.run_to_source_breakpoint.

64 ↗

(On Diff #168389)

Can you also check the source file? The way libcxx does inlining nowadays, all these std::function calls introduce lots of inlining so the chance that you got to the right line but in an inlined function is not zero...

27–28 ↗

(On Diff #168196)

I don't think this test will be sensitive to those differences, so this probably can be a NO_DEBUG_INFO test.

source/Target/CPPLanguageRuntime.cpp

295–301 ↗

(On Diff #168389)

I'm not sure this comment is helpful. You want to do the trampoline detection regardless of the value of the regexp. And we really aren't violating that contract, since we don't intent to STOP on step into std::function...

329 ↗

(On Diff #168389)

You need to explain what this step in is about. Otherwise this is pretty mysterious.

[shafik]

• Addressing comments
• Adding test to make sure we step through a std::function wrapping a member variable

[shafik]

@jingham I believe I addressed your comments, please review again.

[jingham]

Couple more lines you can delete from the test case, and I think you should make this a debug-variant insensitive test. Do that and this is good.

packages/Python/lldbsuite/test/lang/cpp/std-function-step-into-callable/TestStdFunctionStepIntoCallable.py

29–30 ↗

(On Diff #169105)

You don't need these two lines anymore. You never use the exe variable, and the target you make with this "file" command is never used (you correctly use the one returned by run_to_source_breakpoint.)

Also, this doesn't seem like a debug variant sensitive test, so you really should make this a NO_DEBUG_INFO test. We're trying to do that wherever it makes sense just to keep down the number of tests we run.

[shafik]

Removing uneeded code and making test NO_DEBUG_INFO_TESTCASE

[shafik]

@jingham removed code and made it NO_DEBUG_INFO_TESTCASE

[jingham]

This looks great, thanks for making this work, it will be SO helpful for debugging std::function uses.

[davide]

LGTM too. Thanks for taking the time to address the comments.

[stella.stamenova]

This change has the same problem as D49271 - the class name LibCxxFunctionTestCase is re-used from another test case. This will cause issues when the tests report results and create logs. Please use unique names for each class and test.

lldb/trunk/include/lldb/Target/CPPLanguageRuntime.h

public:
 
   virtual bool IsVTableName(const char *name) = 0;
 
   bool GetObjectDescription(Stream &str, ValueObject &object) override;
 
   bool GetObjectDescription(Stream &str, Value &value,
                             ExecutionContextScope *exe_scope) override;
 
+  /// Obtain a ThreadPlan to get us into C++ constructs such as std::function.
+  ///
+  /// @param[in] thread
+  ///     Curent thrad of execution.
+  ///
+  /// @param[in] stop_others
+  ///     True if other threads should pause during execution.
+  ///
+  /// @return
+  ///      A ThreadPlan Shared pointer
+  lldb::ThreadPlanSP GetStepThroughTrampolinePlan(Thread &thread,
+                                                  bool stop_others);
+
 protected:
   //------------------------------------------------------------------
   // Classes that inherit from CPPLanguageRuntime can see and modify these
   //------------------------------------------------------------------
   CPPLanguageRuntime(Process *process);
 
 private:
   DISALLOW_COPY_AND_ASSIGN(CPPLanguageRuntime);
 };
 
 } // namespace lldb_private
 
 #endif // liblldb_CPPLanguageRuntime_h_

lldb/trunk/packages/Python/lldbsuite/test/lang/cpp/std-function-step-into-callable/Makefile

+LEVEL = ../../../make
+
+CXX_SOURCES := main.cpp
+CXXFLAGS += -std=c++11
+USE_LIBCPP := 1
+
+include $(LEVEL)/Makefile.rules

lldb/trunk/packages/Python/lldbsuite/test/lang/cpp/std-function-step-into-callable/TestStdFunctionStepIntoCallable.py

+"""
+Test stepping into std::function
+"""
+
+from __future__ import print_function
+
+
+import lldb
+import sys
+from lldbsuite.test.decorators import *
+from lldbsuite.test.lldbtest import *
+from lldbsuite.test import lldbutil
+
+
+class LibCxxFunctionTestCase(TestBase):
+
+    mydir = TestBase.compute_mydir(__file__)
+
+    NO_DEBUG_INFO_TESTCASE = True
+
+    @add_test_categories(["libc++"])
+    def test(self):
+        """Test that std::function as defined by libc++ is correctly printed by LLDB"""
+        self.build()
+
+        self.main_source = "main.cpp"
+        self.main_source_spec = lldb.SBFileSpec(self.main_source)
+        self.source_foo_line = line_number(
+            self.main_source, '// Source foo start line')
+        self.source_lambda_f2_line = line_number(
+            self.main_source, '// Source lambda used by f2 start line')
+        self.source_lambda_f3_line = line_number(
+            self.main_source, '// Source lambda used by f3 start line')
+        self.source_bar_operator_line = line_number(
+            self.main_source, '// Source Bar::operator()() start line')
+        self.source_bar_add_num_line = line_number(
+            self.main_source, '// Source Bar::add_num start line')
+        self.source_main_invoking_f1 = line_number(
+            self.main_source, '// Source main invoking f1')
+
+        (target, process, thread, bkpt) = lldbutil.run_to_source_breakpoint(
+            self, "// Set break point at this line.", self.main_source_spec)
+
+        thread.StepInto()
+        self.assertEqual( thread.GetFrameAtIndex(0).GetLineEntry().GetLine(), self.source_main_invoking_f1 ) ;
+        self.assertEqual( thread.GetFrameAtIndex(0).GetLineEntry().GetFileSpec().GetFilename(), self.main_source) ;
+
+        thread.StepInto()
+        self.assertEqual( thread.GetFrameAtIndex(0).GetLineEntry().GetLine(), self.source_foo_line ) ;
+        self.assertEqual( thread.GetFrameAtIndex(0).GetLineEntry().GetFileSpec().GetFilename(), self.main_source) ;
+        process.Continue()
+
+        thread.StepInto()
+        self.assertEqual( thread.GetFrameAtIndex(0).GetLineEntry().GetLine(), self.source_lambda_f2_line ) ;
+        self.assertEqual( thread.GetFrameAtIndex(0).GetLineEntry().GetFileSpec().GetFilename(), self.main_source) ;
+        process.Continue()
+
+        thread.StepInto()
+        self.assertEqual( thread.GetFrameAtIndex(0).GetLineEntry().GetLine(), self.source_lambda_f3_line ) ;
+        self.assertEqual( thread.GetFrameAtIndex(0).GetLineEntry().GetFileSpec().GetFilename(), self.main_source) ;
+        process.Continue()
+
+        thread.StepInto()
+        self.assertEqual( thread.GetFrameAtIndex(0).GetLineEntry().GetLine(), self.source_bar_operator_line ) ;
+        self.assertEqual( thread.GetFrameAtIndex(0).GetLineEntry().GetFileSpec().GetFilename(), self.main_source) ;
+        process.Continue()
+
+        thread.StepInto()
+        self.assertEqual( thread.GetFrameAtIndex(0).GetLineEntry().GetLine(), self.source_bar_add_num_line ) ;
+        self.assertEqual( thread.GetFrameAtIndex(0).GetLineEntry().GetFileSpec().GetFilename(), self.main_source) ;
+        process.Continue()

lldb/trunk/packages/Python/lldbsuite/test/lang/cpp/std-function-step-into-callable/main.cpp

+#include <functional>
+
+int foo(int x, int y) {
+  return x + y - 1; // Source foo start line
+}
+
+struct Bar {
+   int operator()() {
+       return 66 ; // Source Bar::operator()() start line
+   }
+   int add_num(int i) const { return i + 3 ; } // Source Bar::add_num start line
+   int num_ = 0 ;
+} ;
+
+int main (int argc, char *argv[])
+{
+  int acc = 42;
+  std::function<int (int,int)> f1 = foo;
+  std::function<int (int)> f2 = [acc,f1] (int x) -> int {
+    return x+f1(acc,x); // Source lambda used by f2 start line
+  };
+
+  auto f = [](int x, int y) { return x + y; }; // Source lambda used by f3 start line
+  auto g = [](int x, int y) { return x * y; } ;
+  std::function<int (int,int)> f3 =  argc %2 ? f : g ;
+
+  Bar bar1 ;
+  std::function<int ()> f4( bar1 ) ;
+  std::function<int (const Bar&, int)> f5 = &Bar::add_num;
+  std::function<int(Bar const&)> f_mem = &Bar::num_;
+
+  return f_mem(bar1) +     // Set break point at this line.
+         f1(acc,acc) +     // Source main invoking f1
+         f2(acc) +         // Set break point at this line.
+         f3(acc+1,acc+2) + // Set break point at this line. 
+         f4() +            // Set break point at this line. 
+         f5(bar1, 10);     // Set break point at this line.
+}

lldb/trunk/source/Target/CPPLanguageRuntime.cpp

 #include "lldb/Core/UniqueCStringMap.h"
 #include "lldb/Symbol/ClangASTContext.h"
 #include "lldb/Target/ABI.h"
 #include "lldb/Target/ExecutionContext.h"
 #include "lldb/Target/RegisterContext.h"
 #include "lldb/Target/SectionLoadList.h"
 #include "lldb/Target/StackFrame.h"
 #include "lldb/Target/ThreadPlanRunToAddress.h"
+#include "lldb/Target/ThreadPlanStepInRange.h"
 
 using namespace lldb;
 using namespace lldb_private;
 
 //----------------------------------------------------------------------
 // Destructor
 //----------------------------------------------------------------------
 CPPLanguageRuntime::~CPPLanguageRuntime() {}
CPPLanguageRuntime::FindLibCppStdFunctionCallableInfo(
   // template parameter
   //
   //  ... __func<main::$_0, std::__1::allocator<main::$_0> ...
   //             ^^^^^^^^^
   //
   // We do this by find the first < and , and extracting in between.
   //
   // This covers the case of the lambda known at compile time.
-  //
   size_t first_open_angle_bracket = vtable_name.find('<') + 1;
   size_t first_comma = vtable_name.find_first_of(',');
 
   llvm::StringRef first_template_parameter =
       vtable_name.slice(first_open_angle_bracket, first_comma);
 
   Address function_address_resolved;
 
if (!symbol->GetName().GetStringRef().startswith("vtable for")) {
     optional_info.callable_address = function_address_resolved;
     optional_info.callable_symbol = *symbol;
 
     return optional_info;
   }
 
   return optional_info;
 }
+
+lldb::ThreadPlanSP
+CPPLanguageRuntime::GetStepThroughTrampolinePlan(Thread &thread,
+                                                 bool stop_others) {
+  ThreadPlanSP ret_plan_sp;
+
+  lldb::addr_t curr_pc = thread.GetRegisterContext()->GetPC();
+
+  TargetSP target_sp(thread.CalculateTarget());
+
+  if (target_sp->GetSectionLoadList().IsEmpty())
+    return ret_plan_sp;
+
+  Address pc_addr_resolved;
+  SymbolContext sc;
+  Symbol *symbol;
+
+  if (!target_sp->GetSectionLoadList().ResolveLoadAddress(curr_pc,
+                                                          pc_addr_resolved))
+    return ret_plan_sp;
+
+  target_sp->GetImages().ResolveSymbolContextForAddress(
+      pc_addr_resolved, eSymbolContextEverything, sc);
+  symbol = sc.symbol;
+
+  if (symbol == nullptr)
+    return ret_plan_sp;
+
+  llvm::StringRef function_name(symbol->GetName().GetCString());
+
+  // Handling the case where we are attempting to step into std::function.
+  // The behavior will be that we will attempt to obtain the wrapped
+  // callable via FindLibCppStdFunctionCallableInfo() and if we find it we
+  // will return a ThreadPlanRunToAddress to the callable. Therefore we will
+  // step into the wrapped callable.
+  //
+  bool found_expected_start_string =
+      function_name.startswith("std::__1::function<");
+
+  if (!found_expected_start_string)
+    return ret_plan_sp;
+
+  AddressRange range_of_curr_func;
+  sc.GetAddressRange(eSymbolContextEverything, 0, false, range_of_curr_func);
+
+  StackFrameSP frame = thread.GetStackFrameAtIndex(0);
+
+  if (frame) {
+    ValueObjectSP value_sp = frame->FindVariable(ConstString("this"));
+
+    CPPLanguageRuntime::LibCppStdFunctionCallableInfo callable_info =
+        FindLibCppStdFunctionCallableInfo(value_sp);
+
+    if (callable_info.callable_case != LibCppStdFunctionCallableCase::Invalid &&
+        value_sp->GetValueIsValid()) {
+      // We found the std::function wrapped callable and we have its address.
+      // We now create a ThreadPlan to run to the callable.
+      ret_plan_sp.reset(new ThreadPlanRunToAddress(
+          thread, callable_info.callable_address, stop_others));
+      return ret_plan_sp;
+    } else {
+      // We are in std::function but we could not obtain the callable.
+      // We create a ThreadPlan to keep stepping through using the address range
+      // of the current function.
+      ret_plan_sp.reset(new ThreadPlanStepInRange(thread, range_of_curr_func,
+                                                  sc, eOnlyThisThread,
+                                                  eLazyBoolYes, eLazyBoolYes));
+      return ret_plan_sp;
+    }
+  }
+
+  return ret_plan_sp;
+}

lldb/trunk/source/Target/ThreadPlanStepThrough.cpp

 //===-- ThreadPlanStepThrough.cpp -------------------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 // C Includes
 // C++ Includes
 // Other libraries and framework includes
 // Project includes
 #include "lldb/Target/ThreadPlanStepThrough.h"
 #include "lldb/Breakpoint/Breakpoint.h"
+#include "lldb/Target/CPPLanguageRuntime.h"
 #include "lldb/Target/DynamicLoader.h"
 #include "lldb/Target/ObjCLanguageRuntime.h"
 #include "lldb/Target/Process.h"
 #include "lldb/Target/RegisterContext.h"
 #include "lldb/Target/Target.h"
 #include "lldb/Utility/Log.h"
 #include "lldb/Utility/Stream.h"
 
void ThreadPlanStepThrough::LookForPlanToStepThroughFromCurrentPC() {
 
   // If that didn't come up with anything, try the ObjC runtime plugin:
   if (!m_sub_plan_sp.get()) {
     ObjCLanguageRuntime *objc_runtime =
         m_thread.GetProcess()->GetObjCLanguageRuntime();
     if (objc_runtime)
       m_sub_plan_sp =
           objc_runtime->GetStepThroughTrampolinePlan(m_thread, m_stop_others);
+
+    CPPLanguageRuntime *cpp_runtime =
+        m_thread.GetProcess()->GetCPPLanguageRuntime();
+
+    // If the ObjC runtime did not provide us with a step though plan then if we
+    // have it check the C++ runtime for a step though plan.
+    if (!m_sub_plan_sp.get() && cpp_runtime)
+      m_sub_plan_sp =
+          cpp_runtime->GetStepThroughTrampolinePlan(m_thread, m_stop_others);
   }
 
   Log *log(lldb_private::GetLogIfAllCategoriesSet(LIBLLDB_LOG_STEP));
   if (log) {
     lldb::addr_t current_address = GetThread().GetRegisterContext()->GetPC(0);
     if (m_sub_plan_sp) {
       StreamString s;
       m_sub_plan_sp->GetDescription(&s, lldb::eDescriptionLevelFull);