Adding support to step into the callable wrapped by libc++ std::function

[shafik]

Summary

• Adding support to step into the callable wrapped by libc++ std::function
• Adding test to validate that functionality

[davide]

Some basic comments. Haven't looked at the implementation very closely, I'll do that probably tomorrow. Thanks for working on this!

include/lldb/Target/CPPLanguageRuntime.h

59–60

Please add a doxygen comment to this function. (I understand the surrounding ones are not commented, but we should start somewhere)/

packages/Python/lldbsuite/test/lang/cpp/std-function-step-into-callable/TestStdFunctionStepIntoCallable.py

1–3

This comment seems a little outdated.

8–9

I'm not sure you need these.

27–28

Is this affected by the debug info variant used? (i.e. dSYM/gmodules/DWARF/dwo).
If not, this could be a NO_DEBUG_INFO test

56–73

All these lines check hardcoded make me a little nervous, as they're really fragile. Can you make them parametric at least?
so that if somebody slides down the test case they're still valid (i.e. base + offset)

packages/Python/lldbsuite/test/lang/cpp/std-function-step-into-callable/main.cpp

1–9

You don't need a license for the test case, I think.

source/Target/ThreadPlanStepThrough.cpp

99–105

Can you add a comment here?

[shafik]

Addressing comments and fixing a bug in ThreadPlanStepThrough.cpp where I would overwrite the result of objc_runtime.

[jingham]

Just a couple of trivial requests, mostly about comments...

packages/Python/lldbsuite/test/lang/cpp/std-function-step-into-callable/TestStdFunctionStepIntoCallable.py

27–28

I don't think this test will be sensitive to those differences, so this probably can be a NO_DEBUG_INFO test.

46–62

Most of this setup (excepting the line_number calls) can be done in one line using lldbutils.run_to_source_breakpoint.

65

Can you also check the source file? The way libcxx does inlining nowadays, all these std::function calls introduce lots of inlining so the chance that you got to the right line but in an inlined function is not zero...

source/Target/CPPLanguageRuntime.cpp

295–301

I'm not sure this comment is helpful. You want to do the trampoline detection regardless of the value of the regexp. And we really aren't violating that contract, since we don't intent to STOP on step into std::function...

329

You need to explain what this step in is about. Otherwise this is pretty mysterious.

[shafik]

• Addressing comments
• Adding test to make sure we step through a std::function wrapping a member variable

[shafik]

@jingham I believe I addressed your comments, please review again.

[jingham]

Couple more lines you can delete from the test case, and I think you should make this a debug-variant insensitive test. Do that and this is good.

packages/Python/lldbsuite/test/lang/cpp/std-function-step-into-callable/TestStdFunctionStepIntoCallable.py

30–31

You don't need these two lines anymore. You never use the exe variable, and the target you make with this "file" command is never used (you correctly use the one returned by run_to_source_breakpoint.)

Also, this doesn't seem like a debug variant sensitive test, so you really should make this a NO_DEBUG_INFO test. We're trying to do that wherever it makes sense just to keep down the number of tests we run.

[shafik]

Removing uneeded code and making test NO_DEBUG_INFO_TESTCASE

[shafik]

@jingham removed code and made it NO_DEBUG_INFO_TESTCASE

[jingham]

This looks great, thanks for making this work, it will be SO helpful for debugging std::function uses.

[davide]

LGTM too. Thanks for taking the time to address the comments.

[stella.stamenova]

This change has the same problem as D49271 - the class name LibCxxFunctionTestCase is re-used from another test case. This will cause issues when the tests report results and create logs. Please use unique names for each class and test.

include/lldb/Target/CPPLanguageRuntime.h

public:
 
   virtual bool IsVTableName(const char *name) = 0;
 
   bool GetObjectDescription(Stream &str, ValueObject &object) override;
 
   bool GetObjectDescription(Stream &str, Value &value,
                             ExecutionContextScope *exe_scope) override;
 
+  lldb::ThreadPlanSP GetStepThroughTrampolinePlan(Thread &thread,
+                                                  bool stop_others);
+
 protected:
   //------------------------------------------------------------------
   // Classes that inherit from CPPLanguageRuntime can see and modify these
   //------------------------------------------------------------------
   CPPLanguageRuntime(Process *process);
 
 private:
   DISALLOW_COPY_AND_ASSIGN(CPPLanguageRuntime);
 };
 
 } // namespace lldb_private
 
 #endif // liblldb_CPPLanguageRuntime_h_

packages/Python/lldbsuite/test/lang/cpp/std-function-step-into-callable/Makefile

+LEVEL = ../../../make
+
+CXX_SOURCES := main.cpp
+CXXFLAGS += -std=c++11
+USE_LIBCPP := 1
+
+include $(LEVEL)/Makefile.rules

packages/Python/lldbsuite/test/lang/cpp/std-function-step-into-callable/TestStdFunctionStepIntoCallable.py

+"""
+Test lldb data formatter subsystem.
+"""
+
+from __future__ import print_function
+
+
+import os
+import time
+import lldb
+import sys
+from lldbsuite.test.decorators import *
+from lldbsuite.test.lldbtest import *
+from lldbsuite.test import lldbutil
+
+
+class LibCxxFunctionTestCase(TestBase):
+
+    mydir = TestBase.compute_mydir(__file__)
+
+    def get_variable(self, name):
+        var = self.frame().FindVariable(name)
+        var.SetPreferDynamicValue(lldb.eDynamicCanRunTarget)
+        var.SetPreferSyntheticValue(True)
+        return var
+
+    @add_test_categories(["libc++"])
+    def test(self):
+        """Test that std::function as defined by libc++ is correctly printed by LLDB"""
+        self.build()
+        self.runCmd("file " + self.getBuildArtifact("a.out"), CURRENT_EXECUTABLE_SET)
+
+        exe = self.getBuildArtifact("a.out")
+        self.main_source = "main.cpp"
+        self.main_source_spec = lldb.SBFileSpec(self.main_source)
+
+        target = self.dbg.CreateTarget(exe)
+        self.assertTrue(target, VALID_TARGET)
+
+        break_in_main = target.BreakpointCreateBySourceRegex('// Set break point at this line.', self.main_source_spec)
+        self.assertTrue(break_in_main, VALID_BREAKPOINT)
+
+        self.process = target.LaunchSimple(
+            None, None, self.get_process_working_directory())
+
+        self.assertTrue(self.process, PROCESS_IS_VALID)
+
+        threads = lldbutil.get_stopped_threads(
+            self.process, lldb.eStopReasonBreakpoint)
+        self.assertTrue(
+            len(threads) == 1,
+            "Successfully ran to breakpoint.")
+        self.thread = threads[0]
+
+        self.thread.StepInto()
+        self.assertEqual( self.thread.GetFrameAtIndex(0).GetLineEntry().GetLine(), 13 ) ;
+        self.process.Continue()
+
+        self.thread.StepInto()
+        self.assertEqual( self.thread.GetFrameAtIndex(0).GetLineEntry().GetLine(), 28 ) ;
+        self.process.Continue()
+
+        self.thread.StepInto()
+        self.assertEqual( self.thread.GetFrameAtIndex(0).GetLineEntry().GetLine(), 31 ) ;
+        self.process.Continue()
+
+        self.thread.StepInto()
+        self.assertEqual( self.thread.GetFrameAtIndex(0).GetLineEntry().GetLine(), 18 ) ;
+        self.process.Continue()
+
+        self.thread.StepInto()
+        self.assertEqual( self.thread.GetFrameAtIndex(0).GetLineEntry().GetLine(), 20 ) ;
+        self.process.Continue()

packages/Python/lldbsuite/test/lang/cpp/std-function-step-into-callable/main.cpp

+//===-- main.cpp --------------------------------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include <functional>
+
+int foo(int x, int y) {
+  return x + y - 1;
+}
+
+struct Bar {
+   int operator()() {
+       return 66 ;
+   }
+   int add_num(int i) const { return i + 3 ; }
+} ;
+
+int main (int argc, char *argv[])
+{
+  int acc = 42;
+  std::function<int (int,int)> f1 = foo;
+  std::function<int (int)> f2 = [acc,f1] (int x) -> int {
+    return x+f1(acc,x);
+  };
+
+  auto f = [](int x, int y) { return x + y; };
+  auto g = [](int x, int y) { return x * y; } ;
+  std::function<int (int,int)> f3 =  argc %2 ? f : g ;
+
+  Bar bar1 ;
+  std::function<int ()> f4( bar1 ) ;
+  std::function<int (const Bar&, int)> f5 = &Bar::add_num;
+
+  return f1(acc,acc) +     // Set break point at this line.
+         f2(acc) +         // Set break point at this line.
+         f3(acc+1,acc+2) + // Set break point at this line. 
+         f4() +            // Set break point at this line. 
+         f5(bar1, 10);     // Set break point at this line.
+}

source/Target/CPPLanguageRuntime.cpp

 #include "lldb/Core/UniqueCStringMap.h"
 #include "lldb/Symbol/ClangASTContext.h"
 #include "lldb/Target/ABI.h"
 #include "lldb/Target/ExecutionContext.h"
 #include "lldb/Target/RegisterContext.h"
 #include "lldb/Target/SectionLoadList.h"
 #include "lldb/Target/StackFrame.h"
 #include "lldb/Target/ThreadPlanRunToAddress.h"
+#include "lldb/Target/ThreadPlanStepInRange.h"
 
 using namespace lldb;
 using namespace lldb_private;
 
 //----------------------------------------------------------------------
 // Destructor
 //----------------------------------------------------------------------
 CPPLanguageRuntime::~CPPLanguageRuntime() {}
CPPLanguageRuntime::FindLibCppStdFunctionCallableInfo(
   // template parameter
   //
   //  ... __func<main::$_0, std::__1::allocator<main::$_0> ...
   //             ^^^^^^^^^
   //
   // We do this by find the first < and , and extracting in between.
   //
   // This covers the case of the lambda known at compile time.
-  //
   size_t first_open_angle_bracket = vtable_name.find('<') + 1;
   size_t first_comma = vtable_name.find_first_of(',');
 
   llvm::StringRef first_template_parameter =
       vtable_name.slice(first_open_angle_bracket, first_comma);
 
   Address function_address_resolved;
 
if (!symbol->GetName().GetStringRef().startswith("vtable for")) {
     optional_info.callable_address = function_address_resolved;
     optional_info.callable_symbol = *symbol;
 
     return optional_info;
   }
 
   return optional_info;
 }
+
+lldb::ThreadPlanSP
+CPPLanguageRuntime::GetStepThroughTrampolinePlan(Thread &thread,
+                                                 bool stop_others) {
+  ThreadPlanSP ret_plan_sp;
+
+  lldb::addr_t curr_pc = thread.GetRegisterContext()->GetPC();
+
+  TargetSP target_sp(thread.CalculateTarget());
+
+  if (target_sp->GetSectionLoadList().IsEmpty())
+    return ret_plan_sp;
+
+  Address pc_addr_resolved;
+  SymbolContext sc;
+  Symbol *symbol;
+
+  if (!target_sp->GetSectionLoadList().ResolveLoadAddress(curr_pc,
+                                                          pc_addr_resolved))
+    return ret_plan_sp;
+
+  target_sp->GetImages().ResolveSymbolContextForAddress(
+      pc_addr_resolved, eSymbolContextEverything, sc);
+  symbol = sc.symbol;
+
+  if (symbol == nullptr)
+    return ret_plan_sp;
+
+  llvm::StringRef function_name(symbol->GetName().GetCString());
+
+  // Handling the case where we are attempting to step into std::function.
+  // Currently due to the the:
+  //
+  //    target.process.thread.step-avoid-regexp
+  //
+  // setting we will currenly step right out of standard library code.
+  //
+  // The new behavior will be that we will attempt to obtain the wrapped
+  // callable via FindLibCppStdFunctionCallableInfo() and if we find it we
+  // will return a ThreadPlanRunToAddress to the callable. Therefore we will
+  // step into the wrapped callable.
+  //
+  bool found_expected_start_string =
+      function_name.startswith("std::__1::function<");
+
+  if (!found_expected_start_string)
+    return ret_plan_sp;
+
+  AddressRange range_of_curr_func;
+  sc.GetAddressRange(eSymbolContextEverything, 0, false, range_of_curr_func);
+
+  StackFrameSP frame = thread.GetStackFrameAtIndex(0);
+
+  if (frame) {
+    ValueObjectSP value_sp = frame->FindVariable(ConstString("this"));
+
+    CPPLanguageRuntime::LibCppStdFunctionCallableInfo callable_info =
+        FindLibCppStdFunctionCallableInfo(value_sp);
+
+    if (callable_info.callable_case != LibCppStdFunctionCallableCase::Invalid &&
+        value_sp->GetValueIsValid()) {
+      ret_plan_sp.reset(new ThreadPlanRunToAddress(
+          thread, callable_info.callable_address, stop_others));
+      return ret_plan_sp;
+    } else {
+      ret_plan_sp.reset(new ThreadPlanStepInRange(thread, range_of_curr_func,
+                                                  sc, eOnlyThisThread,
+                                                  eLazyBoolYes, eLazyBoolYes));
+      return ret_plan_sp;
+    }
+  }
+
+  return ret_plan_sp;
+}

source/Target/ThreadPlanStepThrough.cpp

 //===-- ThreadPlanStepThrough.cpp -------------------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 // C Includes
 // C++ Includes
 // Other libraries and framework includes
 // Project includes
 #include "lldb/Target/ThreadPlanStepThrough.h"
 #include "lldb/Breakpoint/Breakpoint.h"
+#include "lldb/Target/CPPLanguageRuntime.h"
 #include "lldb/Target/DynamicLoader.h"
 #include "lldb/Target/ObjCLanguageRuntime.h"
 #include "lldb/Target/Process.h"
 #include "lldb/Target/RegisterContext.h"
 #include "lldb/Target/Target.h"
 #include "lldb/Utility/Log.h"
 #include "lldb/Utility/Stream.h"
 
void ThreadPlanStepThrough::LookForPlanToStepThroughFromCurrentPC() {
 
   // If that didn't come up with anything, try the ObjC runtime plugin:
   if (!m_sub_plan_sp.get()) {
     ObjCLanguageRuntime *objc_runtime =
         m_thread.GetProcess()->GetObjCLanguageRuntime();
     if (objc_runtime)
       m_sub_plan_sp =
           objc_runtime->GetStepThroughTrampolinePlan(m_thread, m_stop_others);
+
+    CPPLanguageRuntime *cpp_runtime =
+        m_thread.GetProcess()->GetCPPLanguageRuntime();
+
+    if (cpp_runtime)
+      m_sub_plan_sp =
+          cpp_runtime->GetStepThroughTrampolinePlan(m_thread, m_stop_others);
   }
 
   Log *log(lldb_private::GetLogIfAllCategoriesSet(LIBLLDB_LOG_STEP));
   if (log) {
     lldb::addr_t current_address = GetThread().GetRegisterContext()->GetPC(0);
     if (m_sub_plan_sp) {
       StreamString s;
       m_sub_plan_sp->GetDescription(&s, lldb::eDescriptionLevelFull);