[LLDB] Add formatters for PointerIntPair, PointerUnion

[broadwaylamb]

Summary

Also, add summaries for SmallVector and ArrayRef,
and fix the StringRef summary provider so it doesn't ignore the Length field.

[kastiglione]

Thanks for doing this! I have not used PointerIntPair or PointerUnion, but I will take a look at those types. I take it these expression evaluations are necessary?

llvm/utils/lldbDataFormatters.py

176–179

when the string length is greater than max_length, it would be nice for the returned string to indicate truncation. I just checked a long std::string and the output was:

"really long string"...

Also the max length for std::string is 1024, should StringRef use 1024 too?

186–187

should this print an error message?

188

what's the best way to handle quotes inside the string? A quick search shows this could use json (a hack):

return json.dumps(string)

for example:

json.dumps('hello world') # '"hello world"'
json.dumps('hello "quoted" world') # '"hello \\"quoted\\" world"'

[broadwaylamb]

Address review comments

[broadwaylamb]

In D117779#3258798, @kastiglione wrote:

I take it these expression evaluations are necessary?

Well, I think they are. There is too much template magic behind figuring out how many bits in the pointer we use to store an additional integer value.

Expression evaluations are not only slow, but they also prevent us from formatting nested PointerIntPairs, since the expression path is modified and doesn't correspond to the actual implementation anymore, therefore evaluating expressions down the tree will not work.
But I guess we can live with that for now.

llvm/utils/lldbDataFormatters.py

176–179

LLDB supports the max-string-summary-length setting, but unfortunately there is no way to get the value of that setting through Python API.

[kastiglione]

But I guess we can live with that for now.

That's fair. Thanks again for adding more llvm formatters.

[kastiglione]

llvm/utils/lldbDataFormatters.py

185

For some binary strings, decode will thrown an exception.

[kastiglione]

@broadwaylamb hi, have you been using the PointerUnion formatter? We have seen a variety of issues, which don't seem to have easy fixes. If you're not using it, I will disable it. Alternatively, I propose creating a separate file that you and others could choose to import separately from lldbDataFormatters.py.

[broadwaylamb]

In D117779#4497748, @kastiglione wrote:

@broadwaylamb hi, have you been using the PointerUnion formatter? We have seen a variety of issues, which don't seem to have easy fixes. If you're not using it, I will disable it. Alternatively, I propose creating a separate file that you and others could choose to import separately from lldbDataFormatters.py.

No, I haven't been working with LLVM for a while.

[kastiglione]

Thanks, I will disable these for now, and see what can be done to address the issues we've seen.

llvm/utils/lldbDataFormatters.py

 """
 LLDB Formatters for LLVM data types.
 
 Load into LLDB with 'command script import /path/to/lldbDataFormatters.py'
 """
 
 import lldb
+import json
 
 def __lldb_init_module(debugger, internal_dict):
     debugger.HandleCommand('type category define -e llvm -l c++')
     debugger.HandleCommand('type synthetic add -w llvm '
                            '-l lldbDataFormatters.SmallVectorSynthProvider '
                            '-x "^llvm::SmallVectorImpl<.+>$"')
+    debugger.HandleCommand('type summary add -w llvm '
+                           '-s "size=${svar%#}" '
+                           '-x "^llvm::SmallVectorImpl<.+>$"')
     debugger.HandleCommand('type synthetic add -w llvm '
                            '-l lldbDataFormatters.SmallVectorSynthProvider '
                            '-x "^llvm::SmallVector<.+,.+>$"')
+    debugger.HandleCommand('type summary add -w llvm '
+                           '-s "size=${svar%#}" '
+                           '-x "^llvm::SmallVector<.+,.+>$"')
     debugger.HandleCommand('type synthetic add -w llvm '
                            '-l lldbDataFormatters.ArrayRefSynthProvider '
                            '-x "^llvm::ArrayRef<.+>$"')
+    debugger.HandleCommand('type summary add -w llvm '
+                           '-s "size=${svar%#}" '
+                           '-x "^llvm::ArrayRef<.+>$"')
     debugger.HandleCommand('type synthetic add -w llvm '
                            '-l lldbDataFormatters.OptionalSynthProvider '
                            '-x "^llvm::Optional<.+>$"')
     debugger.HandleCommand('type summary add -w llvm '
                            '-F lldbDataFormatters.OptionalSummaryProvider '
                            '-x "^llvm::Optional<.+>$"')
     debugger.HandleCommand('type summary add -w llvm '
                            '-F lldbDataFormatters.SmallStringSummaryProvider '
                            '-x "^llvm::SmallString<.+>$"')
     debugger.HandleCommand('type summary add -w llvm '
                            '-F lldbDataFormatters.StringRefSummaryProvider '
                            '-x "^llvm::StringRef$"')
     debugger.HandleCommand('type summary add -w llvm '
                            '-F lldbDataFormatters.ConstStringSummaryProvider '
                            '-x "^lldb_private::ConstString$"')
+    debugger.HandleCommand('type synthetic add -w llvm '
+                           '-l lldbDataFormatters.PointerIntPairSynthProvider '
+                           '-x "^llvm::PointerIntPair<.+>$"')
+    debugger.HandleCommand('type synthetic add -w llvm '
+                           '-l lldbDataFormatters.PointerUnionSynthProvider '
+                           '-x "^llvm::PointerUnion<.+>$"')
+
 
 # Pretty printer for llvm::SmallVector/llvm::SmallVectorImpl
 class SmallVectorSynthProvider:
     def __init__(self, valobj, internal_dict):
         self.valobj = valobj;
         self.update() # initialize this provider
 
     def num_children(self):
def SmallStringSummaryProvider(valobj, internal_dict):
     res = "\""
     for i in range(0, num_elements):
         c = valobj.GetChildAtIndex(i).GetValue()
         if c:
             res += c.strip("'")
     res += "\""
     return res
 
+
 def StringRefSummaryProvider(valobj, internal_dict):
     if valobj.GetNumChildren() == 2:
         # StringRef's are also used to point at binary blobs in memory,
         # so filter out suspiciously long strings.
-        max_length = 256
-        length = valobj.GetChildAtIndex(1).GetValueAsUnsigned(max_length)
+        max_length = 1024
+        actual_length = valobj.GetChildAtIndex(1).GetValueAsUnsigned()
+        truncate = actual_length > max_length
+        length = min(max_length, actual_length)
         if length == 0:
-            return "NULL"
-        if length < max_length:
-            return valobj.GetChildAtIndex(0).GetSummary()
-    return ""
+            return '""'
+
+        data = valobj.GetChildAtIndex(0).GetPointeeData(item_count=length)
+        error = lldb.SBError()
+        string = data.ReadRawData(error, 0, data.GetByteSize()).decode()
+        if error.Fail():
+            return "<error: %s>" % error.description
+
+        # json.dumps conveniently escapes the string for us.
+        string = json.dumps(string)
+        if truncate:
+            string += "..."
+        return string
+    return None
+
 
 def ConstStringSummaryProvider(valobj, internal_dict):
     if valobj.GetNumChildren() == 1:
         return valobj.GetChildAtIndex(0).GetSummary()
     return ""
+
+
+def get_expression_path(val):
+    stream = lldb.SBStream()
+    if not val.GetExpressionPath(stream):
+        return None
+    return stream.GetData()
+
+
+class PointerIntPairSynthProvider:
+    def __init__(self, valobj, internal_dict):
+        self.valobj = valobj
+        self.update()
+
+    def num_children(self):
+        return 2
+
+    def get_child_index(self, name):
+        if name == 'Pointer':
+            return 0
+        if name == 'Int':
+            return 1
+        return None
+
+    def get_child_at_index(self, index):
+        expr_path = get_expression_path(self.valobj)
+        if index == 0:
+            return self.valobj.CreateValueFromExpression('Pointer', f'({self.pointer_ty.name}){expr_path}.getPointer()')
+        if index == 1:
+            return self.valobj.CreateValueFromExpression('Int', f'({self.int_ty.name}){expr_path}.getInt()')
+        return None
+
+    def update(self):
+        self.pointer_ty = self.valobj.GetType().GetTemplateArgumentType(0)
+        self.int_ty = self.valobj.GetType().GetTemplateArgumentType(2)
+
+
+def parse_template_parameters(typename):
+    """
+    LLDB doesn't support template parameter packs, so let's parse them manually.
+    """
+    result = []
+    start = typename.find('<')
+    end = typename.rfind('>')
+    if start < 1 or end < 2 or end - start < 2:
+        return result
+
+    nesting_level = 0
+    current_parameter_start = start + 1
+
+    for i in range(start + 1, end + 1):
+        c = typename[i]
+        if c == '<':
+            nesting_level += 1
+        elif c == '>':
+            nesting_level -= 1
+        elif c == ',' and nesting_level == 0:
+            result.append(typename[current_parameter_start:i].strip())
+            current_parameter_start = i + 1
+
+    result.append(typename[current_parameter_start:i].strip())
+
+    return result
+
+
+class PointerUnionSynthProvider:
+    def __init__(self, valobj, internal_dict):
+        self.valobj = valobj
+        self.update()
+
+    def num_children(self):
+        return 1
+
+    def get_child_index(self, name):
+        if name == 'Ptr':
+            return 0
+        return None
+
+    def get_child_at_index(self, index):
+        if index != 0:
+            return None
+        ptr_type_name = self.template_args[self.active_type_tag]
+        return self.valobj.CreateValueFromExpression('Ptr', f'({ptr_type_name}){self.val_expr_path}.getPointer()')
+
+    def update(self):
+        self.pointer_int_pair = self.valobj.GetChildMemberWithName('Val')
+        self.val_expr_path = get_expression_path(self.valobj.GetChildMemberWithName('Val'))
+        self.active_type_tag = self.valobj.CreateValueFromExpression('', f'(int){self.val_expr_path}.getInt()').GetValueAsSigned()
+        self.template_args = parse_template_parameters(self.valobj.GetType().name)