[lldb] Disable auto fix-its when evaluating expressions in the test suite

[teemperor]

Summary

Currently the test suite runs with enabled automatically applied Clang fix-its for expressions.
This is causing that sometimes incorrect expressions in tests are still evaluated even though they
are actually incorrect. Let's disable this feature in the test suite so that we know when expressions
are wrong and leave the fix-it testing to the dedicated tests for that feature.

Also updates the lang/cpp/operators/ test as it seems Clang needs the struct keywords
before C and would otherwise fail without fixits.

[shafik]

LGTM outside of my comments

lldb/packages/Python/lldbsuite/test/lldbtest.py

2404

Should this be False or is the comment incorrect?

lldb/test/API/lang/cpp/operators/main.cpp

174

This is puzzling, I don't know why we need the struct for I don't see why an elborated type specifier is required here.

[teemperor]

• Revert dummy change that set auto-apply fixits to true (thanks Shafik!)

[teemperor]

lldb/packages/Python/lldbsuite/test/lldbtest.py

2404

True (heh), I changed that to false as some tests were failing in very unexpected ways on my system with this patch and I wanted to double check that it's not the fixits. Forgot to change this line back afterwards :(

lldb/test/API/lang/cpp/operators/main.cpp

174

I remember I was confused about this when I wrote the line below as Clang forced me to add the 'struct' (as apparently there Clang wasn't smart enough for the automatic fixit). I didn't investigate back then as I already had enough on my plate but with some luck that's an easy fix (maybe we get some language option wrong?).

lldb/packages/Python/lldbsuite/test/lldbtest.py

class Base(unittest2.TestCase):
     @classmethod
     def setUpCommands(cls):
         commands = [
             # Disable Spotlight lookup. The testsuite creates
             # different binaries with the same UUID, because they only
             # differ in the debug info, which is not being hashed.
             "settings set symbols.enable-external-lookup false",
 
+            # Disable fix-its by default so that incorrect expressions in tests don't
+            # pass just because Clang thinks it has a fix-it.
+            "settings set target.auto-apply-fixits false",
+
             # Testsuite runs in parallel and the host can have also other load.
             "settings set plugin.process.gdb-remote.packet-timeout 60",
 
             'settings set symbols.clang-modules-cache-path "{}"'.format(
                 configuration.lldb_module_cache_dir),
             "settings set use-color false",
         ]
         # Make sure that a sanitizer LLDB's environment doesn't get passed on.
def expect_expr(
         :param result_summary: The summary that the expression should have. None if the summary should not be checked.
         :param result_value: The value that the expression should have. None if the value should not be checked.
         :param result_type: The type that the expression result should have. None if the type should not be checked.
         :param error_msg: The error message the expression should return. None if the error output should not be checked.
         """
         self.assertTrue(expr.strip() == expr, "Expression contains trailing/leading whitespace: '" + expr + "'")
 
         frame = self.frame()
-        eval_result = frame.EvaluateExpression(expr)
+        options = lldb.SBExpressionOptions()
+
+        # Disable fix-its that tests don't pass by accident.
+        options.SetAutoApplyFixIts(False)
+
+        # Set the usual default options for normal expressions.
+        options.SetIgnoreBreakpoints(True)
+        options.SetLanguage(frame.GuessLanguage())
+
+        eval_result = frame.EvaluateExpression(expr, options)
 
         if error_msg:
             self.assertFalse(eval_result.IsValid(), "Unexpected success with result: '" + str(eval_result) + "'")
             self.assertEqual(error_msg, eval_result.GetError().GetCString())
             return
 
         if not eval_result.GetError().Success():
             self.assertTrue(eval_result.GetError().Success(),

lldb/test/API/lang/cpp/operators/main.cpp

int main(int argc, char **argv) {
   //% self.expect("expr c,1", endstr=" 2012\n")
   //% self.expect("expr &c", endstr=" 2013\n")
   //% self.expect("expr c(1)", endstr=" 91\n")
   //% self.expect("expr c[1]", endstr=" 92\n")
   //% self.expect("expr static_cast<int>(c)", endstr=" 11\n")
   //% self.expect("expr static_cast<long>(c)", endstr=" 12\n")
   //% self.expect("expr c.operatorint()", endstr=" 13\n")
   //% self.expect("expr c.operatornew()", endstr=" 14\n")
-  //% self.expect("expr (new C)->custom_new", endstr=" true\n")
+  //% self.expect("expr (new struct C)->custom_new", endstr=" true\n")
   //% self.expect("expr (new struct C[1])->custom_new", endstr=" true\n")
   //% self.expect("expr delete c2; side_effect", endstr=" = 1\n")
   //% self.expect("expr delete[] c3; side_effect", endstr=" = 2\n")
   delete c2;
   delete[] c3;
   return 0;
 }

lldb/test/Shell/lit-lldb-init.in

 # LLDB init file for the LIT tests.
 settings set symbols.enable-external-lookup false
 settings set plugin.process.gdb-remote.packet-timeout 60
 settings set interpreter.echo-comment-commands false
 settings set symbols.clang-modules-cache-path "@LLDB_TEST_MODULE_CACHE_LLDB@"
+settings set target.auto-apply-fixits false