Fix test cases for big-endian systems

[uweigand]

Summary

A number of test cases were failing on big-endian systems simply due to
byte order assumptions in the tests themselves, and no underlying bug
in LLDB.

These two test cases:

tools/lldb-server/lldbgdbserverutils.py
python_api/process/TestProcessAPI.py

actually check for big-endian target byte order, but contain Python errors
in the corresponding code paths.

These test cases:

functionalities/data-formatter/data-formatter-python-synth/TestDataFormatterPythonSynth.py
functionalities/data-formatter/data-formatter-smart-array/TestDataFormatterSmartArray.py
functionalities/data-formatter/synthcapping/TestSyntheticCapping.py
lang/cpp/frame-var-anon-unions/TestFrameVariableAnonymousUnions.py
python_api/sbdata/TestSBData.py  (first change)

could be fixed to check for big-endian target byte order and update the
expected result strings accordingly. For the two synthetic tests, I've
also updated the source to make sure the fake_a value is always nonzero
on both big- and little-endian platforms.

These test case:

python_api/sbdata/TestSBData.py  (second change)
functionalities/memory/cache/TestMemoryCache.py

simply accessed memory with the wrong size, which wasn't noticed on LE
but fails on BE.

[clayborg]

So many tests above are going to accept either a little endian or big endian value. This will make most of these tests useless since if a little endian machine fails with a big endian number we won't catch it and vice versa. So we need to expect the correct value for little endian and a different but correct one for big endian tests and only accept the correct one.

packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-python-synth/TestDataFormatterPythonSynth.py

75–79

Zero seems like a bad alternate value as it could cover up a real failure. Can we improve this test so that we are testing for actual values? Or can we change the test by endianness and test for 16777216 for little endian and something else that is not zero for big endian? We don't want zero to be valid for little endian tests.

87

ditto

91

ditto

96

ditto

99

ditto

109

We should expect one thing for little endian and one for big, don't accept either.

116

We should expect one thing for little endian and one for big, don't accept either.

122

We should expect one thing for little endian and one for big, don't accept either.

135

We should expect one thing for little endian and one for big, don't accept either.

147

We should expect one thing for little endian and one for big, don't accept either.

172

We should expect one thing for little endian and one for big, don't accept either.

packages/Python/lldbsuite/test/functionalities/data-formatter/synthcapping/TestSyntheticCapping.py

58

We should expect one thing for little endian and one for big, don't accept either.

66

We should expect one thing for little endian and one for big, don't accept either.

packages/Python/lldbsuite/test/lang/cpp/frame-var-anon-unions/TestFrameVariableAnonymousUnions.py

22

We should expect one thing for little endian and one for big, don't accept either.

[uweigand]

In D18985#397674, @clayborg wrote:

So many tests above are going to accept either a little endian or big endian value. This will make most of these tests useless since if a little endian machine fails with a big endian number we won't catch it and vice versa. So we need to expect the correct value for little endian and a different but correct one for big endian tests and only accept the correct one.

I fully agree this would be preferable. Unfortunately I didn't see a straightforward way to detect in those test cases whether the target platform is big- or little-endian. In the Python API tests this is simple, but these tests here just operate on the console interface. Is there a good way to detect target byte order in such tests?

Zero seems like a bad alternate value as it could cover up a real failure. Can we improve this test so that we are testing for actual values? Or can we change the test by endianness and test for 16777216 for little endian and something else that is not zero for big endian? We don't want zero to be valid for little endian tests.

The test case is a bit weird in how it (apparently deliberately) overlays mis-aligned synthetic fields across an integer array. This is always going to behave quite differently depending on byte order. I'll see if I can change the test so that values will be nonzero both on big- and on little-endian platforms.

[labath]

In D18985#398169, @uweigand wrote:

In D18985#397674, @clayborg wrote:

So many tests above are going to accept either a little endian or big endian value. This will make most of these tests useless since if a little endian machine fails with a big endian number we won't catch it and vice versa. So we need to expect the correct value for little endian and a different but correct one for big endian tests and only accept the correct one.

I fully agree this would be preferable. Unfortunately I didn't see a straightforward way to detect in those test cases whether the target platform is big- or little-endian. In the Python API tests this is simple, but these tests here just operate on the console interface. Is there a good way to detect target byte order in such tests?

Just because the tests test the console interface, it doesn't mean you can't use the python interface to do auxiliary tasks, like computing the expectations. It sounds like you should be able to to a self.target.GetByteOrder() to get the info you need. (use self.dbg.GetSelectedTarget() if self.target happens to be empty, i don't remember if it's always set).

[uweigand]

In D18985#398171, @labath wrote:

Just because the tests test the console interface, it doesn't mean you can't use the python interface to do auxiliary tasks, like computing the expectations. It sounds like you should be able to to a self.target.GetByteOrder() to get the info you need. (use self.dbg.GetSelectedTarget() if self.target happens to be empty, i don't remember if it's always set).

Indeed, that seems to work fine. Thanks for the tip! I'll update the tests accordingly.

[uweigand]

In D18985#397674, @clayborg wrote:

So many tests above are going to accept either a little endian or big endian value. This will make most of these tests useless since if a little endian machine fails with a big endian number we won't catch it and vice versa. So we need to expect the correct value for little endian and a different but correct one for big endian tests and only accept the correct one.

The new version should address this. Each test now only accepts either the little-endian or big-endian result, depending on the target byte order.

Zero seems like a bad alternate value as it could cover up a real failure. Can we improve this test so that we are testing for actual values? Or can we change the test by endianness and test for 16777216 for little endian and something else that is not zero for big endian? We don't want zero to be valid for little endian tests.

I've changed the test source value to use values such that none of the data elements tested by the test case is ever zero, neither on little-endian nor on big-endian.

Tested on both System z and Intel to verify it still works with either byte order.

[clayborg]

Looks good, thanks for updating the changes.

packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-python-synth/TestDataFormatterPythonSynth.py

def data_formatter_commands(self):
                        'r = 33']);
 
         # now set up the synth
         self.runCmd("script from fooSynthProvider import *")
         self.runCmd("type synth add -l fooSynthProvider foo")
         self.expect("type synthetic list foo", substrs=['fooSynthProvider'])
 
         # check that we get the two real vars and the fake_a variables
+        # note that the value of fake_a depends on target byte order
         self.expect("frame variable f00_1",
-                    substrs = ['r = 33',
-                               'fake_a = 16777216',
+                    patterns = ['r = 33',
+                                'fake_a = (16777216|0)',
                                 'a = 0']);
 
         # check that we do not get the extra vars
         self.expect("frame variable f00_1", matching=False,
                     substrs = ['b = 1']);
         
         # check access to members by name
         self.expect('frame variable f00_1.fake_a',
-                substrs = ['16777216'])
+                patterns = ['(16777216|0)'])
         
         # check access to members by index
         self.expect('frame variable f00_1[1]',
-                    substrs = ['16777216'])
+                    patterns = ['(16777216|0)'])
         
         # put synthetic children in summary in several combinations
         self.runCmd("type summary add --summary-string \"fake_a=${svar.fake_a}\" foo")
         self.expect('frame variable f00_1',
-                    substrs = ['fake_a=16777216'])
+                    patterns = ['fake_a=(16777216|0)'])
         self.runCmd("type summary add --summary-string \"fake_a=${svar[1]}\" foo")
         self.expect('frame variable f00_1',
-            substrs = ['fake_a=16777216'])
+            patterns = ['fake_a=(16777216|0)'])
         
         # clear the summary
         self.runCmd("type summary delete foo")
 
         # check that the caching does not span beyond the stopoint
         self.runCmd("n")
 
         self.expect("frame variable f00_1",
-                    substrs = ['r = 33',
-                               'fake_a = 16777216',
+                    patterns = ['r = 33',
+                                'fake_a = (16777216|256)',
                                 'a = 1']);
 
         # check that altering the object also alters fake_a
         self.runCmd("expr f00_1.a = 280")
         self.expect("frame variable f00_1",
-                    substrs = ['r = 33',
-                               'fake_a = 16777217',
+                    patterns = ['r = 33',
+                                'fake_a = (16777217|71680)',
                                 'a = 280']);
 
         # check that expanding a pointer does the right thing
         self.expect("frame variable --ptr-depth 1 f00_ptr",
-            substrs = ['r = 45',
-                       'fake_a = 218103808',
+            patterns = ['r = 45',
+                        'fake_a = (218103808|3072)',
                         'a = 12'])
         
         # now add a filter.. it should fail
         self.expect("type filter add foo --child b --child j", error=True,
                 substrs = ['cannot add'])
         
         # we get the synth again..
         self.expect('frame variable f00_1', matching=False,
             substrs = ['b = 1',
                        'j = 17'])
         self.expect("frame variable --ptr-depth 1 f00_ptr",
-                    substrs = ['r = 45',
-                               'fake_a = 218103808',
+                    patterns = ['r = 45',
+                                'fake_a = (218103808|3072)',
                                 'a = 12'])
         
         # now delete the synth and add the filter
         self.runCmd("type synth delete foo")
         self.runCmd("type filter add foo --child b --child j")
         
         self.expect('frame variable f00_1',
                         substrs = ['b = 1',
                                    'j = 17'])
         self.expect("frame variable --ptr-depth 1 f00_ptr", matching=False,
-                    substrs = ['r = 45',
-                               'fake_a = 218103808',
+                    patterns = ['r = 45',
+                                'fake_a = (218103808|3072)',
                                 'a = 12'])
         
         # now add the synth and it should fail
         self.expect("type synth add -l fooSynthProvider foo", error=True,
                     substrs = ['cannot add'])
         
         # check the listing
         self.expect('type synth list', matching=False,
                     substrs = ['foo',
                                'Python class fooSynthProvider'])
         self.expect('type filter list', 
                     substrs = ['foo',
                                '.b',
                                '.j'])
         
         # delete the filter, add the synth
         self.runCmd("type filter delete foo")
         self.runCmd("type synth add -l fooSynthProvider foo")
         
         self.expect('frame variable f00_1', matching=False,
                     substrs = ['b = 1',
                                'j = 17'])
         self.expect("frame variable --ptr-depth 1 f00_ptr", 
-                    substrs = ['r = 45',
-                               'fake_a = 218103808',
+                    patterns = ['r = 45',
+                                'fake_a = (218103808|3072)',
                                 'a = 12'])
 
         # check the listing
         self.expect('type synth list',
                     substrs = ['foo',
                                'Python class fooSynthProvider'])
         self.expect('type filter list', matching=False,
                     substrs = ['foo',
                                '.b',

packages/Python/lldbsuite/test/functionalities/data-formatter/data-formatter-smart-array/TestDataFormatterSmartArray.py

# using array smart summary strings for pointers should make no sense
         self.expect("frame variable intptr", matching=False,
                     substrs = ['1,1,2,3,5'])
 
 # use y and Y
         self.runCmd("type summary add --summary-string \"arr = ${var%y}\" \"float [7]\"")
         self.runCmd("type summary add --summary-string \"arr = ${var%y}\" \"int [5]\"")
 
         self.expect("frame variable flarr",
-                    substrs = ['flarr = arr =',
-                               '00 00 9d 42,00 80 9a 42,00 00 9c 42,00 40 98 42,00 80 99 42,00 c0 99 42,00 00 9a 42'])
+                    patterns = ['flarr = arr =',
+                                '(00 00 9d 42,00 80 9a 42,00 00 9c 42,00 40 98 42,00 80 99 42,00 c0 99 42,00 00 9a 42|42 9d 00 00,42 9a 80 00,42 9c 00 00,42 98 40 00,42 99 80 00,42 99 c0 00,42 9a 00 00)'])
         
         self.expect("frame variable other.flarr",
-                    substrs = ['flarr = arr =',
-                               '00 00 cc 41,00 00 ca 41,00 00 c9 41,00 00 d6 41,00 00 db 41,00 00 dc 41,00 00 d1 41'])
+                    patterns = ['flarr = arr =',
+                                '(00 00 cc 41,00 00 ca 41,00 00 c9 41,00 00 d6 41,00 00 db 41,00 00 dc 41,00 00 d1 41|41 cc 00 00,41 ca 00 00,41 c9 00 00,41 d6 00 00,41 db 00 00,41 dc 00 00,41 d1 00 00)'])
 
         self.expect("frame variable intarr",
-                    substrs = ['intarr = arr =',
-                               '01 00 00 00,01 00 00 00,02 00 00 00,03 00 00 00,05 00 00 00'])
+                    patterns = ['intarr = arr =',
+                                '(01 00 00 00,01 00 00 00,02 00 00 00,03 00 00 00,05 00 00 00|00 00 00 01,00 00 00 01,00 00 00 02,00 00 00 03,00 00 00 05)'])
         
         self.expect("frame variable other.intarr",
-                    substrs = ['intarr = arr = ',
-                               '09 00 00 00,08 00 00 00,07 00 00 00,06 00 00 00,05 00 00 00'])
+                    patterns = ['intarr = arr = ',
+                                '(09 00 00 00,08 00 00 00,07 00 00 00,06 00 00 00,05 00 00 00|00 00 00 09,00 00 00 08,00 00 00 07,00 00 00 06,00 00 00 05)'])
                     
         self.runCmd("type summary add --summary-string \"arr = ${var%Y}\" \"float [7]\"")
         self.runCmd("type summary add --summary-string \"arr = ${var%Y}\" \"int [5]\"")
             
         self.expect("frame variable flarr",
-                    substrs = ['flarr = arr =',
-                               '00 00 9d 42             ...B,00 80 9a 42             ...B,00 00 9c 42             ...B,00 40 98 42             .@.B,00 80 99 42             ...B,00 c0 99 42             ...B,00 00 9a 42             ...B'])
+                    patterns = ['flarr = arr =',
+                                '(00 00 9d 42             \.\.\.B,00 80 9a 42             \.\.\.B,00 00 9c 42             \.\.\.B,00 40 98 42             \.@\.B,00 80 99 42             \.\.\.B,00 c0 99 42             \.\.\.B,00 00 9a 42             \.\.\.B|42 9d 00 00             B\.\.\.,42 9a 80 00             B\.\.\.,42 9c 00 00             B\.\.\.,42 98 40 00             B\.@\.,42 99 80 00             B\.\.\.,42 99 c0 00             B\.\.\.,42 9a 00 00             B\.\.\.)'])
         
         self.expect("frame variable other.flarr",
-                    substrs = ['flarr = arr =',
-                               '00 00 cc 41             ...A,00 00 ca 41             ...A,00 00 c9 41             ...A,00 00 d6 41             ...A,00 00 db 41             ...A,00 00 dc 41             ...A,00 00 d1 41             ...A'])
+                    patterns = ['flarr =\ arr =',
+                                '(00 00 cc 41             \.\.\.A,00 00 ca 41             \.\.\.A,00 00 c9 41             \.\.\.A,00 00 d6 41             \.\.\.A,00 00 db 41             \.\.\.A,00 00 dc 41             \.\.\.A,00 00 d1 41             \.\.\.A|41 cc 00 00             A\.\.\.,41 ca 00 00             A\.\.\.,41 c9 00 00             A\.\.\.,41 d6 00 00             A\.\.\.,41 db 00 00             A\.\.\.,41 dc 00 00             A\.\.\.,41 d1 00 00             A\.\.\.)'])
         
         self.expect("frame variable intarr",
-                    substrs = ['intarr = arr =',
-                               '....,01 00 00 00',
-                               '....,05 00 00 00'])
+                    patterns = ['intarr = arr =',
+                                '\.\.\.\.,(01 00 00 00|00 00 00 01)',
+                                '\.\.\.\.,(05 00 00 00|00 00 00 05)'])
         
         self.expect("frame variable other.intarr",
-                    substrs = ['intarr = arr = ',
-                               '09 00 00 00',
-                               '....,07 00 00 00'])
+                    patterns = ['intarr = arr = ',
+                                '(09 00 00 00|00 00 00 09)',
+                                '\.\.\.\.,(07 00 00 00|00 00 00 07)'])

packages/Python/lldbsuite/test/functionalities/data-formatter/synthcapping/TestSyntheticCapping.py

def test_with_run_command(self):
         self.addTearDownHook(cleanup)
 
         # set up the synthetic children provider
         self.runCmd("script from fooSynthProvider import *")
         self.runCmd("type synth add -l fooSynthProvider foo")
 
         # check that the synthetic children work, so we know we are doing the right thing
         self.expect("frame variable f00_1",
-                    substrs = ['r = 33',
-                               'fake_a = 16777216',
+                    patterns = ['r = 33',
+                                'fake_a = (16777216|0)',
                                 'a = 0']);
 
         # check that capping works
         self.runCmd("settings set target.max-children-count 2", check=False)
         
         self.expect("frame variable f00_1",
-                    substrs = ['...',
-                               'fake_a = 16777216',
+                    patterns = ['...',
+                                'fake_a = (16777216|0)',
                                 'a = 0']);
         
         self.expect("frame variable f00_1", matching=False,
                     substrs = ['r = 33']);
 
         
         self.runCmd("settings set target.max-children-count 256", check=False)
 
         self.expect("frame variable f00_1", matching=True,
                     substrs = ['r = 33']);

packages/Python/lldbsuite/test/functionalities/memory/cache/TestMemoryCache.py

def test_memory_cache(self):
         # cache is used.
         self.runCmd('memory read -f d -c 201 `&my_ints - 100`')
 
         # Check the value of my_ints[0] is the same as set in main.cpp.
         line = self.res.GetOutput().splitlines()[100]
         self.assertTrue(0x00000042 == int(line.split(':')[1], 0))
 
         # Change the value of my_ints[0] in memory.
-        self.runCmd("memory write `&my_ints` AA")
+        self.runCmd("memory write -s 4 `&my_ints` AA")
 
         # Re-read the chunk of memory. The cache line should have been
         # flushed because of the 'memory write'.
         self.runCmd('memory read -f d -c 201 `&my_ints - 100`')
 
         # Check the value of my_ints[0] have been updated correctly.
         line = self.res.GetOutput().splitlines()[100]
         self.assertTrue(0x000000AA == int(line.split(':')[1], 0))

packages/Python/lldbsuite/test/lang/cpp/frame-var-anon-unions/TestFrameVariableAnonymousUnions.py

def test_with_run_command(self):
         self.build()
         self.runCmd("file a.out", CURRENT_EXECUTABLE_SET)
 
         line = line_number('main.cpp', '// break here')
         lldbutil.run_break_set_by_file_and_line (self, "main.cpp", line, num_expected_locations=-1, loc_exact=False)
 
         self.runCmd("process launch", RUN_SUCCEEDED)
 
-        self.expect('frame variable -f x i', substrs=['ffffff41'])
+        self.expect('frame variable -f x i', patterns=['(ffffff41|41ffff00)'])
         self.expect('frame variable c', substrs=["'A"])
 
         self.expect('frame variable x', matching=False, substrs=['3'])
         self.expect('frame variable y', matching=False, substrs=["'B'"])
         self.expect('frame variable z', matching=False, substrs=['14'])

packages/Python/lldbsuite/test/python_api/process/TestProcessAPI.py

def test_access_my_int(self):
 
         # Now read the memory content.  The bytearray should have (byte)1 as the second element.
         content = process.ReadMemory(location, byteSize, error)
         if not error.Success():
             self.fail("SBProcess.ReadMemory() failed")
 
         # The bytearray_to_int utility function expects a little endian bytearray.
         if byteOrder == lldb.eByteOrderBig:
+            content = bytearray(content, 'ascii')
             content.reverse()
 
         new_value = bytearray_to_int(content, byteSize)
         if new_value != 256:
             self.fail("Memory content read from 'my_int' does not match (int)256")
 
         # Dump the memory content....
         if self.TraceOn():

packages/Python/lldbsuite/test/python_api/sbdata/TestSBData.py

def test_with_run_command(self):
 
         new_object = barfoo.CreateValueFromData("new_object",data,barfoo.GetType().GetBasicType(lldb.eBasicTypeInt))
 
         if self.TraceOn():
             print(new_object)
         
         self.assertTrue(new_object.GetValue() == "1", 'new_object == 1')
 
+        if data.GetByteOrder() == lldb.eByteOrderBig:
+            data.SetData(error, '\0\0\0A', data.GetByteOrder(), data.GetAddressByteSize())
+        else:
             data.SetData(error, 'A\0\0\0', data.GetByteOrder(), data.GetAddressByteSize())
         self.assertTrue(error.Success())
         
         data2 = lldb.SBData()
         data2.SetData(error, 'BCD', data.GetByteOrder(), data.GetAddressByteSize())
         self.assertTrue(error.Success())
 
         data.Append(data2)
         
def test_with_run_command(self):
 
         self.assertTrue(data2.uint64[0:2] == [1,2], 'read_data_helper failure: set data2[0:2] = [1,2]')
 
         data2.SetDataFromSInt32Array([2, -2])
         self.assert_data(data2.GetSignedInt32, 0, 2)
         self.assert_data(data2.GetSignedInt32, 4, -2)
         
         data2.SetDataFromSInt64Array([2, -2])
-        self.assert_data(data2.GetSignedInt32, 0, 2)
-        self.assert_data(data2.GetSignedInt32, 8, -2)
+        self.assert_data(data2.GetSignedInt64, 0, 2)
+        self.assert_data(data2.GetSignedInt64, 8, -2)
         
         data2.SetDataFromUInt32Array([1,2,3,4,5])
         self.assert_data(data2.GetUnsignedInt32, 0, 1)
         self.assert_data(data2.GetUnsignedInt32, 4, 2)
         self.assert_data(data2.GetUnsignedInt32, 8, 3)
         self.assert_data(data2.GetUnsignedInt32, 12, 4)
         self.assert_data(data2.GetUnsignedInt32, 16, 5)
         

packages/Python/lldbsuite/test/tools/lldb-server/lldbgdbserverutils.py

if endian == 'little':
             retval = retval + "{:02x}".format(value & 0xff)
             value = value >> 8
         if byte_size:
             # Add zero-fill to the right/end (MSB side) of the value.
             retval += "00" * (byte_size - len(retval)/2)
         return retval
 
     elif endian == 'big':
-        retval = value.encode("hex")
+        retval = ""
+        while value != 0:
+            retval = "{:02x}".format(value & 0xff) + retval
+            value = value >> 8
         if byte_size:
             # Add zero-fill to the left/front (MSB side) of the value.
             retval = ("00" * (byte_size - len(retval)/2)) + retval
         return retval
 
     else:
         # pdp is valid but need to add parse code once needed.
         raise Exception("unsupported endian:{}".format(endian))