Fix llvm-objcopy/ELF/preserve-segment-contents test on UTF-8 locale

[aganea]

Summary

Previously, I had the following error under WSL/Ubuntu 18.04 with Python 3.6:

+ /usr/bin/python3.6 /mnt/f/svn/buildWSL/test/tools/llvm-objcopy/ELF/Output/preserve-segment-contents.test.tmp.py
Traceback (most recent call last):
  File "/mnt/f/svn/buildWSL/test/tools/llvm-objcopy/ELF/Output/preserve-segment-contents.test.tmp.py", line 8, in <module>
    input.write('\xDE\xAD\xBE\xEF')
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-3: ordinal not in range(128)

Strangely, Python 3.7 on Windows under MINGW32 was fine.

Tested on Python 2.7.16 and 3.7 on Windows. Tested on Python 3.6 on WSL.

[alexander-shaposhnikov]

to me looks good.

Some side notes:

Strangely, Python 3.7 on Windows under MINGW32 was fine

perhaps because the default encoding for those environments is different

https://stackoverflow.com/questions/20923663/unicodeencodeerror-ascii-codec-cant-encode-character-in-position-0-ordinal/20923915#20923915
https://stackoverflow.com/questions/31917595/how-to-write-a-raw-hex-byte-to-stdout-in-python-3

I would check the other tests as well - if there are any similar places where we write raw bytes in Python

[aganea]

Yes the encoding is different. By running https://gist.github.com/zed/5898423 I get:

   Windows 10 cmd shell:                    WSL:
locale(False):     cp1252             locale(False):     UTF-8
device(stdout):    cp850              device(stdout):    UTF-8
stdout.encoding:   utf-8              stdout.encoding:   UTF-8
device(stderr):    cp850              device(stderr):    UTF-8
stderr.encoding:   utf-8              stderr.encoding:   UTF-8
device(stdin):     cp850              device(stdin):     UTF-8
stdin.encoding:    utf-8              stdin.encoding:    UTF-8

I've checked clang/llvm/lld/lldb, this seems to be the only place where raw bytes are encoded in python.

llvm/trunk/test/tools/llvm-objcopy/ELF/preserve-segment-contents.test

 # RUN: od -t x1 -j 0x2100 -N 12 %t.stripped | FileCheck %s --check-prefix=CHECK2 -DPATTERN="00 00 00 00"
 # RUN: od -t x1 -j 0x2200 -N 4  %t.stripped | FileCheck %s --check-prefix=CHECK3 -DPATTERN="00 00 00 00"
 # RUN: od -t x1 -j 0x2300 -N 12 %t.stripped | FileCheck %s --check-prefix=CHECK4 -DPATTERN="00 00 00 00"
 # RUN: od -t x1 -j 0x3000 -N 68 %t.stripped | FileCheck %s --check-prefix=CHECK5 -DPATTERN="00 00 00 00"
 # RUN: od -t x1 -j 0x4000 -N 60 %t.stripped | FileCheck %s --check-prefix=CHECK6 -DPATTERN="00 00 00 00"
 # RUN: od -t x1 -j 0x5000 -N 60 %t.stripped | FileCheck %s --check-prefix=CHECK7 -DPATTERN="00 00 00 00"
 
 # RUN: cp %t.stripped %t.in
-# RUN: echo "with open('%/t.in', 'r+') as input:"                                  > %t.py
+# RUN: echo "with open('%/t.in', 'rb+') as input:"                                 > %t.py
 # RUN: echo "  for offset in ["                                                   >> %t.py
 # RUN: echo "   0x2000, 0x2008, 0x200C, 0x2014, 0x2104, 0x2300,"                  >> %t.py
 # RUN: echo "   0x3008, 0x3010, 0x3018, 0x3020, 0x3028, 0x302C, 0x3034, 0x303C,"  >> %t.py
 # RUN: echo "   0x4000, 0x4008, 0x4010, 0x4014, 0x401C, 0x4024, 0x4034,"          >> %t.py
 # RUN: echo "   0x5000, 0x5008, 0x5010, 0x501C, 0x5024, 0x502C, 0x5030, 0x5038]:" >> %t.py
 # RUN: echo "    input.seek(offset)"                                              >> %t.py
-# RUN: echo "    input.write('\xDE\xAD\xBE\xEF')"                                 >> %t.py
+# RUN: echo "    input.write(bytearray.fromhex('DEADBEEF'))"                      >> %t.py
 # RUN: %python %t.py
 # RUN: llvm-objcopy %t.in %t.out
 # RUN: od -t x1 -j 0x2000 -N 24 %t.out | FileCheck %s --check-prefix=CHECK1 -DPATTERN="de ad be ef"
 # RUN: od -t x1 -j 0x2100 -N 12 %t.out | FileCheck %s --check-prefix=CHECK2 -DPATTERN="de ad be ef"
 # RUN: od -t x1 -j 0x2200 -N 4  %t.out | FileCheck %s --check-prefix=CHECK3 -DPATTERN="de ad be ef"
 # RUN: od -t x1 -j 0x2300 -N 12 %t.out | FileCheck %s --check-prefix=CHECK4 -DPATTERN="de ad be ef"
 # RUN: od -t x1 -j 0x3000 -N 68 %t.out | FileCheck %s --check-prefix=CHECK5 -DPATTERN="de ad be ef"
 # RUN: od -t x1 -j 0x4000 -N 60 %t.out | FileCheck %s --check-prefix=CHECK6 -DPATTERN="de ad be ef"