[AIX] change "llvm-nm" to "env OBJECT_MODE=any llvm-nm" in clang/test for AIX OS

[DiggerLin]

Summary

since default object mode of llvm-nm is change to -X32 (from default -Xany) in patch https://reviews.llvm.org/D132494.In order not effect the test cases in clang/test we need to change "llvm-nm" to "env OBJECT_MODE=any llvm-nm" in clang/test for AIX OS

[jhenderson]

Wouldn't it be better to change the lit config to specify the OBJECT_MODE environment variable on AIX?

[DiggerLin]

In D134284#3802763, @jhenderson wrote:

Wouldn't it be better to change the lit config to specify the OBJECT_MODE environment variable on AIX?

I have tried it before, I added the following in clang/test/lit.cfg.py

if 'system-aix' in config.available_features:
     config.environment['OBJECT_MODE'] = 'any'

it will cause clang some problem in some test cases. something like ,"clang-16: error: OBJECT_MODE setting any is not recognized and is not a valid setting"

[jhenderson]

In D134284#3802766, @DiggerLin wrote:

In D134284#3802763, @jhenderson wrote:

Wouldn't it be better to change the lit config to specify the OBJECT_MODE environment variable on AIX?

I have tried it before, I added the following in clang/test/lit.cfg.py

if 'system-aix' in config.available_features:
     config.environment['OBJECT_MODE'] = 'any'

it will cause clang some problem in some test cases. something like ,"clang-16: error: OBJECT_MODE setting any is not recognized and is not a valid setting"

Not sure I entirely follow. If OBJECT_MODE isn't an environment variable that clang uses, surely it should ignore it? What clang tests throw this sort of error?

[DiggerLin]

In D134284#3802791, @jhenderson wrote:

In D134284#3802766, @DiggerLin wrote:

In D134284#3802763, @jhenderson wrote:

Wouldn't it be better to change the lit config to specify the OBJECT_MODE environment variable on AIX?

I have tried it before, I added the following in clang/test/lit.cfg.py

if 'system-aix' in config.available_features:
     config.environment['OBJECT_MODE'] = 'any'

it will cause clang some problem in some test cases. something like ,"clang-16: error: OBJECT_MODE setting any is not recognized and is not a valid setting"

Not sure I entirely follow. If OBJECT_MODE isn't an environment variable that clang uses, surely it should ignore it? What clang tests throw this sort of error?

for example: clang/test/Parser/parser_overflow.c

Input was:
      1: clang-16: error: OBJECT_MODE setting any is not recognized and is not a valid setting

[jhenderson]

In D134284#3802793, @DiggerLin wrote:

In D134284#3802791, @jhenderson wrote:

In D134284#3802766, @DiggerLin wrote:

In D134284#3802763, @jhenderson wrote:

Wouldn't it be better to change the lit config to specify the OBJECT_MODE environment variable on AIX?

I have tried it before, I added the following in clang/test/lit.cfg.py

if 'system-aix' in config.available_features:
     config.environment['OBJECT_MODE'] = 'any'

it will cause clang some problem in some test cases. something like ,"clang-16: error: OBJECT_MODE setting any is not recognized and is not a valid setting"

Not sure I entirely follow. If OBJECT_MODE isn't an environment variable that clang uses, surely it should ignore it? What clang tests throw this sort of error?

for example: clang/test/Parser/parser_overflow.c

Input was:
      1: clang-16: error: OBJECT_MODE setting any is not recognized and is not a valid setting

It looks to me like https://github.com/llvm/llvm-project/blob/b982ba2a6e0f11340b4e75d1d4eba9ff62a81df7/clang/lib/Driver/Driver.cpp#L554 should be modified to accept the OBJECT_MODE values you've implemented for llvm-nm and llvm-ar. Otherwise, you'll never be able to use the any and 32_64 values supported by those tools as full environment variables (as opposed to variables set for a single or limited set of commands) on a system where clang is also used.

I'm neither a clang nor an AIX developer though, so my opinion is based only on what I've been reviewing in the llvm tools so far. Pinging @daltenty who implemented the functionality (see D82476) and @hubert.reinterpretcast and @jasonliu who reviewed it.

[daltenty]

If OBJECT_MODE isn't an environment variable that clang uses, surely it should ignore it?

The clang driver does in fact respect OBJECT_MODE on AIX, but I'm guessing it doesn't accept the any setting (since it doesn't really make sense in that context).

Maybe it's better to leave the llvm-nm default as any even on AIX? It doesn't match the system 'nm' but other wise we diverge from the llvm-nm behaviour on other platforms it seems, and cause these test issues.

[DiggerLin]

In D134284#3802806, @daltenty wrote:

If OBJECT_MODE isn't an environment variable that clang uses, surely it should ignore it?

The clang driver does in fact respect OBJECT_MODE on AIX, but I'm guessing it doesn't accept the any setting (since it doesn't really make sense in that context).

Maybe it's better to leave the llvm-nm default as any even on AIX? It doesn't match the system 'nm' but other wise we diverge from the llvm-nm behaviour on other platforms it seems, and cause these test issues.

if change to default is Xany, the default value will be different with llvm-ar -X option(default value is -X32), that means llvm tool has different default -X option value on AIX OS.

[daltenty]

It looks to me like https://github.com/llvm/llvm-project/blob/b982ba2a6e0f11340b4e75d1d4eba9ff62a81df7/clang/lib/Driver/Driver.cpp#L554 should be modified to accept the OBJECT_MODE values you've implemented for llvm-nm and llvm-ar. Otherwise, you'll never be able to use the any and 32_64 values supported by those tools as full environment variables (as opposed to variables set for a single or limited set of commands) on a system where clang is also used.

Those settings don't really make sense for clang since you need to pick a single target and aren't typically supported for a compiler driver on AIX. For example the XL compiler would give you:

error: 1501-255 OBJECT_MODE setting is not recognized and is not a valid setting for the compiler.

if you try OBJECT_MODE=any, so I don't think most folks are using such an environment setting on AIX currently (i.e. typically they select either 32 or 64).

I prefect the solution of adding something to the lit config rather than modifying individual tests, since I doubt most LLVM developers are not going to know to add -X any to their tests just for AIX. Ideally we’d find a way to get those tools to act in the AIX fashion, without placing the burden on developers to add options in their tests.

Maybe we can distinguish OBJECT_MODE settings for LLVM tools (which generally accept any) from a setting for the driver (which can't)? Something like a TOOL_OBJECT_MODE env which is preferred for the tools.

[hubert.reinterpretcast]

Instead of applying OBJECT_MODE to everything, maybe we can apply it only to the tool invocations via a LIT expansion?
Something like %llvm-nm that expands (for AIX) to OBJECT_MODE=any llvm-nm?

Would need to check that the LIT internal shell accepts that syntax.

[DiggerLin]

address comment

[daltenty]

LGTM to unblock the AIX buildbots. We can follow up on a separate env later if desired.

[jhenderson]

LGTM, with nit.

clang/test/lit.cfg.py

280

clang/test/lit.cfg.py

 # `AIXTHREAD_STK` environment variable provides a non-intrusive way to request
 # a larger pthread stack size for the tests. Various applications and runtime
 # libraries on AIX use a default pthread stack size of 4 MiB, so we will use
 # that as a default value here.
 if 'AIXTHREAD_STK' in os.environ:
     config.environment['AIXTHREAD_STK'] = os.environ['AIXTHREAD_STK']
 elif platform.system() == 'AIX':
     config.environment['AIXTHREAD_STK'] = '4194304'
+
+# The llvm-nm tool supports an environment variable "OBJECT_MODE" on AIX OS, which
+# controls the kind of objects they will support. If there is no "OBJECT_MODE"
+# environment variable specified, the default behaviour is to support 32-bit
+# objects only. In order to not affect most test cases, which expect to support
+# 32-bit and 64-bit objects by default, set the environment variable
+# "OBJECT_MODE" to 'any' for llvm-nm on AIX OS.
+
+if 'system-aix' in config.available_features:
+        config.substitutions.append(('llvm-nm', 'env OBJECT_MODE=any llvm-nm'))