This is an archive of the discontinued LLVM Phabricator instance.

[inline-asm] Fix scope of assembly macros
AbandonedPublic

Authored by sdmitrouk on Nov 24 2014, 4:51 AM.

Details

Summary

Currently, there is one module-level scope and one more scope per inline
assembly operator inside functions. It means that one can't use assembly
macros defined at global scope in functions as well as that macros are
not shared among assembly blocks defined at function-level. Both seems
to be wrong as the effect must be as if all assembly blocks would be
directly inserted into assembly where macros are visible everywhere
below the point of their definition.

There is not really pretty part with std::unique_ptr in AsmParser,
but it must be done that way at least temporary to do not break any
build (Clang uses this interface).

Diff Detail

Event Timeline

sdmitrouk updated this revision to Diff 16549.Nov 24 2014, 4:51 AM
sdmitrouk retitled this revision from to [inline-asm] Fix scope of assembly macros.
sdmitrouk updated this object.
sdmitrouk edited the test plan for this revision. (Show Details)
sdmitrouk added reviewers: echristo, grosbach.
sdmitrouk set the repository for this revision to rL LLVM.
sdmitrouk added a subscriber: Unknown Object (MLST).
sdmitrouk updated this revision to Diff 16564.Nov 24 2014, 8:08 AM

Replaced StringRef with std::string as related data is stored longer than before.

sdmitrouk planned changes to this revision.Nov 24 2014, 11:16 AM

There is another problem with StringRef, which are used in AsmToken.

sdmitrouk updated this revision to Diff 16618.Nov 25 2014, 7:11 AM
sdmitrouk set the repository for this revision to rL LLVM.

Fix second issue with StringRef by introducing a wrapper class that copies underlying string value. Better ideas are welcomed.

rnk added a subscriber: rnk.Dec 2 2014, 11:31 AM

I guess we need this if we want gcc compatibility. Users shouldn't have to repeat macros in every inline asm blob, which gcc will then reject, forcing them to use ifdefs. =P

lib/MC/MCParser/AsmParser.cpp
55–56

Please don't use Allman bracing, put the following open brace on the same line for ifs, fors, tag types, etc.

527

This constructor will be removed once clang is updated, right?

547–553

Please leave these in initializer lists, even at the cost of duplication. The duplication will be gone once the alternate ctor is deleted.

1913

Underscores in variable names aren't really used in LLVM. Can you name this parameter something like TokenToCopy or CopiedToken?

4797

More Allman

I guess we need this if we want gcc compatibility.

Not sure if it's just about "gcc compatibility". C standard says:

The asm keyword may be used to insert assembly language directly into the translator output.

Assuming that there is one "translator output" per translation unit, the statement above
makes me think that there must be one common scope for all such directives.

Thanks for reviewing this.

lib/MC/MCParser/AsmParser.cpp
55–56

Right, I know how it should be formatted from the style guide, still did it wrong.

527

Correct.

547–553

OK.

1913

I just copied style from "surrounding code" (see AsmParser constructors above), now renamed to OriginalToken.
Maybe I should make additional cleanup commit to remove leading underscores from AsmParser?

4797

I just used output of clang-format as a guide, it found a couple more formatting issues in changed code.

sdmitrouk updated this revision to Diff 16856.Dec 3 2014, 1:31 AM
sdmitrouk updated this object.
sdmitrouk added a reviewer: rnk.
sdmitrouk set the repository for this revision to rL LLVM.

Address review comments.

rnk edited edge metadata.Dec 3 2014, 10:40 AM

How will macros interact with diagnostics emitted in Clang? An SMLoc is just a const char* wrapper. Clang even assumes that this pointer points to the asm buffer. Adding macros that point to strings outside the main buffer seems like it could violate that assumption, right?

See this code in Clang:

SourceLocation translateLocation(const llvm::SourceMgr &LSM, llvm::SMLoc SMLoc) {
  // Compute an offset into the inline asm buffer.
  // FIXME: This isn't right if .macro is involved (but hopefully, no
  // real-world code does that).
  const llvm::MemoryBuffer *LBuf =
      LSM.getMemoryBuffer(LSM.FindBufferContainingLoc(SMLoc));
  unsigned Offset = SMLoc.getPointer() - LBuf->getBufferStart();
echristo edited edge metadata.Dec 3 2014, 10:47 AM
In D6383#15, @sdmitrouk wrote:

I guess we need this if we want gcc compatibility.

Not sure if it's just about "gcc compatibility". C standard says:

The asm keyword may be used to insert assembly language directly into the translator output.

Assuming that there is one "translator output" per translation unit, the statement above
makes me think that there must be one common scope for all such directives.

This is interesting, how does it work in the presence of LTO? One translator per previous translation unit? One per now linked set of modules?

Anyhow, going to look a bit more.

@rnk

How will macros interact with diagnostics emitted in Clang?

Hm, I saw some issues with diagnostics before, that's how I found broken StringRef instances.
I'm not sure how it works or whether it works correctly with this change. Not sure if SMLoc uses any of those tokens.
Thanks for the pointer, I'll look into this.

@echristo

This is interesting, how does it work in the presence of LTO? One translator per previous translation unit? One per now linked set of modules?

Didn't think about LTO, and my knowledge of its implementation isn't
enough to answer. However, there is a restriction on functions: asm-directives block
inlining, maybe the same rule applies during LTO? It would prevent possibly
bad things from happening (e.g. applying wrong set of macros).

rnk added a comment.Dec 3 2014, 11:22 AM

For LTO, what if we make Clang responsible for using the same asm context, and have it expand macros before creating the IR? We would still have the problem that llc's .s emission would produce different object files than it produces directly. We could have LLVM's LTO infrastructure expand .macro prior to linking, but that could kill lazy linking. We could also change AsmPrinter to undefine any macros introduce with .macro at the end of an asm blob, but still have Clang ask for the expansion.

I don't like any of this. Maybe we should just reject .macro in inline asm.

asl added a subscriber: asl.Dec 12 2014, 11:44 PM

I don't like any of this. Maybe we should just reject .macro in inline asm.

I would assume that rejecting macro in inline asm will break a lot of
existing code include e.g. linux kernel, various libraries, etc. So,
this does not seem to be a user-friendly solution :(

grosbach edited edge metadata.Dec 15 2014, 3:38 PM

Use of assembler directives at all in inline assembly has always been an edge case that only sort-of works and is sketchily supported. The canonical answer to anyone having problems with them is "don't do that."

If we can improve the support for them with a small bit of effort, that seems reasonable to me, but if it's opening up even further the can of worms, I'd prefer a sternly worded diagnostic instead.

Use of .macro in function-level inline assembly is a pretty horrible thing. I don't mind at all saying that's completely unsupported. In module level assembly, it's a bit tougher question. If we're going to support the feature at all, the latter seems the right thing to work on.

@rnk:

We could also change AsmPrinter to undefine any macros introduce with .macro at the end of an asm blob, but still have Clang ask for the expansion.

Couldn't understand what you mean for a while, looks like I didn't describe well
what the patch does, sorry. Undefining anything would make sense if functions
share same context, which is still not the case (macros defined in one function
are not visible in any other function). I'm not sure if sharing declarations is
safe even without LTO as it depends on order of functions. So another variant
(next paragraph) wouldn't affect lazy linking.

We could have LLVM's LTO infrastructure expand .macro prior to linking, but that could kill lazy linking.

Tried that, the problem is with determining target machine. Correct target
detection requires all modules to be linked together, linking them requires
macro expansion, macro expansion requires target detection (loop).

Another option would be to define module-level macros at the top of all
functions, but it has quite a lot of overhead and requires parsing to
separate macros from everything else.

For LTO, what if we make Clang responsible for using the same asm context, and have it expand macros before creating the IR?

Couldn't do this before creating the IR, need Target to parse inline asm.
Instead wrote a pass, which expands inline asm before module is emitted.
Please take a look and let me know what you think: http://reviews.llvm.org/P80

I have also discovered that it's easy to make two files that compile fine
without LTO, but fail when it's used. The issue is that module-level asm is
just concatenated, which might cause conflicts. This pass makes situation
somewhat better as it removes module-level macro definitions.

We would still have the problem that llc's .s emission would produce different object files than it produces directly.

I'm not sure if it's a problem, e.g. converting IR->BC->IR can change the code
because of constant folding (at least, there might be more cases), which is in
my opinion way more unexpected than LTO vs. non-LTO output differences.

In general, I don't think that forbidding anything like this is a good idea. I
don't see any good reason that would justify such decision, in fact such
artificial constraints seem to be against users. On the other hand, supporting
every possible use case isn't the best choice as well. I guess something like
what we have here (module-level macros are visible in functions + each function
has its own scope) might be a compromise that should work fine with most of code
bases and don't harm compiler.

Ping. Please see my previous message with changes for LTO.

rnk added a comment.Feb 19 2015, 10:48 AM

For LTO, what if we make Clang responsible for using the same asm context, and have it expand macros before creating the IR?

Couldn't do this before creating the IR, need Target to parse inline asm.
Instead wrote a pass, which expands inline asm before module is emitted.
Please take a look and let me know what you think: http://reviews.llvm.org/P80

For MS inline asm, Clang already constructs an llvm::Target in clang/lib/Parse/ParseStmtAsm.cpp from the triple, so it is possible to do. I think this might be a good path forwards. Once we get to LLVM IR, we've basically lost source order, since inline functions can get emitted some time after their first use.

I don't know what happens when you have inline asm with operands like this:

int global;
asm volatile ( ".macro LEA_GLOBAL\nlea %0, %%rax\n.endm" : "m"(global));
void *foo() {
  void *addr;
  asm volatile ( "LEA_GLOBAL\n\tmov %%rax, %0", : : "=m"(addr));
  return addr;
}

Clang could opt to not expand macros containing operands in a subsequent asm blob.


I'm not particularly in favor of the approach taken in this patch, where only module assembly can define macros which get used in functions. It's not very intuitive.

Honestly, this is all really horrible. I'm pretty content with the status quo. Maybe we can agree to not implement this? Users can this functionality with the C preprocessor.

I'm not particularly in favor of the approach taken in this patch, where only module assembly can define macros which get used in functions. It's not very intuitive.

Honestly, this is all really horrible. I'm pretty content with the status quo. Maybe we can agree to not implement this? Users can this functionality with the C preprocessor.

I completely agree.

sdmitrouk abandoned this revision.Feb 19 2015, 11:24 PM

For MS inline asm, Clang already constructs an llvm::Target in clang/lib/Parse/ParseStmtAsm.cpp from the triple, so it is possible to do.

Yes, it's possible with triple, but there are LLVM LTO tests where at least one of modules doesn't have triple specified (don't remember which one). If you merge all modules before processing them and at least one has triple, everything is OK, if you want to preprocess each module before merging it in, it won't work unless all modules have triple defined (or result will be defined by order of modules).

I'm not particularly in favor of the approach taken in this patch, where only module assembly can define macros which get used in functions. It's not very intuitive.

Well, not being able to use even module level macros or macros defined in inline assembly, say, one line above in the body of the same function seems to be even less intuitive in my opinion, but:

Honestly, this is all really horrible. I'm pretty content with the status quo. Maybe we can agree to not implement this? Users can this functionality with the C preprocessor.

OK, I'm not insisting on anything, just want to see this either accepted or closed. Thanks.