This is an archive of the discontinued LLVM Phabricator instance.

Align definition of DW_OP_plus with DWARF spec [1/3]
ClosedPublic

Authored by sdesmalen on Jun 5 2017, 6:42 AM.

Details

Summary

This patch is part of 3 patches that together form a single patch, but must be introduced in stages in order not to break things.

The way that LLVM interprets DW_OP_plus in DIExpression nodes is basically that of the DW_OP_plus_uconst operator since LLVM expects an unsigned constant operand. This unnecessarily restricts the DW_OP_plus operator, preventing it from being used to describe the evaluation of runtime values on the expression stack. These patches try to align the semantics of DW_OP_plus and DW_OP_minus with that of the DWARF definition, which pops two elements off the expression stack, performs the operation and pushes the result back on the stack.

This is done in three stages:
• The first patch (LLVM) adds support for DW_OP_plus_uconst.
• The second patch (Clang) contains changes all its uses from DW_OP_plus to DW_OP_plus_uconst.
• The third patch (LLVM) changes the semantics of DW_OP_plus and DW_OP_minus to be in line with its DWARF meaning. This patch includes the bitcode upgrade from legacy DIExpressions.

Patch by Sander de Smalen.

Diff Detail

Event Timeline

sdesmalen created this revision.Jun 5 2017, 6:42 AM
aprantl requested changes to this revision.Jun 5 2017, 8:48 AM

Thanks, this is great! The only thing that is missing here is a bitcode upgrade that transforms the old use of DW_OP_plus to DW_OP_plus_uconst. I recommend incrementing the "Version number" in operand 0 of DIExpression to distinguish between the old and new format. You can look at r300522 for an example of how to do this.

This revision now requires changes to proceed.Jun 5 2017, 8:48 AM
fhahn added a subscriber: fhahn.Jun 6 2017, 6:24 AM
sdesmalen updated this revision to Diff 101558.Jun 6 2017, 7:08 AM
sdesmalen edited edge metadata.

Incremented the expression's Version operand and added backwards compatibility with older bitcode files by constructing a DW_OP_plus_uconst from a DW_OP_plus.

Thanks, this is great! The only thing that is missing here is a bitcode upgrade that transforms the old use of DW_OP_plus to DW_OP_plus_uconst. I recommend incrementing the "Version number" in operand 0 of DIExpression to distinguish between the old and new format. You can look at r300522 for an example of how to do this.

Thanks for pointing this out! I hope my changes to the patch reflect what you suggested.

Initially the diff also contained a binary diff for the changed .bc files, but these are not needed now as the old .bc files check backward compatibility as implemented in the 'upgradeMetadataExpression()' function. If I want to add binary changes to my patch though, how could I add these? (Phabricator failed to accept my initial diff that I created using git diff --binary)

aprantl added inline comments.Jun 6 2017, 10:55 AM
lib/Bitcode/Reader/MetadataLoader.cpp
541

Let's rename this to upgradeDIExpression.

541

Also, FromVersion is unused.

555

you could break here, since DW_OP_LLVM_fragment must be the last operator in the expression.

564

clang-format please

573

LLVM coding standard wants a . at the end of each comment.

574

Copying the expression here seems to be unnecessary, but I assume the reason is that the DW_OP_minus upgrade cannot be done in place.

1630

I would move the entire switch statement into the upgradeMetadataExpression helper.

aprantl added inline comments.Jun 6 2017, 10:58 AM
lib/Bitcode/Reader/MetadataLoader.cpp
541

Let's rename this to upgradeDIExpression.

test/Bitcode/DIExpression-deref.ll
20

This seems wrong. This file is supposed to be the disassembled version of the .bc file, so it should contain the old DIExpression to test the bitcode upgrade. (The CHECK is fine though).

sdesmalen updated this revision to Diff 101713.Jun 7 2017, 5:52 AM
sdesmalen marked 6 inline comments as done.

Refactoring of upgradeDIExpression()

sdesmalen added inline comments.Jun 7 2017, 5:59 AM
lib/Bitcode/Reader/MetadataLoader.cpp
541

This will be used now that I've moved the whole switch statement to upgradeDIExpression().

549

You'll notice that for up-to-date expressions it does an unnecessary copy to NewExpr.
I did this to make the interface to upgradeDIExpression() more clear, i.e. the new expression is always in NewExpr and the old one is left alone. Otherwise the caller would need to figure out where the result is, either in the first or the second argument.

553

I'm starting with upgrading from version 2 to 0 (rather than in reverse order), so that we only need to do the 'copy' to NewExpr once.

574

correct.

test/Bitcode/DIExpression-deref.ll
20

Well spotted! Thanks.
I did the same for DIGlobalVariableExpression.ll (that has a corresponding .bc file too).

aprantl added inline comments.Jun 7 2017, 11:20 AM
lib/Bitcode/Reader/MetadataLoader.cpp
553

I'm not sure I'm comfortable with reversing the order. In the original implementation we incrementally upgrade the expression from version N-1 to version N. Some of the upgrades (including the one that is added in this patch!) are changing how to iterate over the elements, so it is not generally safe to apply the upgrades in reverse order. If we are upgrading in chronological order then a new upgrade doesn't have to care about the old upgrades, but in the reverse order they have to inspect all other upgrades if they are still correct.

I recommend to leave the order as in the original implementation. I wouldn't worry too much about performance — the upgrade path is not the normal use-case.

sdesmalen updated this revision to Diff 101922.Jun 8 2017, 8:01 AM
sdesmalen added inline comments.Jun 8 2017, 8:04 AM
lib/Bitcode/Reader/MetadataLoader.cpp
553

Valid point about changing the order of applying updates!
I've changed it back to the original order.

aprantl added inline comments.Jun 8 2017, 3:40 PM
lib/Bitcode/Reader/MetadataLoader.cpp
586

This will crash if the expression is malformed and ends after the DW_OP_plus.
It would be better to just push a DW_OP_plus_uconst to the buffer and then fall through.

589

This will crash if the expression is malformed and ends after the operator.
Can you make sure not copy more elements than are in SubExpr here?

Almost there! Couple more comments inline.

thanks,
adrian

lib/Bitcode/Reader/MetadataLoader.cpp
579

This no longer handling DW_OP_LLVM_fragment?

sdesmalen updated this revision to Diff 102026.Jun 9 2017, 7:04 AM

Make sure not to read more elements than are in the array.

sdesmalen marked 2 inline comments as done.Jun 9 2017, 7:08 AM
sdesmalen added inline comments.
lib/Bitcode/Reader/MetadataLoader.cpp
579

Added again.

Thanks! I noticed that in the remainder of the patch there are still references to DW_OP_plus left, but they are completely untested. I would recommend removing all referenced to DW_OP_plus in this patch and then adding support for the "real" DW_OP_plus back in a separate commit, together with testcases.

lib/Bitcode/Reader/MetadataLoader.cpp
555

remove extra newline :-)

575

For readability, let's move the HistoricSize = 1 initialization here.

590

This could be expressed more compactly as:

auto Op = SubExpr.front();
Buffer.push_back(Op == DW_OP_plus ? dwarf::DW_OP_plus_uconst : Op);
Buffer.append(Args.begin(), Args.end());
lib/CodeGen/AsmPrinter/DwarfExpression.cpp
325

Either we handle DW_OP_plus correctly here, or it should be removed.

lib/IR/DebugInfoMetadata.cpp
601

This must be removed.

646

This must be removed.

Thanks! I noticed that in the remainder of the patch there are still references to DW_OP_plus left, but they are completely untested. I would recommend removing all referenced to DW_OP_plus in this patch and then adding support for the "real" DW_OP_plus back in a separate commit, together with testcases.

The reason for not yet removing support for DW_OP_plus is because Clang still emits DW_OP_plus, and removing support for DW_OP_plus would break things.

This patch removes a number of test-cases for DW_OP_plus, but some tests remain in MetadataTest.cpp. The plan was to first migrate all (LLVM) uses of DW_OP_plus to DW_OP_plus_uconst in patch [1/3], then to change all Clang uses of DW_OP_plus to DW_OP_plus_uconst in patch [2/3], and then change the meaning of DW_OP_plus in patch [3/3].

sdesmalen updated this revision to Diff 102162.Jun 12 2017, 2:54 AM
sdesmalen edited the summary of this revision. (Show Details)

Patch [1/3], now just adds support for DW_OP_plus_uconst.
Patch [3/3] changes meaning of DW_OP_plus and DW_OP_minus to be in line with its DWARF meaning. This patch includes the bitcode upgrade from legacy DIExpressions.

sdesmalen marked an inline comment as done.Jun 12 2017, 3:04 AM
sdesmalen added inline comments.
lib/Bitcode/Reader/MetadataLoader.cpp
555

Perhaps Phabricator doesn't display it properly, but I don't see an extra newline here.
I did remove the extra newline between 'auto N = ..' and the switch statement.

590

Now that I have reshuffled the functionality within patches, I kept the less compact notation because the code also supports DW_OP_minus, which needs to be distinguished as a separate case.

aprantl accepted this revision.Jun 12 2017, 9:09 AM

Technically this is missing a DWARF backend test, but since the next review will rewrite all DW_OP_plus to DW_OP_plus_uconst anyway, this is fine.

Thanks!

This revision is now accepted and ready to land.Jun 12 2017, 9:09 AM
fhahn edited the summary of this revision. (Show Details)Jun 13 2017, 9:54 AM
fhahn closed this revision.Jun 13 2017, 9:55 AM

Hm.. I think the other commit in the blamelist looks far more promising.

I think the self-hosting builds should be fixed by https://reviews.llvm.org/rL305313