This is an archive of the discontinued LLVM Phabricator instance.

[ASTImporter] Change the return result of Decl import to Optional
AbandonedPublic

Authored by a_sidorin on Aug 13 2018, 3:52 PM.

Details

Summary

This is a small step to simplify ASTImporter internals and make them more robust. As it was discussed in the mailing lists, the following changes are included:

  • Introduced helper methods importNonNull() and importNullable() with LLVM_NODISCARD attribute. These methods are used to avoid redundant casting and to simplify checks for import return result. LLVM_NODISCARD will also print a warning every time a return result of import is unchecked.
  • The return result of Import*Decl() methods is changed to Optional<Decl *>. The value of nullptr doesn't signal an error anymore (as it should be); None object are used instead in case of import failure. I hope that the need for dereferencing the result will remind about the need to check it first.
  • ASTimporter and its clients are changed to use the new API. The patch for LLDB is coming soon. Some unchecked imports were found and fixed.

Unfortunately, the patch became very big. However, most changes are same. The code became a bit smaller and, as it looks to me, a bit more readable.

Diff Detail

Repository
rC Clang

Event Timeline

a_sidorin created this revision.Aug 13 2018, 3:52 PM

This change is in conflict with our similar patch to introduce error handling in ASTImporter. That patch would be a much bigger one, because (almost) every import can fail, return of every Import (not only Decl) is changed from X to Expected<X>. After every import (inside the import functions too) the result needs to be checked always for error.

Hi Aleksei,

Thank you for this patch.
With Balazs, we are working on something similar, but with a different, fine grained error value mechanism. Unfortunately we were not aware of that you have been working on error handling, and we didn't say that we are working on error handling recently, I am terribly sorry about this.

From the CSA perspective, we realized that there may be several different error cases which has to be handled differently in CrossTranslationUnit.cpp.
For example, there are unsupported constructs which we do not support to import (like a struct definition as a parameter of a function).
Another example is when there is a name conflict between the decl in the "From" context and the decl in the "To" context, this usually means an ODR error.
We have to handle these errors in a different way after we imported one function during CTU analysis.
The fact that there may be more than one kind of errors yields for the use of the designated LLVM types: Error and Expected<T>. A simple Optional is probably not generic enough.

I find the importNonNull and generally the new family of import functions useful, but I am not sure how they could cooperate with Expected<T>. Especially, I have some concerns with output parameters.
If we have an Expected<T> as a result type, then there is no way to acquire the T if there was an error. However, when we have output parameters, then even if there was an error some output params could have been set ... and those can be reused even after the return. If error handling is not proper on the callee site then we may continue with stale values, which is not possible if we use Expected<T> as a return value.

Do you think we can hold back this patch for a few days until Balazs prepares the Expected<T> based version? Then I think we could compare the patches and we could merge the best from the two of them.

Do you think we can hold back this patch for a few days until Balazs prepares the Expected<T> based version? Then I think we could compare the patches and we could merge the best from the two of them.

I mean, once Balazs is also ready, then we can create a combined patch which holds the good things from both of yours and his patches.

Roughly, this is the direction where we are heading with Expected<T>:
https://github.com/Ericsson/clang/pull/457/commits/783b7f5cce9de589a5c3c3ae983398cf499077ec

(If you are interested, here is our whole thought process:
https://github.com/Ericsson/clang/pull/457)

Hello Gabor and Balazs,

With Balazs, we are working on something similar, but with a different, fine grained error value mechanism. Unfortunately we were not aware of that you have been working on error handling, and we didn't say that we are working on error handling recently, I am terribly sorry about this.

That's a bit disappointing because I was thinking that the status of error handling strategy discussion and implementation is reflected in the mailing list. But well, let's think what we can do about this.

From the CSA perspective, we realized that there may be several different error cases which has to be handled differently in CrossTranslationUnit.cpp.
For example, there are unsupported constructs which we do not support to import (like a struct definition as a parameter of a function).
Another example is when there is a name conflict between the decl in the "From" context and the decl in the "To" context, this usually means an ODR error.
We have to handle these errors in a different way after we imported one function during CTU analysis.
The fact that there may be more than one kind of errors yields for the use of the designated LLVM types: Error and Expected<T>. A simple Optional is probably not generic enough.

Yes, I was thinking about it too. The reason why I chose Optional was that ASTImporter clients don't use the error kind. If you have any plans on changing this, Expected is a preferred approach.

I find the importNonNull and generally the new family of import functions useful, but I am not sure how they could cooperate with Expected<T>. Especially, I have some concerns with output parameters.
If we have an Expected<T> as a result type, then there is no way to acquire the T if there was an error. However, when we have output parameters, then even if there was an error some output params could have been set ... and those can be reused even after the return. If error handling is not proper on the callee site then we may continue with stale values, which is not possible if we use Expected<T> as a return value.

If I understand you correctly, importNonNull and importNullable should work with Expected pretty well. Changing import*Into return result to Error can partially help in avoiding usage of an unchecked result. These functions already guarantee that their parameters are changed only if the internal import was successful.

Do you think we can hold back this patch for a few days until Balazs prepares the Expected<T> based version? Then I think we could compare the patches and we could merge the best from the two of them.

Sure.

Yes, I was thinking about it too. The reason why I chose Optional was that ASTImporter clients don't use the error kind. If you have any plans on changing this, Expected is a preferred approach.

Yes, we plan to submit a patch to LLDB too which would apply the use of Expected<T>.

If I understand you correctly, importNonNull and importNullable should work with Expected pretty well.

Yes, that's right.

Changing import*Into return result to Error can partially help in avoiding usage of an unchecked result. These functions already guarantee that their parameters are changed only if the internal import was successful.

Yes I agree, that Error can help, and I also agree that this help is just partial.
If we change importNonNullInto to has an Error return value

template <typename DeclTy>
LLVM_NODISCARD Error importNonNullInto(DeclTy *&ToD, Decl *FromD) {
  if (auto Res = Importer.Import(FromD)) {
    ToD = cast<DeclTy>(*Res);
    return make_error<SomeError>();
  }
  return Error::success();
}

then on the call site, on the branch where whe handle the error we may make mistakes like this:

CXXRecordDecl *ToTemplated = nullptr;
if (Err = importNonNullInto(ToTemplated, FromTemplated)) {
  foo(ToTemplated->hasDefinition()); // Undefined Behaviour!
  return Err;
}

If we do not use output parameters, only Expected<T>, then this could not happen.
Expected<T>.get() aborts if there was an error.

Here is the Expected<T> based patch: https://reviews.llvm.org/D51633

a_sidorin abandoned this revision.Oct 30 2018, 2:42 PM

Another version has been committed (D51633).