This is an archive of the discontinued LLVM Phabricator instance.

[X86] initial -mfunction-return=thunk-extern support
ClosedPublic

Authored by nickdesaulniers on Jul 12 2022, 9:07 AM.

Details

Summary

Adds support for:

  • -mfunction-return=<value> command line flag, and
  • __attribute__((function_return("<value>"))) function attribute

Where the supported <value>s are:

  • keep (disable)
  • thunk-extern (enable)

thunk-extern enables clang to change ret instructions into jmps to an
external symbol named __x86_return_thunk, implemented as a new
MachineFunctionPass named "x86-return-thunks", keyed off the new IR
attribute fn_ret_thunk_extern.

The symbol x86_return_thunk is expected to be provided by the runtime
the compiled code is linked against and is not defined by the compiler.
Enabling this option alone doesn't provide mitigations without
corresponding definitions of
x86_return_thunk!

This new MachineFunctionPass is very similar to "x86-lvi-ret".

The <value>s "thunk" and "thunk-inline" are currently unsupported. It's
not clear yet that they are necessary: whether the thunk pattern they
would emit is beneficial or used anywhere.

Should the <value>s "thunk" and "thunk-inline" become necessary,
x86-return-thunks could probably be merged into x86-retpoline-thunks
which has pre-existing machinery for emitting thunks (which could be
used to implement the <value> "thunk").

Has been found to build+boot with corresponding Linux
kernel patches. This helps the Linux kernel mitigate RETBLEED.

  • CVE-2022-23816
  • CVE-2022-28693
  • CVE-2022-29901

See also:

  • "RETBLEED: Arbitrary Speculative Code Execution with Return

Instructions."

  • AMD SECURITY NOTICE AMD-SN-1037: AMD CPU Branch Type Confusion
  • TECHNICAL GUIDANCE FOR MITIGATING BRANCH TYPE CONFUSION REVISION 1.0 2022-07-12
  • Return Stack Buffer Underflow / Return Stack Buffer Underflow / CVE-2022-29901, CVE-2022-28693 / INTEL-SA-00702

SystemZ may eventually want to support "thunk-extern" and "thunk"; both
options are used by the Linux kernel's CONFIG_EXPOLINE.

This functionality has been available in GCC since the 8.1 release, and
was backported to the 7.3 release.

Many thanks for folks that provided discrete review off list due to the
embargoed nature of this hardware vulnerability. Many Bothans died to
bring us this information.

Link: https://www.youtube.com/watch?v=IF6HbCKQHK8
Link: https://github.com/llvm/llvm-project/issues/54404
Link: https://gcc.gnu.org/legacy-ml/gcc-patches/2018-01/msg01197.html
Link: https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/advisory-guidance/return-stack-buffer-underflow.html
Link: https://arstechnica.com/information-technology/2022/07/intel-and-amd-cpus-vulnerable-to-a-new-speculative-execution-attack/?comments=1
Link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=ce114c866860aa9eae3f50974efc68241186ba60

Diff Detail

Event Timeline

Herald added a project: Restricted Project. · View Herald Transcript
nickdesaulniers requested review of this revision.Jul 12 2022, 9:07 AM
Herald added projects: Restricted Project, Restricted Project. · View Herald TranscriptJul 12 2022, 9:07 AM
aaron.ballman accepted this revision.Jul 12 2022, 9:09 AM

Many thanks for folks that provided discrete review off list due to the embargoed nature of this hardware vulnerability.

I was one of those off-list reviewers, and the Clang parts LGTM!

This revision is now accepted and ready to land.Jul 12 2022, 9:09 AM
craig.topper accepted this revision.Jul 12 2022, 9:16 AM

I also reviewed off-list and the X86 parts LGTM

  • add more links to commit message as they come in
This revision was landed with ongoing or failed builds.Jul 12 2022, 9:18 AM
This revision was automatically updated to reflect the committed changes.
  • add more links to commit message as they come in

Oh, right, updating the commit message doesn't update the description in phab. Oops!

Did a post-commit review on the CFE changes, and all look OK to me. That FIXME is a shame, we should see if we can fix that ASAP. We should AT LEAST document in the FIXME what semantics we are looking to emulate from GCC, and what those semantics ARE.

Did a post-commit review on the CFE changes, and all look OK to me.

Thanks for the review!

That FIXME is a shame, we should see if we can fix that ASAP. We should AT LEAST document in the FIXME what semantics we are looking to emulate from GCC, and what those semantics ARE.

Eh, so the discussion I had off list with @aaron.ballman was:

"Ok, here's where I feel like I now have a catch-22. I now have:

diff --git a/clang/include/clang/Sema/Sema.h b/clang/include/clang/Sema/Sema.h
index 7d33b5047a67..73afedad4a9d 100644
--- a/clang/include/clang/Sema/Sema.h
+++ b/clang/include/clang/Sema/Sema.h
@@ -3496,6 +3496,9 @@ public:
                                               int X, int Y, int Z);
   HLSLShaderAttr *mergeHLSLShaderAttr(Decl *D, const AttributeCommonInfo &AL,
                                       HLSLShaderAttr::ShaderType ShaderType);
+  FunctionReturnThunksAttr *
+  mergeFunctionReturnThunksAttr(Decl *D, const AttributeCommonInfo &AL,
+                                const FunctionReturnThunksAttr::Kind K);

   void mergeDeclAttributes(NamedDecl *New, Decl *Old,
                            AvailabilityMergeKind AMK = AMK_Redeclaration);
diff --git a/clang/lib/Sema/SemaDecl.cpp b/clang/lib/Sema/SemaDecl.cpp
index f2b87c6d2e37..7e9507735b93 100644
--- a/clang/lib/Sema/SemaDecl.cpp
+++ b/clang/lib/Sema/SemaDecl.cpp
@@ -2809,6 +2809,8 @@ static bool mergeDeclAttribute(Sema &S, NamedDecl *D,
         S.mergeHLSLNumThreadsAttr(D, *NT, NT->getX(), NT->getY(), NT->getZ());
   else if (const auto *SA = dyn_cast<HLSLShaderAttr>(Attr))
     NewAttr = S.mergeHLSLShaderAttr(D, *SA, SA->getType());
+  else if (const auto *FA = dyn_cast<FunctionReturnThunksAttr>(Attr))
+    NewAttr = S.mergeFunctionReturnThunksAttr(D, *FA, FA->getThunkType());
   else if (Attr->shouldInheritEvenIfAlreadyPresent() || !DeclHasAttr(D, Attr))
     NewAttr = cast<InheritableAttr>(Attr->clone(S.Context));
 diff --git a/clang/lib/Sema/SemaDeclAttr.cpp b/clang/lib/Sema/SemaDeclAttr.cpp
index 9b127fbc395b..69c18a48680a 100644
--- a/clang/lib/Sema/SemaDeclAttr.cpp
+++ b/clang/lib/Sema/SemaDeclAttr.cpp
@@ -6973,6 +6973,19 @@ Sema::mergeHLSLShaderAttr(Decl *D, const
AttributeCommonInfo &AL,
   return HLSLShaderAttr::Create(Context, ShaderType, AL);
 }

+FunctionReturnThunksAttr *
+Sema::mergeFunctionReturnThunksAttr(Decl *D, const AttributeCommonInfo &AL,
+                                    const FunctionReturnThunksAttr::Kind K) {
+  if (const auto *FRT = D->getAttr<FunctionReturnThunksAttr>()) {
+    if (FRT->getThunkType() != K) {
+      Diag(AL.getLoc(), diag::warn_duplicate_attribute) << AL;
+      Diag(FRT->getLoc(), diag::note_conflicting_attribute);
+    }
+    return nullptr;
+  }
+  return FunctionReturnThunksAttr::Create(Context, K, AL);
+}
+
 static void handleMSInheritanceAttr(Sema &S, Decl *D, const ParsedAttr &AL) {
   if (!S.LangOpts.CPlusPlus) {
     S.Diag(AL.getLoc(), diag::err_attribute_not_supported_in_lang)
@@ -8043,8 +8056,10 @@ static void handleFunctionReturnThunksAttr(Sema
&S, Decl *D,
         << AL << KindStr;
     return;
   }
-  D->dropAttr<FunctionReturnThunksAttr>();
-  D->addAttr(FunctionReturnThunksAttr::Create(S.Context, Kind, AL));
+
+  if (FunctionReturnThunksAttr *FA =
+          S.mergeFunctionReturnThunksAttr(D, AL, Kind))
+    D->addAttr(FA);
 }

 static void handleSYCLKernelAttr(Sema &S, Decl *D, const ParsedAttr &AL) {

Consider the following 2 cases.

Case 1:

__attribute__((function_return("thunk-extern"))) void f(void);
__attribute__((function_return("keep"))) void f(void) {};
$ clang z.c
z.c:1:16: warning: attribute 'function_return' is already applied with
different arguments [-Wignored-attributes]
__attribute__((function_return("thunk-extern"))) void f(void);
               ^
z.c:2:16: note: conflicting attribute is here
__attribute__((function_return("keep"))) void f(void) {};
               ^

This LGTM; we want to keep the latest version. So the warning is on
the initial declaration that its attribute will be ignored, and we
point to the definition to identify the conflict. LGTM.

But now consider this case:

__attribute__((function_return("thunk-extern")))
__attribute__((function_return("keep")))
void double_thunk_keep(void) {}
$ clang z.c
z.c:18:16: warning: attribute 'function_return' is already applied
with different arguments [-Wignored-attributes]
__attribute__((function_return("keep")))
               ^
z.c:17:16: note: conflicting attribute is here
__attribute__((function_return("thunk-extern")))
               ^

No bueno. We want to point to the first instance as the one being
ignored ("thunk-extern"), not ("keep").

If I flip the Diags around in my added
Sema::mergeFunctionReturnThunksAttr(), then the second case passes
but the first now fails...i.e.

Diag(FRT->getLoc(), diag::warn_duplicate_attribute) << FRT;
Diag(AL.getLoc(), diag::note_conflicting_attribute);

instead of

Diag(AL.getLoc(), diag::warn_duplicate_attribute) << AL;
Diag(FRT->getLoc(), diag::note_conflicting_attribute);

Any idea what I'm doing wrong?

(The behavior of GCC is to keep the last occurence of this fn attr,
whether through multiple redeclarations, or one attribute with
multiple instances of function_return).

I feel like the semantics of the merging routines would be clearer if
they were passed the previous Decl, and the new Decl."


Personally, I don't feel the need to fix attribute merging in clang "ASAP." It does seem like a general problem, not specific to _this_ particular fn attr. Hence the FIXME.

Did a post-commit review on the CFE changes, and all look OK to me.

Thanks for the review!

That FIXME is a shame, we should see if we can fix that ASAP. We should AT LEAST document in the FIXME what semantics we are looking to emulate from GCC, and what those semantics ARE.

Eh, so the discussion I had off list with @aaron.ballman was:

"Ok, here's where I feel like I now have a catch-22. I now have:

diff --git a/clang/include/clang/Sema/Sema.h b/clang/include/clang/Sema/Sema.h
index 7d33b5047a67..73afedad4a9d 100644
--- a/clang/include/clang/Sema/Sema.h
+++ b/clang/include/clang/Sema/Sema.h
@@ -3496,6 +3496,9 @@ public:
                                               int X, int Y, int Z);
   HLSLShaderAttr *mergeHLSLShaderAttr(Decl *D, const AttributeCommonInfo &AL,
                                       HLSLShaderAttr::ShaderType ShaderType);
+  FunctionReturnThunksAttr *
+  mergeFunctionReturnThunksAttr(Decl *D, const AttributeCommonInfo &AL,
+                                const FunctionReturnThunksAttr::Kind K);

   void mergeDeclAttributes(NamedDecl *New, Decl *Old,
                            AvailabilityMergeKind AMK = AMK_Redeclaration);
diff --git a/clang/lib/Sema/SemaDecl.cpp b/clang/lib/Sema/SemaDecl.cpp
index f2b87c6d2e37..7e9507735b93 100644
--- a/clang/lib/Sema/SemaDecl.cpp
+++ b/clang/lib/Sema/SemaDecl.cpp
@@ -2809,6 +2809,8 @@ static bool mergeDeclAttribute(Sema &S, NamedDecl *D,
         S.mergeHLSLNumThreadsAttr(D, *NT, NT->getX(), NT->getY(), NT->getZ());
   else if (const auto *SA = dyn_cast<HLSLShaderAttr>(Attr))
     NewAttr = S.mergeHLSLShaderAttr(D, *SA, SA->getType());
+  else if (const auto *FA = dyn_cast<FunctionReturnThunksAttr>(Attr))
+    NewAttr = S.mergeFunctionReturnThunksAttr(D, *FA, FA->getThunkType());
   else if (Attr->shouldInheritEvenIfAlreadyPresent() || !DeclHasAttr(D, Attr))
     NewAttr = cast<InheritableAttr>(Attr->clone(S.Context));
 diff --git a/clang/lib/Sema/SemaDeclAttr.cpp b/clang/lib/Sema/SemaDeclAttr.cpp
index 9b127fbc395b..69c18a48680a 100644
--- a/clang/lib/Sema/SemaDeclAttr.cpp
+++ b/clang/lib/Sema/SemaDeclAttr.cpp
@@ -6973,6 +6973,19 @@ Sema::mergeHLSLShaderAttr(Decl *D, const
AttributeCommonInfo &AL,
   return HLSLShaderAttr::Create(Context, ShaderType, AL);
 }

+FunctionReturnThunksAttr *
+Sema::mergeFunctionReturnThunksAttr(Decl *D, const AttributeCommonInfo &AL,
+                                    const FunctionReturnThunksAttr::Kind K) {
+  if (const auto *FRT = D->getAttr<FunctionReturnThunksAttr>()) {
+    if (FRT->getThunkType() != K) {
+      Diag(AL.getLoc(), diag::warn_duplicate_attribute) << AL;
+      Diag(FRT->getLoc(), diag::note_conflicting_attribute);
+    }
+    return nullptr;
+  }
+  return FunctionReturnThunksAttr::Create(Context, K, AL);
+}
+
 static void handleMSInheritanceAttr(Sema &S, Decl *D, const ParsedAttr &AL) {
   if (!S.LangOpts.CPlusPlus) {
     S.Diag(AL.getLoc(), diag::err_attribute_not_supported_in_lang)
@@ -8043,8 +8056,10 @@ static void handleFunctionReturnThunksAttr(Sema
&S, Decl *D,
         << AL << KindStr;
     return;
   }
-  D->dropAttr<FunctionReturnThunksAttr>();
-  D->addAttr(FunctionReturnThunksAttr::Create(S.Context, Kind, AL));
+
+  if (FunctionReturnThunksAttr *FA =
+          S.mergeFunctionReturnThunksAttr(D, AL, Kind))
+    D->addAttr(FA);
 }

 static void handleSYCLKernelAttr(Sema &S, Decl *D, const ParsedAttr &AL) {

Consider the following 2 cases.

Case 1:

__attribute__((function_return("thunk-extern"))) void f(void);
__attribute__((function_return("keep"))) void f(void) {};
$ clang z.c
z.c:1:16: warning: attribute 'function_return' is already applied with
different arguments [-Wignored-attributes]
__attribute__((function_return("thunk-extern"))) void f(void);
               ^
z.c:2:16: note: conflicting attribute is here
__attribute__((function_return("keep"))) void f(void) {};
               ^

This LGTM; we want to keep the latest version. So the warning is on
the initial declaration that its attribute will be ignored, and we
point to the definition to identify the conflict. LGTM.

But now consider this case:

__attribute__((function_return("thunk-extern")))
__attribute__((function_return("keep")))
void double_thunk_keep(void) {}
$ clang z.c
z.c:18:16: warning: attribute 'function_return' is already applied
with different arguments [-Wignored-attributes]
__attribute__((function_return("keep")))
               ^
z.c:17:16: note: conflicting attribute is here
__attribute__((function_return("thunk-extern")))
               ^

No bueno. We want to point to the first instance as the one being
ignored ("thunk-extern"), not ("keep").

If I flip the Diags around in my added
Sema::mergeFunctionReturnThunksAttr(), then the second case passes
but the first now fails...i.e.

Diag(FRT->getLoc(), diag::warn_duplicate_attribute) << FRT;
Diag(AL.getLoc(), diag::note_conflicting_attribute);

instead of

Diag(AL.getLoc(), diag::warn_duplicate_attribute) << AL;
Diag(FRT->getLoc(), diag::note_conflicting_attribute);

Any idea what I'm doing wrong?

(The behavior of GCC is to keep the last occurence of this fn attr,
whether through multiple redeclarations, or one attribute with
multiple instances of function_return).

I feel like the semantics of the merging routines would be clearer if
they were passed the previous Decl, and the new Decl."


Personally, I don't feel the need to fix attribute merging in clang "ASAP." It does seem like a general problem, not specific to _this_ particular fn attr. Hence the FIXME.

Are we not evaluating attributes in lexical order? I could have sworn we fixed that not long ago. I think part of the issue is that you want to merge these 'differently' than we typically do. Our typical rule is to keep the 1st one I think, and reject the 2nd.

I suspect you'd need a new diagnostic for this since this is a 'new' behavior. Something like, `attribute %0 replaces conflicting attribute on this declaration' (then the note). I think this is important to diagnose (which is why I said ASAP), as this seems like a somewhat easy thing to miss, and likely causes shocking/dangerous behavior when done 'wrong'. AND since this is a 'security concern' it moves from "we should fix this soon" to "ASAP" for me.

The other open question for me is how we handle attributes-after-definition:

__attribute__((function_return("keep")))
void foo(void){}
__attribute__((function_return("thunk-extern")))
void foo(void);

OR

void foo(void){}
__attribute__((function_return("thunk-extern")))
void foo(void);

One COULD decide that the 'newest declaration' wins here, but, IMO that is the wrong answer. This makes something where mis-ordered #includes could change the behavior here in a shocking way, particularly without a diagnostic.

_I_ believe that 'definition should win', but that isn't something we do with attributes generally I believe (though ones that have semantic effect might enforce this ad-hoc).

ALSO, I wonder if the 'conflicting attribute' warning for THIS attribute might be better as an error, simply because of the security concerns.

Our typical rule is to keep the 1st one I think, and reject the 2nd.

But then the _codegen_ will differ from GCC. And we _want_ clang to be a drop in replacement, so differing in behavior there is unacceptable.

https://godbolt.org/z/rf16T83Kj

IMO, the standards bodies focusing on standardizing attributes should clarify the semantics of attribute merging, _then_ compiler vendors should fix their compilers.

Our typical rule is to keep the 1st one I think, and reject the 2nd.

But then the _codegen_ will differ from GCC. And we _want_ clang to be a drop in replacement, so differing in behavior there is unacceptable.

Right, I get that, which is why I said we need a different diagnostic than is 'typical' attribute merging.

https://godbolt.org/z/rf16T83Kj

IMO, the standards bodies focusing on standardizing attributes should clarify the semantics of attribute merging, _then_ compiler vendors should fix their compilers.

They HAVE FWIW, by not creating attributes that have a 'merge' problem(yet). They end up being able to be completely additive (with the exception of the 'reason' ones like nodiscard/deprecated, at which point the standards decide that is implementation defined IIRC).

https://godbolt.org/z/rf16T83Kj

IMO, the standards bodies focusing on standardizing attributes should clarify the semantics of attribute merging, _then_ compiler vendors should fix their compilers.

They HAVE FWIW, by not creating attributes that have a 'merge' problem(yet). They end up being able to be completely additive (with the exception of the 'reason' ones like nodiscard/deprecated, at which point the standards decide that is implementation defined IIRC).

In case of a conflict, picking the first does not appear to be universal in GCC attributes. I haven't made a thorough survey,
but visibility picks the first and patchable_function_entry picks the second. Someone can ask what should be the rule and which attribute behavior should be considered a bug.

https://godbolt.org/z/rf16T83Kj

IMO, the standards bodies focusing on standardizing attributes should clarify the semantics of attribute merging, _then_ compiler vendors should fix their compilers.

They HAVE FWIW, by not creating attributes that have a 'merge' problem(yet). They end up being able to be completely additive (with the exception of the 'reason' ones like nodiscard/deprecated, at which point the standards decide that is implementation defined IIRC).

In case of a conflict, picking the first does not appear to be universal in GCC attributes. I haven't made a thorough survey,
but visibility picks the first and patchable_function_entry picks the second. Someone can ask what should be the rule and which attribute behavior should be considered a bug.

Correct, there are examples of vendor attributes which want to resolve in both directions when merging. I don't envision the standard specifying the merging behavior because I think that's going to need to be on a per-attribute basis and so I would imagine it would remain implementation defined.

clang/test/CodeGen/attr-function-return.c
42

I just noticed this typo, from an earlier version that did implement thunk-keep (which isn't useful ATM). So I should fix this up; nothing is being merged here.

I need to s/thunk-//.