[ARM] Mitigate the cve-2021-35465 security vulnurability.

[labrinea]

Summary

Recently a vulnerability issue is found in the implementation of VLLDM instruction in the Arm Cortex-M33, Cortex-M35P and Cortex-M55. If the VLLDM instruction is abandoned due to an exception when it is partially completed, it is possible for subsequent non-secure handler to access and modify the partial restored register values. This vulnerability is identified as CVE-2021-35465. The mitigation sequence varies between v8-m and v8.1-m as follows:

v8-m.main

mrs        r5, control
tst        r5, #8       /* CONTROL_S.SFPA */
it         ne
.inst.w    0xeeb00a40   /* vmovne s0, s0 */
1:
vlldm      sp           /* Lazy restore of d0-d16 and FPSCR. */

v8.1-m.main

vscclrm    {vpr}        /* Clear VPR. */
vlldm      sp           /* Lazy restore of d0-d16 and FPSCR. */

More details on https://developer.arm.com/support/arm-security-updates/vlldm-instruction-security-vulnerability

[ostannard]

clang/lib/Driver/ToolChains/Clang.cpp

1665 ↗

(On Diff #370264)

Are these optional also being passed through to the linker when doing LTO?

clang/test/Driver/arm-cmse-cve-2021-35465.c

4

The last few paragraphs of https://developer.arm.com/support/arm-security-updates/vlldm-instruction-security-vulnerability claim that this is enabled by default for -march=armv8-m.main in AC6 and GCC, is there a reason we're not matching that?

llvm/lib/Target/ARM/ARMExpandPseudoInsts.cpp

1562

Why are these needed?

1627

This pass runs before the final scheduler pass, so is there a risk that the IT and VMOV instructions could be moved apart? I think it would be safer to either put the IT instruction inside the inline asm block, or make this a new pseudo-instruction expanded in the asm printer.

[SjoerdMeijer]

clang/lib/Driver/ToolChains/Clang.cpp

1666 ↗

(On Diff #370264)

If -mcpu=cortex-[m33|m35|m55] was provided, then -arm-fix-cmse-cve-2021-35465=1 is already set and we are adding another option here? For example, for

-mcpu=cortex-m33 -mcmse -mfix-cmse-cve-2021-35465

I am expecting:

"-mllvm" "-arm-fix-cmse-cve-2021-35465=1"  "-mllvm" "-arm-fix-cmse-cve-2021-35465=1"

and with -mno-fix-cmse-cve-2021-35465:

"-mllvm" "-arm-fix-cmse-cve-2021-35465=1"  "-mllvm" "-arm-fix-cmse-cve-2021-35465=0"

Probably it's nicer to just pass this once.

Also, in the tests, I think cases are missing for -mcpu=... and -m[no-]fix-cmse-cve-2021-35465.

[labrinea]

clang/lib/Driver/ToolChains/Clang.cpp

1665 ↗

(On Diff #370264)

No, the mitigation is only performed in the compiler. Also, I believe that -flto and -mcmse are incompatible options.

1666 ↗

(On Diff #370264)

Your interpretetion is correct. The established policy is "last option wins", but I could make the Driver pass only one option if that's preferable.

clang/test/Driver/arm-cmse-cve-2021-35465.c

4

Yes, the inconsistency lies on the fact that GCC implements the mitigation in library code, therefore it is always present for -march=armv8-m.main, whereas in llvm there's no such limitation. We've contacted the authors of this page to rectify the documentation.

llvm/lib/Target/ARM/ARMExpandPseudoInsts.cpp

1562

These prevent the reordering with the mitigation sequence. It answers your next question.

1627

The use of .addReg(ARM::CPSR, RegState::ImplicitDefine) prevents the reordering. There are regression tests in place that check the mitigation sequence ordering.

Is this what you meant? Where you refering specifically to the case where !STI->hasFPRegs(), when we generate inline asm, or to both scenarios?

[ostannard]

clang/lib/Driver/ToolChains/Clang.cpp

1665 ↗

(On Diff #370264)

The mitigation is performed in the backend, which is run from the linker when doing LTO.

Also, I believe that -flto and -mcmse are incompatible options.

Fair enough

1666 ↗

(On Diff #370264)

I agree with Sjoerd, the ""last option wins" policy should be implemented in the driver, and only the winning option passed through to CC1. You can use Args.hasFlag instead of getLastArg here, with the default set based on the CPU option.

clang/test/Driver/arm-cmse-cve-2021-35465.c

4

This still sounds like an inconsistency which will catch out users migrating between GCC and clang. I'd prefer that we match GCC's behaviour, though I'd also be OK with leaving it like this as long as these defaults are clearly documented in ClangCommandLineReference.rst.

llvm/lib/Target/ARM/ARMExpandPseudoInsts.cpp

1562

Do you mean that this is modeling the effect of this instruction on the CONTROL register, to prevent it from being re-ordered with the MRS instruction? If so, then this is unrelated to my next question, and could do with a comment because I wouldn't expect any CONTROL bits to be part of ARM::CPSR (they are different registers).

1627

My concern is that these two instructions (IT and VMOV) have to be adjacent, otherwise the IT would apply to some other instruction, but I don't think there's anything here which guarantees that. I don't think a regression test is enough here, because the re-ordering could happen with a future scheduling model not covered by the tests, or with different code to the tests.

[ostannard]

clang/lib/Driver/ToolChains/Clang.cpp

1665 ↗

(On Diff #370264)

Actually, I just checked and these options are accepted together, and I can't find any docs saying they are incompatible. Do you have a link to something I've missed? Since there isn't already an error, I think we should either fix this to work with LTO (my preference), or add an error when using the options together, and document that.

[labrinea]

clang/lib/Driver/ToolChains/Clang.cpp

1665 ↗

(On Diff #370264)

I have addressed all the other comments, but I am not sure how to go about this one. What does it take to make the cve-2021-35465 option work with LTO? Could you elaborate on this?

[labrinea]

Changes in this revision:

• pass the -arm-fix-cmse-cve-2021-35465 option once
• document -m(no)fix-cmse-cve-2021-35465 in ClangCommandLineReference.rst
• add clang tests with the mitigation expicitely disabled on affected cpus
• removed .addReg(ARM::CPSR, RegState::ImplicitDefine)
• moved the mitigation sequence inside a bundle to prevent reordering

[SjoerdMeijer]

clang/lib/Driver/ToolChains/Clang.cpp

1646 ↗

(On Diff #371028)

Nit: capital F in variable name.

1666 ↗

(On Diff #371028)

I am wondering if this should use getLastArg and what happens with test cases (which I guess need adding) that have both:

-mno-fix-cmse-cve-2021-35465  -mfix-cmse-cve-2021-35465

or

-mfix-cmse-cve-2021-35465  -mno-fix-cmse-cve-2021-35465

[labrinea]

clang/lib/Driver/ToolChains/Clang.cpp

1666 ↗

(On Diff #371028)

That's the whole point of getLastArg as far as I understand: for options that can either enable or disable a feature, so that the last one wins. I'll add more tests.

[SjoerdMeijer]

clang/lib/Driver/ToolChains/Clang.cpp

1666 ↗

(On Diff #371028)

Ah, got confused, it's indeed right there!

[labrinea]

Changes in this revision:

• renamed fix_cve_2021_35465 to Fix_CVE_2021_35465
• added more Driver tests to cover the use of -mfix-cmse-cve-2021-35465 with -mno-fix-cmse-cve-2021-35465
• fixed code indentation

[SjoerdMeijer]

The driver part looks good to me. I will let Oli do the approval as he also looked at the codegen (and I didn't).

clang/include/clang/Driver/Options.td

3284

Nit: I think Arm -> ARM is better as we we're talking about the ARM backend here.

[SjoerdMeijer]

llvm/test/CodeGen/ARM/cmse-cve-2021-35465-return.ll

7

As you're creating machine instructions, I think it is better to run this and the other test with -verify-machineinstrs.

[labrinea]

@ostannard, can you explain what you meant with supporting LTO? I didn't quite undestand. Are you happy with the rest of the changes?

[labrinea]

Changes in this revision:

• Replaced the backend option that enables the mitigation with a subtarget feature so that it works with LTO (@lenary thanks for the offline hint)
• Enabled the subtarget feature on the affected CPUs

[labrinea]

Changes in this revision:

• added -verify-machineinstrs to the tests
• that yield two bugs that I had to address:

*** Bad machine code: Explicit operand marked as def ***
- function:    func
- basic block: %bb.0 entry (0x890b6d8)
- instruction: $d3 = VSTRD $sp, 6, 14, $noreg
- operand 0:   $d3

*** Bad machine code: Explicit definition marked as use ***
- function:    non_secure_call
- basic block: %bb.0  (0x8e0bed8)
- instruction: t2MRS_M $r12, 20, 14, $noreg
- operand 0:   $r12

[lenary]

LGTM, but the most recent way of implementing this (using target features) was something I suggested to Alexandros based on @ostannard's feedback about LTO. I think it is cleaner, and this patch is good, but a re-review from others would be helpful.

clang/docs/ClangCommandLineReference.rst

 .. option:: -ffixed-r9
 
 Reserve the r9 register (ARM only)
 
 .. option:: -mcmse
 
 Allow use of CMSE (Armv8-M Security Extensions)
 
+.. option:: -mfix-cmse-cve-2021-35465, -mno-fix-cmse-cve-2021-35465
+
+Enable the cve-2021-35465 security vulnerability mitigation (ARM only).
+
 .. option:: -mexecute-only, -mno-execute-only, -mpure-code
 
 Disallow generation of data access to code sections (ARM only)
 
 .. option:: -mno-movt
 
 Disallow use of movt/movw pairs (ARM only)
 

clang/include/clang/Driver/Options.td

defm aapcs_bitfield_width : BoolOption<"f", "aapcs-bitfield-width",
   CodeGenOpts<"AAPCSBitfieldWidth">, DefaultTrue,
   NegFlag<SetFalse, [], "Do not follow">, PosFlag<SetTrue, [], "Follow">,
   BothFlags<[NoXarchOption, CC1Option], " the AAPCS standard requirement stating that"
             " volatile bit-field width is dictated by the field container type. (ARM only).">>,
   Group<m_arm_Features_Group>;
 
 def mgeneral_regs_only : Flag<["-"], "mgeneral-regs-only">, Group<m_Group>,
   HelpText<"Generate code which only uses the general purpose registers (AArch64/x86 only)">;
+def mfix_cmse_cve_2021_35465 : Flag<["-"], "mfix-cmse-cve-2021-35465">,
+  Group<m_arm_Features_Group>,
+  HelpText<"Work around VLLDM erratum CVE-2021-35465 (ARM only)">;
+def mno_fix_cmse_cve_2021_35465 : Flag<["-"], "mno-fix-cmse-cve-2021-35465">,
+  Group<m_arm_Features_Group>,
+  HelpText<"Don't work around VLLDM erratum CVE-2021-35465 (ARM only)">;
 def mfix_cortex_a53_835769 : Flag<["-"], "mfix-cortex-a53-835769">,
   Group<m_aarch64_Features_Group>,
   HelpText<"Workaround Cortex-A53 erratum 835769 (AArch64 only)">;
 def mno_fix_cortex_a53_835769 : Flag<["-"], "mno-fix-cortex-a53-835769">,
   Group<m_aarch64_Features_Group>,
   HelpText<"Don't workaround Cortex-A53 erratum 835769 (AArch64 only)">;
 def mmark_bti_property : Flag<["-"], "mmark-bti-property">,
   Group<m_aarch64_Features_Group>,

clang/lib/Driver/ToolChains/Arch/ARM.cpp

if (!((llvm::ARM::parseArchVersion(ArchSuffix) >= 8) &&
       }
     }
   }
 
   // CMSE: Check for target 8M (for -mcmse to be applicable) is performed later.
   if (Args.getLastArg(options::OPT_mcmse))
     Features.push_back("+8msecext");
 
+  if (Arg *A = Args.getLastArg(options::OPT_mfix_cmse_cve_2021_35465,
+                               options::OPT_mno_fix_cmse_cve_2021_35465)) {
+    if (!Args.getLastArg(options::OPT_mcmse))
+      D.Diag(diag::err_opt_not_valid_without_opt)
+          << A->getOption().getName() << "-mcmse";
+
+    if (A->getOption().matches(options::OPT_mfix_cmse_cve_2021_35465))
+      Features.push_back("+fix-cmse-cve-2021-35465");
+    else
+      Features.push_back("-fix-cmse-cve-2021-35465");
+  }
+
   // Look for the last occurrence of -mlong-calls or -mno-long-calls. If
   // neither options are specified, see if we are compiling for kernel/kext and
   // decide whether to pass "+long-calls" based on the OS and its version.
   if (Arg *A = Args.getLastArg(options::OPT_mlong_calls,
                                options::OPT_mno_long_calls)) {
     if (A->getOption().matches(options::OPT_mlong_calls))
       Features.push_back("+long-calls");
   } else if (KernelOrKext && (!Triple.isiOS() || Triple.isOSVersionLT(6)) &&

clang/test/Driver/arm-cmse-cve-2021-35465.c

+// Disable the fix
+//
+// RUN: %clang --target=arm-arm-none-eabi -march=armv8-m.main %s -### \
+// RUN:   -mcmse -mno-fix-cmse-cve-2021-35465 2>&1 |\
+// RUN:   FileCheck %s --check-prefix=CHECK-NOFIX
+//
+// RUN: %clang --target=arm-arm-none-eabi -march=armv8.1-m.main %s -### \
+// RUN:   -mcmse -mno-fix-cmse-cve-2021-35465 2>&1 |\
+// RUN:   FileCheck %s --check-prefix=CHECK-NOFIX
+//
+// RUN: %clang --target=arm-arm-none-eabi -march=armv8-m.main %s -### \
+// RUN:   -mcmse -mfix-cmse-cve-2021-35465 -mno-fix-cmse-cve-2021-35465 2>&1 |\
+// RUN:   FileCheck %s --check-prefix=CHECK-NOFIX
+//
+// CHECK-NOFIX: "-target-feature" "-fix-cmse-cve-2021-35465"
+
+
+// Enable the fix
+//
+// RUN: %clang --target=arm-arm-none-eabi -march=armv8-m.main %s -### \
+// RUN:   -mcmse -mfix-cmse-cve-2021-35465 2>&1 |\
+// RUN:   FileCheck %s --check-prefix=CHECK-FIX
+//
+// RUN: %clang --target=arm-arm-none-eabi -march=armv8.1-m.main %s -### \
+// RUN:   -mcmse -mfix-cmse-cve-2021-35465 2>&1 |\
+// RUN:   FileCheck %s --check-prefix=CHECK-FIX
+//
+// RUN: %clang --target=arm-arm-none-eabi -march=armv8-m.main %s -### \
+// RUN:   -mcmse -mno-fix-cmse-cve-2021-35465 -mfix-cmse-cve-2021-35465 2>&1 |\
+// RUN:   FileCheck %s --check-prefix=CHECK-FIX
+//
+// CHECK-FIX: "-target-feature" "+fix-cmse-cve-2021-35465"
+
+
+// Diagnose the option when used without -mcmse
+//
+// RUN: %clang --target=arm-arm-none-eabi -march=armv8-m.main %s -### \
+// RUN:   -mfix-cmse-cve-2021-35465 2>&1 |\
+// RUN:   FileCheck %s --check-prefix=CHECK-DIAG
+//
+// RUN: %clang --target=arm-arm-none-eabi -march=armv8.1-m.main %s -### \
+// RUN:   -mno-fix-cmse-cve-2021-35465 2>&1 |\
+// RUN:   FileCheck %s --check-prefix=CHECK-DIAG
+//
+// CHECK-DIAG: error: option 'm{{.*}}fix-cmse-cve-2021-35465' cannot be specified without '-mcmse'

llvm/lib/Target/ARM/ARM.td

def FeatureMatMulInt8 : SubtargetFeature<"i8mm", "HasMatMulInt8",
     "true", "Enable Matrix Multiply Int8 Extension", [FeatureNEON]>;
 
 // Armv8.1-M extensions
 
 def FeatureLOB            : SubtargetFeature<"lob", "HasLOB", "true",
                                              "Enable Low Overhead Branch "
                                              "extensions">;
 
+def FeatureFixCMSE_CVE_2021_35465 : SubtargetFeature<"fix-cmse-cve-2021-35465",
+                                        "FixCMSE_CVE_2021_35465", "true",
+                                        "Mitigate against the cve-2021-35465 "
+                                        "security vulnurability">;
+
 //===----------------------------------------------------------------------===//
 // ARM architecture class
 //
 
 // A-series ISA
 def FeatureAClass : SubtargetFeature<"aclass", "ARMProcClass", "AClass",
                                      "Is application profile ('A' series)">;
 

 
 def : ProcessorModel<"cortex-m33", CortexM4Model,       [ARMv8mMainline,
                                                          FeatureDSP,
                                                          FeatureFPARMv8_D16_SP,
                                                          FeaturePrefLoopAlign32,
                                                          FeatureHasSlowFPVMLx,
                                                          FeatureHasSlowFPVFMx,
                                                          FeatureUseMISched,
-                                                         FeatureHasNoBranchPredictor]>;
+                                                         FeatureHasNoBranchPredictor,
+                                                         FeatureFixCMSE_CVE_2021_35465]>;
 
 def : ProcessorModel<"cortex-m35p", CortexM4Model,      [ARMv8mMainline,
                                                          FeatureDSP,
                                                          FeatureFPARMv8_D16_SP,
                                                          FeaturePrefLoopAlign32,
                                                          FeatureHasSlowFPVMLx,
                                                          FeatureHasSlowFPVFMx,
                                                          FeatureUseMISched,
-                                                         FeatureHasNoBranchPredictor]>;
+                                                         FeatureHasNoBranchPredictor,
+                                                         FeatureFixCMSE_CVE_2021_35465]>;
 
 def : ProcessorModel<"cortex-m55", CortexM4Model,      [ARMv81mMainline,
                                                          FeatureDSP,
                                                          FeatureFPARMv8_D16,
                                                          FeatureUseMISched,
                                                          FeatureHasNoBranchPredictor,
                                                          FeaturePrefLoopAlign32,
                                                          FeatureHasSlowFPVMLx,
-                                                         HasMVEFloatOps]>;
+                                                         HasMVEFloatOps,
+                                                         FeatureFixCMSE_CVE_2021_35465]>;
 
 def : ProcNoItin<"cortex-a32",                           [ARMv8a,
                                                          FeatureHWDivThumb,
                                                          FeatureHWDivARM,
                                                          FeatureCrypto,
                                                          FeatureCRC]>;
 
 def : ProcNoItin<"cortex-a35",                          [ARMv8a, ProcA35,

llvm/lib/Target/ARM/ARMExpandPseudoInsts.cpp

void ARMExpandPseudo::CMSERestoreFPRegs(
   else if (STI->hasV8MMainlineOps())
     CMSERestoreFPRegsV8(MBB, MBBI, DL, AvailableRegs);
 }
 
 void ARMExpandPseudo::CMSERestoreFPRegsV8(
     MachineBasicBlock &MBB, MachineBasicBlock::iterator MBBI, DebugLoc &DL,
     SmallVectorImpl<unsigned> &AvailableRegs) {
 
+  // Keep a scratch register for the mitigation sequence.
+  unsigned ScratchReg = ARM::NoRegister;
+  if (STI->fixCMSE_CVE_2021_35465())
+    ScratchReg = AvailableRegs.pop_back_val();
+
   // Use AvailableRegs to store the fp regs
   std::vector<std::tuple<unsigned, unsigned, unsigned>> ClearedFPRegs;
   std::vector<unsigned> NonclearedFPRegs;
   for (const MachineOperand &Op : MBBI->operands()) {
     if (Op.isReg() && Op.isDef()) {
       unsigned Reg = Op.getReg();
       assert(!ARM::DPRRegClass.contains(Reg) ||
              ARM::DPR_VFP2RegClass.contains(Reg));
       assert(!ARM::QPRRegClass.contains(Reg));
       if (ARM::DPR_VFP2RegClass.contains(Reg)) {
         if (AvailableRegs.size() >= 2) {
           unsigned SaveReg2 = AvailableRegs.pop_back_val();
           unsigned SaveReg1 = AvailableRegs.pop_back_val();
           ClearedFPRegs.emplace_back(Reg, SaveReg1, SaveReg2);
 
           // Save the fp register to the normal registers
           BuildMI(MBB, MBBI, DL, TII->get(ARM::VMOVRRD))
               .addReg(SaveReg1, RegState::Define)
               .addReg(SaveReg2, RegState::Define)
               .addReg(Reg)
               .add(predOps(ARMCC::AL));
         } else {
           NonclearedFPRegs.push_back(Reg);
         }
       } else if (ARM::SPRRegClass.contains(Reg)) {
         if (AvailableRegs.size() >= 1) {
           unsigned SaveReg = AvailableRegs.pop_back_val();
           ClearedFPRegs.emplace_back(Reg, SaveReg, 0);
 
void ARMExpandPseudo::CMSERestoreFPRegsV8(
   bool returnsFPReg = (!NonclearedFPRegs.empty() || !ClearedFPRegs.empty());
 
   if (returnsFPReg)
     assert(STI->hasFPRegs() && "Subtarget needs fpregs");
 
   // Push FP regs that cannot be restored via normal registers on the stack
   for (unsigned Reg : NonclearedFPRegs) {
     if (ARM::DPR_VFP2RegClass.contains(Reg))
-      BuildMI(MBB, MBBI, DL, TII->get(ARM::VSTRD), Reg)
+      BuildMI(MBB, MBBI, DL, TII->get(ARM::VSTRD))
+          .addReg(Reg)
           .addReg(ARM::SP)
           .addImm((Reg - ARM::D0) * 2)
           .add(predOps(ARMCC::AL));
     else if (ARM::SPRRegClass.contains(Reg))
-      BuildMI(MBB, MBBI, DL, TII->get(ARM::VSTRS), Reg)
+      BuildMI(MBB, MBBI, DL, TII->get(ARM::VSTRS))
+          .addReg(Reg)
           .addReg(ARM::SP)
           .addImm(Reg - ARM::S0)
           .add(predOps(ARMCC::AL));
   }
 
   // Lazy load fp regs from stack.
   // This executes as NOP in the absence of floating-point support.
-  BuildMI(MBB, MBBI, DL, TII->get(ARM::VLLDM))
+  MachineInstrBuilder VLLDM = BuildMI(MBB, MBBI, DL, TII->get(ARM::VLLDM))
                                   .addReg(ARM::SP)
                                   .add(predOps(ARMCC::AL));
 
+  if (STI->fixCMSE_CVE_2021_35465()) {
+    auto Bundler = MIBundleBuilder(MBB, VLLDM);
+    // Read the CONTROL register.
+    Bundler.append(BuildMI(*MBB.getParent(), DL, TII->get(ARM::t2MRS_M))
+                       .addReg(ScratchReg, RegState::Define)
+                       .addImm(20)
+                       .add(predOps(ARMCC::AL)));
+    // Check bit 3 (SFPA).
+    Bundler.append(BuildMI(*MBB.getParent(), DL, TII->get(ARM::t2TSTri))
+                       .addReg(ScratchReg)
+                       .addImm(8)
+                       .add(predOps(ARMCC::AL)));
+    // Emit the IT block.
+    Bundler.append(BuildMI(*MBB.getParent(), DL, TII->get(ARM::t2IT))
+                       .addImm(ARMCC::NE)
+                       .addImm(8));
+    // If SFPA is clear jump over to VLLDM, otherwise execute an instruction
+    // which has no functional effect apart from causing context creation:
+    // vmovne s0, s0. In the absence of FPU we emit .inst.w 0xeeb00a40,
+    // which is defined as NOP if not executed.
+    if (STI->hasFPRegs())
+      Bundler.append(BuildMI(*MBB.getParent(), DL, TII->get(ARM::VMOVS))
+                         .addReg(ARM::S0, RegState::Define)
+                         .addReg(ARM::S0, RegState::Undef)
+                         .add(predOps(ARMCC::NE)));
+    else
+      Bundler.append(BuildMI(*MBB.getParent(), DL, TII->get(ARM::INLINEASM))
+                         .addExternalSymbol(".inst.w 0xeeb00a40")
+                         .addImm(InlineAsm::Extra_HasSideEffects));
+    finalizeBundle(MBB, Bundler.begin(), Bundler.end());
+  }
+
   // Restore all FP registers via normal registers
   for (const auto &Regs : ClearedFPRegs) {
     unsigned Reg, SaveReg1, SaveReg2;
     std::tie(Reg, SaveReg1, SaveReg2) = Regs;
     if (ARM::DPR_VFP2RegClass.contains(Reg))
       BuildMI(MBB, MBBI, DL, TII->get(ARM::VMOVDRR), Reg)
           .addReg(SaveReg1)
           .addReg(SaveReg2)
static bool definesOrUsesFPReg(const MachineInstr &MI) {
   }
   return false;
 }
 
 void ARMExpandPseudo::CMSERestoreFPRegsV81(
     MachineBasicBlock &MBB, MachineBasicBlock::iterator MBBI, DebugLoc &DL,
     SmallVectorImpl<unsigned> &AvailableRegs) {
   if (!definesOrUsesFPReg(*MBBI)) {
+    if (STI->fixCMSE_CVE_2021_35465()) {
+      BuildMI(MBB, MBBI, DL, TII->get(ARM::VSCCLRMS))
+          .add(predOps(ARMCC::AL))
+          .addReg(ARM::VPR, RegState::Define);
+    }
+
     // Load FP registers from stack.
     BuildMI(MBB, MBBI, DL, TII->get(ARM::VLLDM))
         .addReg(ARM::SP)
         .add(predOps(ARMCC::AL));
 
     // Pop the stack space
     BuildMI(MBB, MBBI, DL, TII->get(ARM::tADDspi), ARM::SP)
         .addReg(ARM::SP)

llvm/lib/Target/ARM/ARMSubtarget.h

protected:
 
   /// Has speculation barrier
   bool HasSB = false;
 
   /// Implicitly convert an instruction to a different one if its immediates
   /// cannot be encoded. For example, ADD r0, r1, #FFFFFFFF -> SUB r0, r1, #1.
   bool NegativeImmediates = true;
 
+  /// Mitigate against the cve-2021-35465 security vulnurability.
+  bool FixCMSE_CVE_2021_35465 = false;
+
   /// Harden against Straight Line Speculation for Returns and Indirect
   /// Branches.
   bool HardenSlsRetBr = false;
 
   /// Harden against Straight Line Speculation for indirect calls.
   bool HardenSlsBlr = false;
 
   /// Generate thunk code for SLS mitigation in the normal text section.
if (CostKind == TargetTransformInfo::TCK_CodeSize)
       return 1;
     return MVEVectorCostFactor;
   }
 
   bool ignoreCSRForAllocationOrder(const MachineFunction &MF,
                                    unsigned PhysReg) const override;
   unsigned getGPRAllocationOrder(const MachineFunction &MF) const;
 
+  bool fixCMSE_CVE_2021_35465() const { return FixCMSE_CVE_2021_35465; }
+
   bool hardenSlsRetBr() const { return HardenSlsRetBr; }
   bool hardenSlsBlr() const { return HardenSlsBlr; }
   bool hardenSlsNoComdat() const { return HardenSlsNoComdat; }
 };
 
 } // end namespace llvm
 
 #endif  // LLVM_LIB_TARGET_ARM_ARMSUBTARGET_H

llvm/test/CodeGen/ARM/cmse-cve-2021-35465-return.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+;
+; RUN: llc %s -o - -mtriple=thumbv8m.main -verify-machineinstrs \
+; RUN:   -mattr=+fp-armv8d16sp,+fix-cmse-cve-2021-35465 -float-abi=hard | \
+; RUN:   FileCheck %s --check-prefix=CHECK-8M-FP-CVE-2021-35465
+
+%indirect = type { double, double, double, double, double, double, double, double }
+
+define %indirect @func(%indirect (float, i32, double, i32, float, i32, float, i32, double, double, double, double, float, float)* %fu, float %a, i32 %b, double %c, i32 %d, float %e, i32 %f, float %g, i32 %h, double %i, double %j, double %k, double %l, float %m, float %n) {
+; CHECK-8M-FP-CVE-2021-35465-LABEL: func:
+; CHECK-8M-FP-CVE-2021-35465:       @ %bb.0: @ %entry
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    push {r7, lr}
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    mov lr, r3
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    mov r12, r0
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    mov r0, r1
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    mov r1, r2
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    ldr r3, [sp, #8]
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    mov r2, lr
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, r11}
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    bic r12, r12, #1
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    sub sp, #136
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vmov r4, s5
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vmov r11, s0
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vmov r9, r10, d1
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vmov r8, s1
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vmov r7, s4
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vmov r5, r6, d3
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vlstm sp
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vmov s0, r11
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vmov d1, r9, r10
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vmov s1, r8
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vmov s4, r7
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vmov d3, r5, r6
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vmov s5, r4
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vldr d4, [sp, #32]
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vldr d5, [sp, #40]
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vldr d6, [sp, #48]
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vldr s14, [sp, #56]
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    ldr r4, [sp, #64]
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    bic r4, r4, #159
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    bic r4, r4, #4026531840
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vmsr fpscr, r4
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    msr apsr_nzcvq, r12
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    blxns r12
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vmov r9, r10, d0
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vstr d3, [sp, #24]
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vmov r7, r8, d1
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vstr d4, [sp, #32]
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vmov r5, r6, d2
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vstr d5, [sp, #40]
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vstr d6, [sp, #48]
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vstr d7, [sp, #56]
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    mrs r11, control
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    tst.w r11, #8
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    it ne
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vmovne.f32 s0, s0
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vlldm sp
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vmov d0, r9, r10
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vmov d1, r7, r8
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vmov d2, r5, r6
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    add sp, #136
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, r11}
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    pop {r7, pc}
+entry:
+  %res = call %indirect %fu(float %a, i32 %b, double %c, i32 %d, float %e, i32 %f, float %g, i32 %h, double %i, double %j, double %k, double %l, float %m, float %n) #0
+  ret %indirect %res
+}
+
+attributes #0 = { "cmse_nonsecure_call" }

llvm/test/CodeGen/ARM/cmse-cve-2021-35465.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+;
+; RUN: llc %s -o - -mtriple=thumbv8m.main -verify-machineinstrs \
+; RUN:   -mattr=+fp-armv8d16sp,+fix-cmse-cve-2021-35465 | \
+; RUN:   FileCheck %s --check-prefix=CHECK-8M-FP-CVE-2021-35465
+;
+; RUN: llc %s -o - -mtriple=thumbv8m.main -mcpu=cortex-m33 -verify-machineinstrs | \
+; RUN:   FileCheck %s --check-prefix=CHECK-8M-FP-CVE-2021-35465
+;
+; RUN: llc %s -o - -mtriple=thumbv8m.main -mcpu=cortex-m35p -verify-machineinstrs | \
+; RUN:   FileCheck %s --check-prefix=CHECK-8M-FP-CVE-2021-35465
+;
+; RUN: llc %s -o - -mtriple=thumbv8m.main -verify-machineinstrs \
+; RUN:   -mattr=-fpregs,+fix-cmse-cve-2021-35465 | \
+; RUN:   FileCheck %s --check-prefix=CHECK-8M-NOFP-CVE-2021-35465
+;
+; RUN: llc %s -o - -mtriple=thumbv8m.main -mcpu=cortex-m33 -mattr=-fpregs -verify-machineinstrs | \
+; RUN:   FileCheck %s --check-prefix=CHECK-8M-NOFP-CVE-2021-35465
+;
+; RUN: llc %s -o - -mtriple=thumbv8m.main -mcpu=cortex-m35p -mattr=-fpregs -verify-machineinstrs | \
+; RUN:   FileCheck %s --check-prefix=CHECK-8M-NOFP-CVE-2021-35465
+;
+; RUN: llc %s -o - -mtriple=thumbv8.1m.main -verify-machineinstrs \
+; RUN:   -mattr=+fp-armv8d16sp,+fix-cmse-cve-2021-35465 | \
+; RUN:   FileCheck %s --check-prefix=CHECK-81M-CVE-2021-35465
+;
+; RUN: llc %s -o - -mtriple=thumbv8.1m.main -mcpu=cortex-m55 -verify-machineinstrs | \
+; RUN:   FileCheck %s --check-prefix=CHECK-81M-CVE-2021-35465
+;
+; RUN: llc %s -o - -mtriple=thumbv8.1m.main -verify-machineinstrs \
+; RUN:   -mattr=-fpregs,+fix-cmse-cve-2021-35465 | \
+; RUN:   FileCheck %s --check-prefix=CHECK-81M-CVE-2021-35465
+;
+; RUN: llc %s -o - -mtriple=thumbv8.1m.main -mcpu=cortex-m55 -mattr=-fpregs -verify-machineinstrs | \
+; RUN:   FileCheck %s --check-prefix=CHECK-81M-CVE-2021-35465
+;
+
+define void @non_secure_call(void ()* %fptr) {
+; CHECK-8M-FP-CVE-2021-35465-LABEL: non_secure_call:
+; CHECK-8M-FP-CVE-2021-35465:       @ %bb.0:
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    push {r7, lr}
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, r11}
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    bic r0, r0, #1
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    sub sp, #136
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vlstm sp
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    mov r1, r0
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    mov r2, r0
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    mov r3, r0
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    mov r4, r0
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    mov r5, r0
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    mov r6, r0
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    mov r7, r0
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    mov r8, r0
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    mov r9, r0
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    mov r10, r0
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    mov r11, r0
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    mov r12, r0
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    msr apsr_nzcvq{{g?}}, r0
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    blxns r0
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    mrs r12, control
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    tst.w r12, #8
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    it ne
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vmovne.f32 s0, s0
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    vlldm sp
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    add sp, #136
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, r11}
+; CHECK-8M-FP-CVE-2021-35465-NEXT:    pop {r7, pc}
+;
+; CHECK-8M-NOFP-CVE-2021-35465-LABEL: non_secure_call:
+; CHECK-8M-NOFP-CVE-2021-35465:       @ %bb.0:
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    push {r7, lr}
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, r11}
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    bic r0, r0, #1
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    sub sp, #136
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    vlstm sp
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    mov r1, r0
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    mov r2, r0
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    mov r3, r0
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    mov r4, r0
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    mov r5, r0
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    mov r6, r0
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    mov r7, r0
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    mov r8, r0
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    mov r9, r0
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    mov r10, r0
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    mov r11, r0
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    mov r12, r0
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    msr apsr_nzcvq{{g?}}, r0
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    blxns r0
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    mrs r12, control
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    tst.w r12, #8
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    it ne
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    @APP
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    .inst.w 0xeeb00a40
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    @NO_APP
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    vlldm sp
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    add sp, #136
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, r11}
+; CHECK-8M-NOFP-CVE-2021-35465-NEXT:    pop {r7, pc}
+;
+; CHECK-81M-CVE-2021-35465-LABEL: non_secure_call:
+; CHECK-81M-CVE-2021-35465:       @ %bb.0:
+; CHECK-81M-CVE-2021-35465-NEXT:    push {r7, lr}
+; CHECK-81M-CVE-2021-35465-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, r11}
+; CHECK-81M-CVE-2021-35465-NEXT:    bic r0, r0, #1
+; CHECK-81M-CVE-2021-35465-NEXT:    sub sp, #136
+; CHECK-81M-CVE-2021-35465-NEXT:    vlstm sp
+; CHECK-81M-CVE-2021-35465-NEXT:    clrm {r1, r2, r3, r4, r5, r6, r7, r8, r9, r10, r11, r12, apsr}
+; CHECK-81M-CVE-2021-35465-NEXT:    blxns r0
+; CHECK-81M-CVE-2021-35465-NEXT:    vscclrm {vpr}
+; CHECK-81M-CVE-2021-35465-NEXT:    vlldm sp
+; CHECK-81M-CVE-2021-35465-NEXT:    add sp, #136
+; CHECK-81M-CVE-2021-35465-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, r11}
+; CHECK-81M-CVE-2021-35465-NEXT:    pop {r7, pc}
+  call void %fptr() #0
+  ret void
+}
+
+attributes #0 = { "cmse_nonsecure_call" }

llvm/test/CodeGen/ARM/cmse-vlldm-no-reorder.mir

-# RUN: llc -mtriple=thumbv8m.main -mcpu=cortex-m33 --float-abi=hard --run-pass=arm-pseudo %s -o - | \
+# RUN: llc -mtriple=thumbv8m.main -mcpu=cortex-m33 -mattr=-fix-cmse-cve-2021-35465 --float-abi=hard --run-pass=arm-pseudo %s -o - | \
 # RUN: FileCheck %s
 --- |
   ; ModuleID = 'cmse-vlldm-no-reorder.ll'
   source_filename = "cmse-vlldm-no-reorder.ll"
   target datalayout = "e-m:e-p:32:32-Fi8-i64:64-v128:64:128-a:0:32-n32-S64"
   target triple = "thumbv8m.main"
   
   @g = hidden local_unnamed_addr global float (...)* null, align 4

 # CHECK-NEXT:  $r12 = tMOVr $r0, 14 /* CC::al */, $noreg
 # CHECK-NEXT:  t2MSR_M 3072, $r0, 14 /* CC::al */, $noreg, implicit-def $cpsr
 # CHECK-NEXT:  tBLXNSr 14 /* CC::al */, $noreg, killed $r0, csr_aapcs, implicit-def $lr, implicit $sp, implicit-def dead $lr, implicit $sp, implicit-def $sp, implicit-def $s0
 # CHECK-NEXT:  $r12 = VMOVRS $s0, 14 /* CC::al */, $noreg
 # CHECK-NEXT:  VLLDM $sp, 14 /* CC::al */, $noreg, implicit-def $q0, implicit-def $q1, implicit-def $q2, implicit-def $q3, implicit-def $q4, implicit-def $q5, implicit-def $q6, implicit-def $q7, implicit-def $vpr, implicit-def $fpscr, implicit-def $fpscr_nzcv
 # CHECK-NEXT:  $s0 = VMOVSR $r12, 14 /* CC::al */, $noreg
 # CHECK-NEXT:  $sp = tADDspi $sp, 34, 14 /* CC::al */, $noreg
 # CHECK-NEXT:  $sp = t2LDMIA_UPD $sp, 14 /* CC::al */, $noreg, def $r4, def $r5, def $r6, def $r7, def $r8, def $r9, def $r10, def $r11
+