Implement DW_CFA_LLVM_* for Heterogeneous Debugging

[scott.linder]

Summary

Add support in MC/MIR for writing/parsing, and DebugInfo.

This is part of the Extensions for Heterogeneous Debugging defined at
https://llvm.org/docs/AMDGPUDwarfExtensionsForHeterogeneousDebugging.html

Specifically the CFI instructions implemented here are defined at
https://llvm.org/docs/AMDGPUDwarfExtensionsForHeterogeneousDebugging.html#cfa-definition-instructions

[scott.linder]

git clang-format

[scott.linder]

The new operation is defined at https://llvm.org/docs/AMDGPUUsage.html#cfa-definition-instructions. @t-tye is working on formatting the proposal to be sent to upstream DWARF as an RFC, and I am working to land these patches as an initial implementation.

[scott.linder]

Update to match changes in proposal.

[scott.linder]

Update encodings of DW_OP_CFA_* to avoid conflict with a GNU extension.

[scott.linder]

Update docs for encoding change

[RamNalamothu]

llvm/include/llvm/DebugInfo/DWARF/DWARFDebugFrame.h

36

Now, maximum number of operands is 3

llvm/lib/MC/MCDwarf.cpp

1446

Emitting DW_CFA_def_aspace_cfa_sf is yet to be handled?

I see that even the emission of DWARF 5 supported DW_CFA_def_cfa_sf and DW_CFA_def_cfa_offset_sf are also yet to be handled.

[scott.linder]

llvm/include/llvm/DebugInfo/DWARF/DWARFDebugFrame.h

36

Thank you, I'll update the patch.

llvm/lib/MC/MCDwarf.cpp

1446

Right, this should probably be a TODO in the code. As the existing _sf variants are not implemented yet either and AMDGPU does not currently require any of them (we always retain a copy of the stack pointer on entry in some form) I didn't implement it here.

[scott.linder]

Address feedback

[arsenm]

llvm/lib/CodeGen/MIRParser/MILexer.cpp

215–218

Unrelated change

236–238

Unrelated change

llvm/lib/MC/MCParser/AsmParser.cpp

4195

Tests missing for the parse errors?

[scott.linder]

Add tests for parse errors, fix formatting

llvm/lib/CodeGen/MIRParser/MILexer.cpp

215–218

I wish all of this rote mechanical stuff were just automated. Do you have a preference on how I resolve this? The options I'm aware of:

• I can factor out every clang-format diff into a separate NFC review, but effectively every non-trivial patch would need one, so I would end up having to manage on the order of twice as many Phabricator reviews.
• I can make one pass at the very end to try to pull all the unrelated NFC formatting changes into one commit, but I would rather do this just before committing so I am not constantly having to comb through every patch in the set trying to clean them up.
• I can aggressively commit these NFC cleanups as separate patches without reviews as they come up, before posting the actual review.

Or I can lobby on llvm-dev to just lint/format the whole codebase once, and then gate commits on patches not breaking it. This would probably save everyone too much time, and make git blame too useful though.

[dblaikie]

llvm/lib/CodeGen/MIRParser/MILexer.cpp

215–218

Most of it is automated or can be - if you're only clang-formatting the lines of code you've edited, it's usually enough - just in this instance happens to be a very long statement & so it gets reformatted together even if you changed unrelated lines in the statement. /maybe/ clang-format could be modified to not do that (to be able to reformat only the edited lines, rather than the whole statement)

Something like this one - yeah, you could commit that specific formatting (not reformatting the whole file) in advance to ensure the review is simpler/clearer. I think this fits under the general idea of reformatting early if you're making significant changes - like if you're majorly reworking a whole file, it's not uncommon to reformat the whole file. But yeah, if you're touching a statement/block in depth and it's significantly misformatted, easy thing to fix/commit in advance - doesn't need a formal pre-commit review (so long as it's code that's going to be touched/changed soon - don't go reformatting whole files without such motivation, unless they're really egregiously formatted).

[scott.linder]

llvm/lib/CodeGen/MIRParser/MILexer.cpp

215–218

Is there any document with the rationale for why LLVM prefers incrementally massaging code into the canonical format, rather than just fixing the entire codebase at once? Like a section in some docs somewhere, or an old llvm-dev thread? It might make the pill easier to swallow if I understand the ideal to which I'm dedicating time fiddling with it.

[scott.linder]

Use llvm-readelf for more compact tests

[scott.linder]

Correct test file names. I missed this when renaming the CFA operation itself
to match the updated proposal.

[RamNalamothu]

Rebase and address feedback.

[RamNalamothu]

Rebase and update CFIProgram::parse (DWARFDebugFrame.cpp) changes as
per https://reviews.llvm.org/rG1e820e82b1438a52124512175a0e7c6f8d23e158

[RamNalamothu]

Rebase onto LLVM master

[scott.linder]

Rebase, update x86 test (dwarfdump now prefers register names for some
targets), update link to the extensions in a comment.

[scott.linder]

Also, ping; is there anyone able to take a look?

[MaskRay]

In D76877#1946283, @scott.linder wrote:

The new operation is defined at https://llvm.org/docs/AMDGPUUsage.html#cfa-definition-instructions. @t-tye is working on formatting the proposal to be sent to upstream DWARF as an RFC, and I am working to land these patches as an initial implementation.

Can you link the documentation in the summary? Currently https://llvm.org/docs/AMDGPUUsage.html does not mention DW_CFA_LLVM_

llvm/lib/MC/MCDwarf.cpp

1446

The variables are used only once and can be inlined

1449

The negative offset looks strange.

Before May, there was a double negation problem which has been fixed by 0840d725c4e7c98bb440db7b886054525be6ddf1

llvm/test/MC/ELF/cfi-llvm-def-aspace-cfa-errors.s

8

[[@LINE]] is deprecated syntax. Please use [[#@LINE+1]] for new tests.

[scott.linder]

Address review comments:

• Include link to extension docs in commit message
• Eliminate some variables
• Fix double-negation in createLLVMDefAspaceCfa
• Fix use of some deprecated lit syntax

[scott.linder]

In D76877#2322112, @MaskRay wrote:

In D76877#1946283, @scott.linder wrote:

The new operation is defined at https://llvm.org/docs/AMDGPUUsage.html#cfa-definition-instructions. @t-tye is working on formatting the proposal to be sent to upstream DWARF as an RFC, and I am working to land these patches as an initial implementation.

Can you link the documentation in the summary? Currently https://llvm.org/docs/AMDGPUUsage.html does not mention DW_CFA_LLVM_

Thank you for the review! I included the updated link directly to the documentation for these new instructions in the commit message.

[scott.linder]

Tweak which links are present where: link directly to the description of the
new CFI operations in the source, and put both links in the commit message.

[RamNalamothu]

Rebase on top of LLVM master

[RamNalamothu]

Rebase and extend the patch to cover https://reviews.llvm.org/rGf8122d35325d9a5d3db7d4c0d40bcdeae15bee5a.

[clayborg]

llvm/include/llvm/DebugInfo/DWARF/DWARFDebugFrame.h

28

Is this the address segment for the address of the row? If so we should change "Address" above to be a:

Optional<object::SectionedAddress> Address;

If this address space qualifies the CFAValue below, then we should move it into UnwindLocation as a "Optional<uint32_t> AddressSpace;" variable

31

You are returning uint8_t but storing this as a CFAAddressSpace as a uint32_t above.

36

Make a constexpr for MaxOperands and use it everywhere so if we bump this number again, we don't need to modify a bunch of places in the code.

llvm/lib/DebugInfo/DWARF/DWARFDebugFrame.cpp

47

not sure printing something like "as2" will be easy for people to know that this means "address space 2". Maybe "address space 2" would be better?

190–191

190–191

250

258

306

" address space %"?

[RamNalamothu]

llvm/include/llvm/DebugInfo/DWARF/DWARFDebugFrame.h

28

Is this the address segment for the address of the row? If so we should change "Address" above to be a:

Optional<object::SectionedAddress> Address;

If this address space qualifies the CFAValue below, then we should move it into UnwindLocation as a "Optional<uint32_t> AddressSpace;" variable

28

No, the address space here qualifies the CFA and specifies, among the different address spaces the target supports, in which address space the relevant stack frame (CFA location) resides. For the sake of understanding this, think of the address spaces as different memory regions, with unique characteristics.

Thought about moving it into UnwindLocation but I noticed that UnwindLocation is being used by both CFA and register rules which, theoretically, making it possible to say CFA's location is CFAPlusOffset and also looks like there aren't enough checks to catch this scenario. Being not sure, I have added it to UnwindRow which has CFAValue.

Does it make sense to split UnwindLocation suitably, so that CFA and register rules could use only what is possible as per DWARF spec?

31

Typo, will fix it. Thank you.

36

Will change it.

llvm/lib/DebugInfo/DWARF/DWARFDebugFrame.cpp

47

Yes.

How about "addrspace 2", which is similar to what LLVM IR/MIR uses?

190–191

Will change.

[RamNalamothu]

Addressed the feedback.

This revision is now ready for review again.

[clayborg]

llvm/include/llvm/DebugInfo/DWARF/DWARFDebugFrame.h

28

Thought about moving it into UnwindLocation but I noticed that UnwindLocation is being used by both CFA and register rules which, theoretically, making it possible to say CFA's location is CFAPlusOffset and also looks like there aren't enough checks to catch this scenario. Being not sure, I have added it to UnwindRow which has CFAValue.

UnwindLocation does get used for both CFA and register values, and yes a UnwindLocation does have more values for registers, but since we since CFA values are parsed using special opcodes, there should be no need to worry as those are what creates the right locations.

That being said, I think moving the address space into UnwindLocation seems like the right thing to do here. Nothing that parses the regular register locations should be adding an address space to the location, though I could see this being needed for some registers locations in the future.

Does it make sense to split UnwindLocation suitably, so that CFA and register rules could use only what is possible as per DWARF spec?

I believe the LLDB unwind code does this but I feel this splits up the location code a bit since the CFA usually is register + offset or DWARF expression which are both valid register rules. You could make a CFALocation that inherits from UnwindLocation and adds the address space if that makes things more clear? Then you can also add some asserts when someone tries to set the location to something that the DWARF spec doesn't allow for CFA values.

llvm/lib/DebugInfo/DWARF/DWARFDebugFrame.cpp

47

That is fine.

[RamNalamothu]

ping; is there anyone be able to take a look at the complete patch?

llvm/include/llvm/DebugInfo/DWARF/DWARFDebugFrame.h

28

Thought about moving it into UnwindLocation but I noticed that UnwindLocation is being used by both CFA and register rules which, theoretically, making it possible to say CFA's location is CFAPlusOffset and also looks like there aren't enough checks to catch this scenario. Being not sure, I have added it to UnwindRow which has CFAValue.

UnwindLocation does get used for both CFA and register values, and yes a UnwindLocation does have more values for registers, but since we since CFA values are parsed using special opcodes, there should be no need to worry as those are what creates the right locations.

That being said, I think moving the address space into UnwindLocation seems like the right thing to do here. Nothing that parses the regular register locations should be adding an address space to the location, though I could see this being needed for some registers locations in the future.

Will move address space into UnwindLocation and fit that in the existing way of how locations are handled.

Does it make sense to split UnwindLocation suitably, so that CFA and register rules could use only what is possible as per DWARF spec?

I believe the LLDB unwind code does this but I feel this splits up the location code a bit since the CFA usually is register + offset or DWARF expression which are both valid register rules. You could make a CFALocation that inherits from UnwindLocation and adds the address space if that makes things more clear? Then you can also add some asserts when someone tries to set the location to something that the DWARF spec doesn't allow for CFA values.

I think that will make things more clear. However, since that split up is independent of this patch and the address space changes here are not significant, I think the split up can be done in a different patch, may be a bit later?

[clayborg]

llvm/include/llvm/DebugInfo/DWARF/DWARFDebugFrame.h

28

Just moving the address space to UnwindLocation should be fine. Might be best to use a "Optional<uint32_t> AddressSpace;" as the new ivar in UnwindLocation.

[RamNalamothu]

Moved the AddrSpace into UnwindLocation.

[clayborg]

llvm/include/llvm/DebugInfo/DWARF/DWARFDebugFrame.h

27–28

27–28

27–28

You could add address space args to this as well, not required though

28–30

Revert and use above constructor

28–31

I don't think we need the "RegPlusOffsetInAddrSpace" enumeration here, we can just use "RegPlusOffset" as the AddrSpace could be used for RegPlusOffset or CFAPlusOffset.

28–31

[RamNalamothu]

Removed RegPlusOffsetInAddrSpace.

[RamNalamothu]

llvm/include/llvm/DebugInfo/DWARF/DWARFDebugFrame.h

27–28

The current intent is that address space qualifies only the CFA and since CFAPlusOffset refers to just an offset from CFA, probably address space is not needed here.

Skipping it for now.

28–30

Ah, I didn't know earlier that Optional<uint32_t> AS = None is possible.

Done.

28–31

I would prefer to retain the assert to catch any possible unintended use of address space.

[clayborg]

I am ok with all of the DWARF unwind changes. Someone else should give the ok for the MC stuff.

[RamNalamothu]

Rename few lit test filenames to match with the downstream.

[RamNalamothu]

In D76877#2782745, @clayborg wrote:

I am ok with all of the DWARF unwind changes. Someone else should give the ok for the MC stuff.

ping: this revision (and the patch series) is under review since March-2020 and would appreciate if someone could review the overall changes and approve.

[RamNalamothu]

In D76877#2785517, @RamNalamothu wrote:

In D76877#2782745, @clayborg wrote:

I am ok with all of the DWARF unwind changes. Someone else should give the ok for the MC stuff.

ping: this revision (and the patch series) is under review since March-2020 and would appreciate if someone could review the overall changes and approve.

Ping

[clayborg]

If no one else comments in the next 2 days, I will accept this patch. So speak up soon if you have an opinion.

[RamNalamothu]

Thank you @clayborg.

Looks like there aren't going to be any further comments.

[RamNalamothu]

ping

[clayborg]

Accepting since no one else had comments. Sorry for the delay.

[RamNalamothu]

No problem.

Thank you very much @clayborg

llvm/include/llvm/BinaryFormat/Dwarf.def

 HANDLE_DW_CFA(0x14, val_offset)
 HANDLE_DW_CFA(0x15, val_offset_sf)
 HANDLE_DW_CFA(0x16, val_expression)
 // Vendor extensions:
 HANDLE_DW_CFA_PRED(0x1d, MIPS_advance_loc8, SELECT_MIPS64)
 HANDLE_DW_CFA_PRED(0x2d, GNU_window_save, SELECT_SPARC)
 HANDLE_DW_CFA_PRED(0x2d, AARCH64_negate_ra_state, SELECT_AARCH64)
 HANDLE_DW_CFA_PRED(0x2e, GNU_args_size, SELECT_X86)
+// Heterogeneous Debugging Extension defined at
+// https://llvm.org/docs/AMDGPUDwarfExtensionsForHeterogeneousDebugging.html#cfa-definition-instructions
+HANDLE_DW_CFA(0x30, LLVM_def_aspace_cfa)
+HANDLE_DW_CFA(0x31, LLVM_def_aspace_cfa_sf)
 
 // Apple Objective-C Property Attributes.
 // Keep this list in sync with clang's DeclObjCCommon.h
 // ObjCPropertyAttribute::Kind!
 HANDLE_DW_APPLE_PROPERTY(0x01, readonly)
 HANDLE_DW_APPLE_PROPERTY(0x02, getter)
 HANDLE_DW_APPLE_PROPERTY(0x04, assign)
 HANDLE_DW_APPLE_PROPERTY(0x08, readwrite)

llvm/include/llvm/DebugInfo/DWARF/DWARFDebugFrame.h

 #include <memory>
 #include <vector>
 
 namespace llvm {
 
 class raw_ostream;
 
 namespace dwarf {
 
 /// Represent a sequence of Call Frame Information instructions that, when read
 /// in order, construct a table mapping PC to frame state. This can also be
 /// referred to as "CFI rules" in DWARF literature to avoid confusion with
 /// computer programs in the broader sense, and in this context each instruction
 /// would be a rule to establish the mapping. Refer to pg. 172 in the DWARF5
 /// manual, "6.4.1 Structure of Call Frame Information".
 class CFIProgram {
 public:
-  typedef SmallVector<uint64_t, 2> Operands;
+  typedef SmallVector<uint64_t, 3> Operands;
 
   /// An instruction consists of a DWARF CFI opcode and an optional sequence of
   /// operands. If it refers to an expression, then this expression has its own
   /// sequence of operations and operands handled separately by DWARFExpression.
   struct Instruction {
     Instruction(uint8_t Opcode) : Opcode(Opcode) {}
 
     uint8_t Opcode;
private:
 
   /// Add a new instruction that has two operands.
   void addInstruction(uint8_t Opcode, uint64_t Operand1, uint64_t Operand2) {
     Instructions.push_back(Instruction(Opcode));
     Instructions.back().Ops.push_back(Operand1);
     Instructions.back().Ops.push_back(Operand2);
   }
 
+  /// Add a new instruction that has three operands.
+  void addInstruction(uint8_t Opcode, uint64_t Operand1, uint64_t Operand2,
+                      uint64_t Operand3) {
+    Instructions.push_back(Instruction(Opcode));
+    Instructions.back().Ops.push_back(Operand1);
+    Instructions.back().Ops.push_back(Operand2);
+    Instructions.back().Ops.push_back(Operand3);
+  }
+
   /// Types of operands to CFI instructions
   /// In DWARF, this type is implicitly tied to a CFI instruction opcode and
   /// thus this type doesn't need to be explictly written to the file (this is
   /// not a DWARF encoding). The relationship of instrs to operand types can
   /// be obtained from getOperandTypes() and is only used to simplify
   /// instruction printing.
   enum OperandType {
     OT_Unset,
     OT_None,
     OT_Address,
     OT_Offset,
     OT_FactoredCodeOffset,
     OT_SignedFactDataOffset,
     OT_UnsignedFactDataOffset,
     OT_Register,
+    OT_AddressSpace,
     OT_Expression
   };
 
   /// Retrieve the array describing the types of operands according to the enum
   /// above. This is indexed by opcode.
-  static ArrayRef<OperandType[2]> getOperandTypes();
+  static ArrayRef<OperandType[3]> getOperandTypes();
 
   /// Print \p Opcode's operand number \p OperandIdx which has value \p Operand.
   void printOperand(raw_ostream &OS, DIDumpOptions DumpOpts,
                     const MCRegisterInfo *MRI, bool IsEH,
                     const Instruction &Instr, unsigned OperandIdx,
                     uint64_t Operand) const;
 };
 

llvm/include/llvm/MC/MCDwarf.h

 
 class MCCFIInstruction {
 public:
   enum OpType {
     OpSameValue,
     OpRememberState,
     OpRestoreState,
     OpOffset,
+    OpLLVMDefAspaceCfa,
     OpDefCfaRegister,
     OpDefCfaOffset,
     OpDefCfa,
     OpRelOffset,
     OpAdjustCfaOffset,
     OpEscape,
     OpRestore,
     OpUndefined,
     OpRegister,
     OpWindowSave,
     OpNegateRAState,
     OpGnuArgsSize
   };
 
 private:
   OpType Operation;
   MCSymbol *Label;
   unsigned Register;
   union {
     int Offset;
     unsigned Register2;
   };
+  unsigned AddressSpace;
   std::vector<char> Values;
   std::string Comment;
 
   MCCFIInstruction(OpType Op, MCSymbol *L, unsigned R, int O, StringRef V,
                    StringRef Comment = "")
       : Operation(Op), Label(L), Register(R), Offset(O),
         Values(V.begin(), V.end()), Comment(Comment) {
-    assert(Op != OpRegister);
+    assert(Op != OpRegister && Op != OpLLVMDefAspaceCfa);
   }
 
   MCCFIInstruction(OpType Op, MCSymbol *L, unsigned R1, unsigned R2)
       : Operation(Op), Label(L), Register(R1), Register2(R2) {
     assert(Op == OpRegister);
   }
 
+  MCCFIInstruction(OpType Op, MCSymbol *L, unsigned R, int O, unsigned AS)
+      : Operation(Op), Label(L), Register(R), Offset(O), AddressSpace(AS) {
+    assert(Op == OpLLVMDefAspaceCfa);
+  }
+
 public:
   /// .cfi_def_cfa defines a rule for computing CFA as: take address from
   /// Register and add Offset to it.
   static MCCFIInstruction cfiDefCfa(MCSymbol *L, unsigned Register,
                                     int Offset) {
     return MCCFIInstruction(OpDefCfa, L, Register, Offset, "");
   }
 
public:
 
   /// .cfi_adjust_cfa_offset Same as .cfi_def_cfa_offset, but
   /// Offset is a relative value that is added/subtracted from the previous
   /// offset.
   static MCCFIInstruction createAdjustCfaOffset(MCSymbol *L, int Adjustment) {
     return MCCFIInstruction(OpAdjustCfaOffset, L, 0, Adjustment, "");
   }
 
+  // FIXME: Update the remaining docs to use the new proposal wording.
+  /// .cfi_llvm_def_aspace_cfa defines the rule for computing the CFA to
+  /// be the result of evaluating the DWARF operation expression
+  /// `DW_OP_constu AS; DW_OP_aspace_bregx R, B` as a location description.
+  static MCCFIInstruction createLLVMDefAspaceCfa(MCSymbol *L, unsigned Register,
+                                                 int Offset,
+                                                 unsigned AddressSpace) {
+    return MCCFIInstruction(OpLLVMDefAspaceCfa, L, Register, Offset,
+                            AddressSpace);
+  }
+
   /// .cfi_offset Previous value of Register is saved at offset Offset
   /// from CFA.
   static MCCFIInstruction createOffset(MCSymbol *L, unsigned Register,
                                        int Offset) {
     return MCCFIInstruction(OpOffset, L, Register, Offset, "");
   }
 
   /// .cfi_rel_offset Previous value of Register is saved at offset
public:
 
   OpType getOperation() const { return Operation; }
   MCSymbol *getLabel() const { return Label; }
 
   unsigned getRegister() const {
     assert(Operation == OpDefCfa || Operation == OpOffset ||
            Operation == OpRestore || Operation == OpUndefined ||
            Operation == OpSameValue || Operation == OpDefCfaRegister ||
-           Operation == OpRelOffset || Operation == OpRegister);
+           Operation == OpRelOffset || Operation == OpRegister ||
+           Operation == OpLLVMDefAspaceCfa);
     return Register;
   }
 
   unsigned getRegister2() const {
     assert(Operation == OpRegister);
     return Register2;
   }
 
+  unsigned getAddressSpace() const {
+    assert(Operation == OpLLVMDefAspaceCfa);
+    return AddressSpace;
+  }
+
   int getOffset() const {
     assert(Operation == OpDefCfa || Operation == OpOffset ||
            Operation == OpRelOffset || Operation == OpDefCfaOffset ||
-           Operation == OpAdjustCfaOffset || Operation == OpGnuArgsSize);
+           Operation == OpAdjustCfaOffset || Operation == OpGnuArgsSize ||
+           Operation == OpLLVMDefAspaceCfa);
     return Offset;
   }
 
   StringRef getValues() const {
     assert(Operation == OpEscape);
     return StringRef(&Values[0], Values.size());
   }
 

llvm/include/llvm/MC/MCStreamer.h

public:
 
   virtual MCSymbol *getDwarfLineTableSymbol(unsigned CUID);
   virtual void emitCFISections(bool EH, bool Debug);
   void emitCFIStartProc(bool IsSimple, SMLoc Loc = SMLoc());
   void emitCFIEndProc();
   virtual void emitCFIDefCfa(int64_t Register, int64_t Offset);
   virtual void emitCFIDefCfaOffset(int64_t Offset);
   virtual void emitCFIDefCfaRegister(int64_t Register);
+  virtual void emitCFILLVMDefAspaceCfa(int64_t Register, int64_t Offset,
+                                       int64_t AddressSpace);
   virtual void emitCFIOffset(int64_t Register, int64_t Offset);
   virtual void emitCFIPersonality(const MCSymbol *Sym, unsigned Encoding);
   virtual void emitCFILsda(const MCSymbol *Sym, unsigned Encoding);
   virtual void emitCFIRememberState();
   virtual void emitCFIRestoreState();
   virtual void emitCFISameValue(int64_t Register);
   virtual void emitCFIRestore(int64_t Register);
   virtual void emitCFIRelOffset(int64_t Register, int64_t Offset);

llvm/lib/CodeGen/AsmPrinter/AsmPrinterDwarf.cpp

case MCCFIInstruction::OpAdjustCfaOffset:
     OutStreamer->emitCFIAdjustCfaOffset(Inst.getOffset());
     break;
   case MCCFIInstruction::OpDefCfa:
     OutStreamer->emitCFIDefCfa(Inst.getRegister(), Inst.getOffset());
     break;
   case MCCFIInstruction::OpDefCfaRegister:
     OutStreamer->emitCFIDefCfaRegister(Inst.getRegister());
     break;
+  case MCCFIInstruction::OpLLVMDefAspaceCfa:
+    OutStreamer->emitCFILLVMDefAspaceCfa(Inst.getRegister(), Inst.getOffset(),
+                                         Inst.getAddressSpace());
+    break;
   case MCCFIInstruction::OpOffset:
     OutStreamer->emitCFIOffset(Inst.getRegister(), Inst.getOffset());
     break;
   case MCCFIInstruction::OpRegister:
     OutStreamer->emitCFIRegister(Inst.getRegister(), Inst.getRegister2());
     break;
   case MCCFIInstruction::OpWindowSave:
     OutStreamer->emitCFIWindowSave();

llvm/lib/CodeGen/CFIInstrInserter.cpp

if (MI.isCFIInstruction()) {
         CSRReg = CFI.getRegister2();
         break;
       case MCCFIInstruction::OpRelOffset:
         CSROffset = CFI.getOffset() - SetOffset;
         break;
       case MCCFIInstruction::OpRestore:
         CSRRestored.set(CFI.getRegister());
         break;
+      case MCCFIInstruction::OpLLVMDefAspaceCfa:
+        // TODO: Add support for handling cfi_def_aspace_cfa.
+#ifndef NDEBUG
+        report_fatal_error(
+            "Support for cfi_llvm_def_aspace_cfa not implemented! Value of CFA "
+            "may be incorrect!\n");
+#endif
+        break;
       case MCCFIInstruction::OpRememberState:
         // TODO: Add support for handling cfi_remember_state.
 #ifndef NDEBUG
         report_fatal_error(
             "Support for cfi_remember_state not implemented! Value of CFA "
             "may be incorrect!\n");
 #endif
         break;

llvm/lib/CodeGen/MIRParser/MILexer.h

enum TokenKind {
     kw_cfi_same_value,
     kw_cfi_offset,
     kw_cfi_rel_offset,
     kw_cfi_def_cfa_register,
     kw_cfi_def_cfa_offset,
     kw_cfi_adjust_cfa_offset,
     kw_cfi_escape,
     kw_cfi_def_cfa,
+    kw_cfi_llvm_def_aspace_cfa,
     kw_cfi_register,
     kw_cfi_remember_state,
     kw_cfi_restore,
     kw_cfi_restore_state,
     kw_cfi_undefined,
     kw_cfi_window_save,
     kw_cfi_aarch64_negate_ra_sign_state,
     kw_blockaddress,

llvm/lib/CodeGen/MIRParser/MILexer.cpp

return StringSwitch<MIToken::TokenKind>(Identifier)
       .Case("frame-destroy", MIToken::kw_frame_destroy)
       .Case("nnan", MIToken::kw_nnan)
       .Case("ninf", MIToken::kw_ninf)
       .Case("nsz", MIToken::kw_nsz)
       .Case("arcp", MIToken::kw_arcp)
       .Case("contract", MIToken::kw_contract)
       .Case("afn", MIToken::kw_afn)
       .Case("reassoc", MIToken::kw_reassoc)
       .Case("nuw", MIToken::kw_nuw)
       .Case("nsw", MIToken::kw_nsw)
       .Case("exact", MIToken::kw_exact)
       .Case("nofpexcept", MIToken::kw_nofpexcept)
       .Case("debug-location", MIToken::kw_debug_location)
       .Case("debug-instr-number", MIToken::kw_debug_instr_number)
       .Case("same_value", MIToken::kw_cfi_same_value)
       .Case("offset", MIToken::kw_cfi_offset)
       .Case("rel_offset", MIToken::kw_cfi_rel_offset)
       .Case("def_cfa_register", MIToken::kw_cfi_def_cfa_register)
       .Case("def_cfa_offset", MIToken::kw_cfi_def_cfa_offset)
       .Case("adjust_cfa_offset", MIToken::kw_cfi_adjust_cfa_offset)
       .Case("escape", MIToken::kw_cfi_escape)
       .Case("def_cfa", MIToken::kw_cfi_def_cfa)
+      .Case("llvm_def_aspace_cfa", MIToken::kw_cfi_llvm_def_aspace_cfa)
       .Case("remember_state", MIToken::kw_cfi_remember_state)
       .Case("restore", MIToken::kw_cfi_restore)
       .Case("restore_state", MIToken::kw_cfi_restore_state)
       .Case("undefined", MIToken::kw_cfi_undefined)
       .Case("register", MIToken::kw_cfi_register)
       .Case("window_save", MIToken::kw_cfi_window_save)
       .Case("negate_ra_sign_state",
             MIToken::kw_cfi_aarch64_negate_ra_sign_state)
       .Case("blockaddress", MIToken::kw_blockaddress)
       .Case("intrinsic", MIToken::kw_intrinsic)
       .Case("target-index", MIToken::kw_target_index)
       .Case("half", MIToken::kw_half)
       .Case("float", MIToken::kw_float)
       .Case("double", MIToken::kw_double)
       .Case("x86_fp80", MIToken::kw_x86_fp80)
       .Case("fp128", MIToken::kw_fp128)
       .Case("ppc_fp128", MIToken::kw_ppc_fp128)

llvm/lib/CodeGen/MIRParser/MIParser.cpp

public:
   bool parseExternalSymbolOperand(MachineOperand &Dest);
   bool parseMCSymbolOperand(MachineOperand &Dest);
   bool parseMDNode(MDNode *&Node);
   bool parseDIExpression(MDNode *&Expr);
   bool parseDILocation(MDNode *&Expr);
   bool parseMetadataOperand(MachineOperand &Dest);
   bool parseCFIOffset(int &Offset);
   bool parseCFIRegister(Register &Reg);
+  bool parseCFIAddressSpace(unsigned &AddressSpace);
   bool parseCFIEscapeValues(std::string& Values);
   bool parseCFIOperand(MachineOperand &Dest);
   bool parseIRBlock(BasicBlock *&BB, const Function &F);
   bool parseBlockAddressOperand(MachineOperand &Dest);
   bool parseIntrinsicOperand(MachineOperand &Dest);
   bool parsePredicateOperand(MachineOperand &Dest);
   bool parseShuffleMaskOperand(MachineOperand &Dest);
   bool parseTargetIndexOperand(MachineOperand &Dest);
bool MIParser::parseCFIRegister(Register &Reg) {
   int DwarfReg = TRI->getDwarfRegNum(LLVMReg, true);
   if (DwarfReg < 0)
     return error("invalid DWARF register");
   Reg = (unsigned)DwarfReg;
   lex();
   return false;
 }
 
+bool MIParser::parseCFIAddressSpace(unsigned &AddressSpace) {
+  if (Token.isNot(MIToken::IntegerLiteral))
+    return error("expected a cfi address space");
+  if (Token.integerValue().isSigned())
+    return error("expected an unsigned integer (cfi address space)");
+  AddressSpace = Token.integerValue().getZExtValue();
+  lex();
+  return false;
+}
+
 bool MIParser::parseCFIEscapeValues(std::string &Values) {
   do {
     if (Token.isNot(MIToken::HexLiteral))
       return error("expected a hexadecimal literal");
     unsigned Value;
     if (getUnsigned(Value))
       return true;
     if (Value > UINT8_MAX)
       return error("expected a 8-bit integer (too large)");
     Values.push_back(static_cast<uint8_t>(Value));
     lex();
   } while (consumeIfPresent(MIToken::comma));
   return false;
 }
 
 bool MIParser::parseCFIOperand(MachineOperand &Dest) {
   auto Kind = Token.kind();
   lex();
   int Offset;
   Register Reg;
+  unsigned AddressSpace;
   unsigned CFIIndex;
   switch (Kind) {
   case MIToken::kw_cfi_same_value:
     if (parseCFIRegister(Reg))
       return true;
     CFIIndex = MF.addFrameInst(MCCFIInstruction::createSameValue(nullptr, Reg));
     break;
   case MIToken::kw_cfi_offset:
case MIToken::kw_cfi_adjust_cfa_offset:
     break;
   case MIToken::kw_cfi_def_cfa:
     if (parseCFIRegister(Reg) || expectAndConsume(MIToken::comma) ||
         parseCFIOffset(Offset))
       return true;
     CFIIndex =
         MF.addFrameInst(MCCFIInstruction::cfiDefCfa(nullptr, Reg, Offset));
     break;
+  case MIToken::kw_cfi_llvm_def_aspace_cfa:
+    if (parseCFIRegister(Reg) || expectAndConsume(MIToken::comma) ||
+        parseCFIOffset(Offset) || expectAndConsume(MIToken::comma) ||
+        parseCFIAddressSpace(AddressSpace))
+      return true;
+    CFIIndex = MF.addFrameInst(MCCFIInstruction::createLLVMDefAspaceCfa(
+        nullptr, Reg, Offset, AddressSpace));
+    break;
   case MIToken::kw_cfi_remember_state:
     CFIIndex = MF.addFrameInst(MCCFIInstruction::createRememberState(nullptr));
     break;
   case MIToken::kw_cfi_restore:
     if (parseCFIRegister(Reg))
       return true;
     CFIIndex = MF.addFrameInst(MCCFIInstruction::createRestore(nullptr, Reg));
     break;
bool MIParser::parseMachineOperand(const unsigned OpCode, const unsigned OpIdx,
   case MIToken::kw_cfi_same_value:
   case MIToken::kw_cfi_offset:
   case MIToken::kw_cfi_rel_offset:
   case MIToken::kw_cfi_def_cfa_register:
   case MIToken::kw_cfi_def_cfa_offset:
   case MIToken::kw_cfi_adjust_cfa_offset:
   case MIToken::kw_cfi_escape:
   case MIToken::kw_cfi_def_cfa:
+  case MIToken::kw_cfi_llvm_def_aspace_cfa:
   case MIToken::kw_cfi_register:
   case MIToken::kw_cfi_remember_state:
   case MIToken::kw_cfi_restore:
   case MIToken::kw_cfi_restore_state:
   case MIToken::kw_cfi_undefined:
   case MIToken::kw_cfi_window_save:
   case MIToken::kw_cfi_aarch64_negate_ra_sign_state:
     return parseCFIOperand(Dest);

llvm/lib/CodeGen/MachineOperand.cpp

case MCCFIInstruction::OpDefCfaOffset:
     break;
   case MCCFIInstruction::OpDefCfa:
     OS << "def_cfa ";
     if (MCSymbol *Label = CFI.getLabel())
       MachineOperand::printSymbol(OS, *Label);
     printCFIRegister(CFI.getRegister(), OS, TRI);
     OS << ", " << CFI.getOffset();
     break;
+  case MCCFIInstruction::OpLLVMDefAspaceCfa:
+    OS << "llvm_def_aspace_cfa ";
+    if (MCSymbol *Label = CFI.getLabel())
+      MachineOperand::printSymbol(OS, *Label);
+    printCFIRegister(CFI.getRegister(), OS, TRI);
+    OS << ", " << CFI.getOffset();
+    OS << ", " << CFI.getAddressSpace();
+    break;
   case MCCFIInstruction::OpRelOffset:
     OS << "rel_offset ";
     if (MCSymbol *Label = CFI.getLabel())
       MachineOperand::printSymbol(OS, *Label);
     printCFIRegister(CFI.getRegister(), OS, TRI);
     OS << ", " << CFI.getOffset();
     break;
   case MCCFIInstruction::OpAdjustCfaOffset:

llvm/lib/DebugInfo/DWARF/DWARFDebugFrame.cpp

if (Optional<unsigned> LLVMRegNum = MRI->getLLVMRegNum(RegNum, IsEH)) {
         return;
       }
     }
   }
   OS << "reg" << RegNum;
 }
 
 // See DWARF standard v3, section 7.23
 const uint8_t DWARF_CFI_PRIMARY_OPCODE_MASK = 0xc0;
 const uint8_t DWARF_CFI_PRIMARY_OPERAND_MASK = 0x3f;
 
 Error CFIProgram::parse(DWARFDataExtractor Data, uint64_t *Offset,
                         uint64_t EndOffset) {
   DataExtractor::Cursor C(*Offset);
   while (C && C.tell() < EndOffset) {
     uint8_t Opcode = Data.getRelocatedValue(C, 1);
     if (!C)
while (C && C.tell() < EndOffset) {
     case DW_CFA_GNU_args_size:
       // Operands: ULEB128
       addInstruction(Opcode, Data.getULEB128(C));
       break;
     case DW_CFA_def_cfa_offset_sf:
       // Operands: SLEB128
       addInstruction(Opcode, Data.getSLEB128(C));
       break;
+    case DW_CFA_LLVM_def_aspace_cfa:
+    case DW_CFA_LLVM_def_aspace_cfa_sf: {
+      auto RegNum = Data.getULEB128(C);
+      auto CfaOffset = Opcode == DW_CFA_LLVM_def_aspace_cfa
+                           ? Data.getULEB128(C)
+                           : Data.getSLEB128(C);
+      auto AddressSpace = Data.getULEB128(C);
+      addInstruction(Opcode, RegNum, CfaOffset, AddressSpace);
+      break;
+    }
     case DW_CFA_offset_extended:
     case DW_CFA_register:
     case DW_CFA_def_cfa:
     case DW_CFA_val_offset: {
       // Operands: ULEB128, ULEB128
       // Note: We can not embed getULEB128 directly into function
       // argument list. getULEB128 changes Offset and order of evaluation
       // for arguments is unspecified.
Error CFIProgram::parse(DWARFDataExtractor Data, uint64_t *Offset,
   }
 
   *Offset = C.tell();
   return C.takeError();
 }
 
 namespace {
 
 
 } // end anonymous namespace
 
-ArrayRef<CFIProgram::OperandType[2]> CFIProgram::getOperandTypes() {
-  static OperandType OpTypes[DW_CFA_restore+1][2];
+ArrayRef<CFIProgram::OperandType[3]> CFIProgram::getOperandTypes() {
+  static OperandType OpTypes[DW_CFA_restore + 1][3];
   static bool Initialized = false;
   if (Initialized) {
-    return ArrayRef<OperandType[2]>(&OpTypes[0], DW_CFA_restore+1);
+    return ArrayRef<OperandType[3]>(&OpTypes[0], DW_CFA_restore + 1);
   }
   Initialized = true;
 
-#define DECLARE_OP2(OP, OPTYPE0, OPTYPE1)       \
-  do {                                          \
-    OpTypes[OP][0] = OPTYPE0;                   \
-    OpTypes[OP][1] = OPTYPE1;                   \
+#define DECLARE_OP3(OP, OPTYPE0, OPTYPE1, OPTYPE2)                             \
+  do {                                                                         \
+    OpTypes[OP][0] = OPTYPE0;                                                  \
+    OpTypes[OP][1] = OPTYPE1;                                                  \
+    OpTypes[OP][2] = OPTYPE2;                                                  \
   } while (false)
+#define DECLARE_OP2(OP, OPTYPE0, OPTYPE1)                                      \
+  DECLARE_OP3(OP, OPTYPE0, OPTYPE1, OT_None)
 #define DECLARE_OP1(OP, OPTYPE0) DECLARE_OP2(OP, OPTYPE0, OT_None)
 #define DECLARE_OP0(OP) DECLARE_OP1(OP, OT_None)
 
   DECLARE_OP1(DW_CFA_set_loc, OT_Address);
   DECLARE_OP1(DW_CFA_advance_loc, OT_FactoredCodeOffset);
   DECLARE_OP1(DW_CFA_advance_loc1, OT_FactoredCodeOffset);
   DECLARE_OP1(DW_CFA_advance_loc2, OT_FactoredCodeOffset);
   DECLARE_OP1(DW_CFA_advance_loc4, OT_FactoredCodeOffset);
   DECLARE_OP1(DW_CFA_MIPS_advance_loc8, OT_FactoredCodeOffset);
   DECLARE_OP2(DW_CFA_def_cfa, OT_Register, OT_Offset);
   DECLARE_OP2(DW_CFA_def_cfa_sf, OT_Register, OT_SignedFactDataOffset);
   DECLARE_OP1(DW_CFA_def_cfa_register, OT_Register);
+  DECLARE_OP3(DW_CFA_LLVM_def_aspace_cfa, OT_Register, OT_Offset,
+              OT_AddressSpace);
+  DECLARE_OP3(DW_CFA_LLVM_def_aspace_cfa_sf, OT_Register,
+              OT_SignedFactDataOffset, OT_AddressSpace);
   DECLARE_OP1(DW_CFA_def_cfa_offset, OT_Offset);
   DECLARE_OP1(DW_CFA_def_cfa_offset_sf, OT_SignedFactDataOffset);
   DECLARE_OP1(DW_CFA_def_cfa_expression, OT_Expression);
   DECLARE_OP1(DW_CFA_undefined, OT_Register);
   DECLARE_OP1(DW_CFA_same_value, OT_Register);
   DECLARE_OP2(DW_CFA_offset, OT_Register, OT_UnsignedFactDataOffset);
   DECLARE_OP2(DW_CFA_offset_extended, OT_Register, OT_UnsignedFactDataOffset);
   DECLARE_OP2(DW_CFA_offset_extended_sf, OT_Register, OT_SignedFactDataOffset);
#define DECLARE_OP0(OP) DECLARE_OP1(OP, OT_None)
   DECLARE_OP0(DW_CFA_GNU_window_save);
   DECLARE_OP1(DW_CFA_GNU_args_size, OT_Offset);
   DECLARE_OP0(DW_CFA_nop);
 
 #undef DECLARE_OP0
 #undef DECLARE_OP1
 #undef DECLARE_OP2
 
-  return ArrayRef<OperandType[2]>(&OpTypes[0], DW_CFA_restore+1);
+  return ArrayRef<OperandType[3]>(&OpTypes[0], DW_CFA_restore + 1);
 }
 
 /// Print \p Opcode's operand number \p OperandIdx which has value \p Operand.
 void CFIProgram::printOperand(raw_ostream &OS, DIDumpOptions DumpOpts,
                               const MCRegisterInfo *MRI, bool IsEH,
                               const Instruction &Instr, unsigned OperandIdx,
                               uint64_t Operand) const {
-  assert(OperandIdx < 2);
+  assert(OperandIdx < 3);
   uint8_t Opcode = Instr.Opcode;
   OperandType Type = getOperandTypes()[Opcode][OperandIdx];
 
   switch (Type) {
   case OT_Unset: {
     OS << " Unsupported " << (OperandIdx ? "second" : "first") << " operand to";
     auto OpcodeName = CallFrameString(Opcode, Arch);
     if (!OpcodeName.empty())
if (DataAlignmentFactor)
       OS << format(" %" PRId64, Operand * DataAlignmentFactor);
     else
       OS << format(" %" PRId64 "*data_alignment_factor" , Operand);
     break;
   case OT_Register:
     OS << ' ';
     printRegister(OS, MRI, IsEH, Operand);
     break;
+  case OT_AddressSpace:
+    OS << format(" as%" PRId64, Operand);
+    break;
   case OT_Expression:
     assert(Instr.Expression && "missing DWARFExpression object");
     OS << " ";
     Instr.Expression->print(OS, DumpOpts, MRI, nullptr, IsEH);
     break;
   }
 }
 

llvm/lib/MC/MCAsmStreamer.cpp

public:
   void EmitCVFPOData(const MCSymbol *ProcSym, SMLoc L) override;
 
   void emitIdent(StringRef IdentString) override;
   void emitCFIBKeyFrame() override;
   void emitCFISections(bool EH, bool Debug) override;
   void emitCFIDefCfa(int64_t Register, int64_t Offset) override;
   void emitCFIDefCfaOffset(int64_t Offset) override;
   void emitCFIDefCfaRegister(int64_t Register) override;
+  void emitCFILLVMDefAspaceCfa(int64_t Register, int64_t Offset,
+                               int64_t AddressSpace) override;
   void emitCFIOffset(int64_t Register, int64_t Offset) override;
   void emitCFIPersonality(const MCSymbol *Sym, unsigned Encoding) override;
   void emitCFILsda(const MCSymbol *Sym, unsigned Encoding) override;
   void emitCFIRememberState() override;
   void emitCFIRestoreState() override;
   void emitCFIRestore(int64_t Register) override;
   void emitCFISameValue(int64_t Register) override;
   void emitCFIRelOffset(int64_t Register, int64_t Offset) override;

 }
 
 void MCAsmStreamer::emitCFIDefCfaOffset(int64_t Offset) {
   MCStreamer::emitCFIDefCfaOffset(Offset);
   OS << "\t.cfi_def_cfa_offset " << Offset;
   EmitEOL();
 }
 
+void MCAsmStreamer::emitCFILLVMDefAspaceCfa(int64_t Register, int64_t Offset,
+                                            int64_t AddressSpace) {
+  MCStreamer::emitCFILLVMDefAspaceCfa(Register, Offset, AddressSpace);
+  OS << "\t.cfi_llvm_def_aspace_cfa ";
+  EmitRegisterName(Register);
+  OS << ", " << Offset;
+  OS << ", " << AddressSpace;
+  // FIXME: When going from ASM->ASM a lot of these directives seem to add too
+  // much whitespace; maybe something isn't consuming whitespace eagerly enough
+  // while parsing?
+  EmitEOL();
+}
+
 static void PrintCFIEscape(llvm::formatted_raw_ostream &OS, StringRef Values) {
   OS << "\t.cfi_escape ";
   if (!Values.empty()) {
     size_t e = Values.size() - 1;
     for (size_t i = 0; i < e; ++i)
       OS << format("0x%02x", uint8_t(Values[i])) << ", ";
     OS << format("0x%02x", uint8_t(Values[e]));
   }

llvm/lib/MC/MCDwarf.cpp

case MCCFIInstruction::OpDefCfaRegister: {
     unsigned Reg = Instr.getRegister();
     if (!IsEH)
       Reg = MRI->getDwarfRegNumFromDwarfEHRegNum(Reg);
     Streamer.emitInt8(dwarf::DW_CFA_def_cfa_register);
     Streamer.emitULEB128IntValue(Reg);
 
     return;
   }
+  // TODO: Implement `_sf` variants if/when they need to be emitted.
+  case MCCFIInstruction::OpLLVMDefAspaceCfa: {
+    Streamer.emitIntValue(dwarf::DW_CFA_LLVM_def_aspace_cfa, 1);
+    Streamer.emitULEB128IntValue(Instr.getRegister());
+    CFAOffset = Instr.getOffset();
+    Streamer.emitULEB128IntValue(CFAOffset);
+    Streamer.emitULEB128IntValue(Instr.getAddressSpace());
+
+    return;
+  }
   case MCCFIInstruction::OpOffset:
   case MCCFIInstruction::OpRelOffset: {
     const bool IsRelative =
       Instr.getOperation() == MCCFIInstruction::OpRelOffset;
 
     unsigned Reg = Instr.getRegister();
     if (!IsEH)
       Reg = MRI->getDwarfRegNumFromDwarfEHRegNum(Reg);

llvm/lib/MC/MCParser/AsmParser.cpp

enum DirectiveKind {
     DK_CV_FPO_DATA,
     DK_CFI_SECTIONS,
     DK_CFI_STARTPROC,
     DK_CFI_ENDPROC,
     DK_CFI_DEF_CFA,
     DK_CFI_DEF_CFA_OFFSET,
     DK_CFI_ADJUST_CFA_OFFSET,
     DK_CFI_DEF_CFA_REGISTER,
+    DK_CFI_LLVM_DEF_ASPACE_CFA,
     DK_CFI_OFFSET,
     DK_CFI_REL_OFFSET,
     DK_CFI_PERSONALITY,
     DK_CFI_LSDA,
     DK_CFI_REMEMBER_STATE,
     DK_CFI_RESTORE_STATE,
     DK_CFI_SAME_VALUE,
     DK_CFI_RESTORE,
private:
   bool parseDirectiveCFIWindowSave();
   bool parseDirectiveCFISections();
   bool parseDirectiveCFIStartProc();
   bool parseDirectiveCFIEndProc();
   bool parseDirectiveCFIDefCfaOffset();
   bool parseDirectiveCFIDefCfa(SMLoc DirectiveLoc);
   bool parseDirectiveCFIAdjustCfaOffset();
   bool parseDirectiveCFIDefCfaRegister(SMLoc DirectiveLoc);
+  bool parseDirectiveCFILLVMDefAspaceCfa(SMLoc DirectiveLoc);
   bool parseDirectiveCFIOffset(SMLoc DirectiveLoc);
   bool parseDirectiveCFIRelOffset(SMLoc DirectiveLoc);
   bool parseDirectiveCFIPersonalityOrLsda(bool IsPersonality);
   bool parseDirectiveCFIRememberState();
   bool parseDirectiveCFIRestoreState();
   bool parseDirectiveCFISameValue(SMLoc DirectiveLoc);
   bool parseDirectiveCFIRestore(SMLoc DirectiveLoc);
   bool parseDirectiveCFIEscape();
if (IDVal.startswith(".") && IDVal != ".") {
     case DK_CFI_DEF_CFA:
       return parseDirectiveCFIDefCfa(IDLoc);
     case DK_CFI_DEF_CFA_OFFSET:
       return parseDirectiveCFIDefCfaOffset();
     case DK_CFI_ADJUST_CFA_OFFSET:
       return parseDirectiveCFIAdjustCfaOffset();
     case DK_CFI_DEF_CFA_REGISTER:
       return parseDirectiveCFIDefCfaRegister(IDLoc);
+    case DK_CFI_LLVM_DEF_ASPACE_CFA:
+      return parseDirectiveCFILLVMDefAspaceCfa(IDLoc);
     case DK_CFI_OFFSET:
       return parseDirectiveCFIOffset(IDLoc);
     case DK_CFI_REL_OFFSET:
       return parseDirectiveCFIRelOffset(IDLoc);
     case DK_CFI_PERSONALITY:
       return parseDirectiveCFIPersonalityOrLsda(true);
     case DK_CFI_LSDA:
       return parseDirectiveCFIPersonalityOrLsda(false);
bool AsmParser::parseDirectiveCFIDefCfaRegister(SMLoc DirectiveLoc) {
   int64_t Register = 0;
   if (parseRegisterOrRegisterNumber(Register, DirectiveLoc))
     return true;
 
   getStreamer().emitCFIDefCfaRegister(Register);
   return false;
 }
 
+/// parseDirectiveCFILLVMDefAspaceCfa
+/// ::= .cfi_llvm_def_aspace_cfa register, offset, address_space
+bool AsmParser::parseDirectiveCFILLVMDefAspaceCfa(SMLoc DirectiveLoc) {
+  int64_t Register = 0, Offset = 0, AddressSpace = 0;
+  if (parseRegisterOrRegisterNumber(Register, DirectiveLoc) ||
+      parseToken(AsmToken::Comma, "unexpected token in directive") ||
+      parseAbsoluteExpression(Offset) ||
+      parseToken(AsmToken::Comma, "unexpected token in directive") ||
+      parseAbsoluteExpression(AddressSpace))
+    return true;
+
+  getStreamer().emitCFILLVMDefAspaceCfa(Register, Offset, AddressSpace);
+  return false;
+}
+
 /// parseDirectiveCFIOffset
 /// ::= .cfi_offset register, offset
 bool AsmParser::parseDirectiveCFIOffset(SMLoc DirectiveLoc) {
   int64_t Register = 0;
   int64_t Offset = 0;
 
   if (parseRegisterOrRegisterNumber(Register, DirectiveLoc) ||
       parseToken(AsmToken::Comma, "unexpected token in directive") ||
void AsmParser::initializeDirectiveKindMap() {
   DirectiveKindMap[".uleb128"] = DK_ULEB128;
   DirectiveKindMap[".cfi_sections"] = DK_CFI_SECTIONS;
   DirectiveKindMap[".cfi_startproc"] = DK_CFI_STARTPROC;
   DirectiveKindMap[".cfi_endproc"] = DK_CFI_ENDPROC;
   DirectiveKindMap[".cfi_def_cfa"] = DK_CFI_DEF_CFA;
   DirectiveKindMap[".cfi_def_cfa_offset"] = DK_CFI_DEF_CFA_OFFSET;
   DirectiveKindMap[".cfi_adjust_cfa_offset"] = DK_CFI_ADJUST_CFA_OFFSET;
   DirectiveKindMap[".cfi_def_cfa_register"] = DK_CFI_DEF_CFA_REGISTER;
+  DirectiveKindMap[".cfi_llvm_def_aspace_cfa"] = DK_CFI_LLVM_DEF_ASPACE_CFA;
   DirectiveKindMap[".cfi_offset"] = DK_CFI_OFFSET;
   DirectiveKindMap[".cfi_rel_offset"] = DK_CFI_REL_OFFSET;
   DirectiveKindMap[".cfi_personality"] = DK_CFI_PERSONALITY;
   DirectiveKindMap[".cfi_lsda"] = DK_CFI_LSDA;
   DirectiveKindMap[".cfi_remember_state"] = DK_CFI_REMEMBER_STATE;
   DirectiveKindMap[".cfi_restore_state"] = DK_CFI_RESTORE_STATE;
   DirectiveKindMap[".cfi_same_value"] = DK_CFI_SAME_VALUE;
   DirectiveKindMap[".cfi_restore"] = DK_CFI_RESTORE;

llvm/lib/MC/MCStreamer.cpp

MCCFIInstruction Instruction =
     MCCFIInstruction::createDefCfaRegister(Label, Register);
   MCDwarfFrameInfo *CurFrame = getCurrentDwarfFrameInfo();
   if (!CurFrame)
     return;
   CurFrame->Instructions.push_back(Instruction);
   CurFrame->CurrentCfaRegister = static_cast<unsigned>(Register);
 }
 
+void MCStreamer::emitCFILLVMDefAspaceCfa(int64_t Register, int64_t Offset,
+                                         int64_t AddressSpace) {
+  MCSymbol *Label = emitCFILabel();
+  MCCFIInstruction Instruction = MCCFIInstruction::createLLVMDefAspaceCfa(
+      Label, Register, Offset, AddressSpace);
+  MCDwarfFrameInfo *CurFrame = getCurrentDwarfFrameInfo();
+  if (!CurFrame)
+    return;
+  CurFrame->Instructions.push_back(Instruction);
+  CurFrame->CurrentCfaRegister = static_cast<unsigned>(Register);
+}
+
 void MCStreamer::emitCFIOffset(int64_t Register, int64_t Offset) {
   MCSymbol *Label = emitCFILabel();
   MCCFIInstruction Instruction =
     MCCFIInstruction::createOffset(Label, Register, Offset);
   MCDwarfFrameInfo *CurFrame = getCurrentDwarfFrameInfo();
   if (!CurFrame)
     return;
   CurFrame->Instructions.push_back(Instruction);

llvm/test/CodeGen/MIR/AArch64/cfi.mir

 body: |
   bb.0.entry:
     ; CHECK: CFI_INSTRUCTION def_cfa $w29, 16
     frame-setup CFI_INSTRUCTION def_cfa $w29, 16
     ; CHECK: CFI_INSTRUCTION def_cfa_register $w29
     frame-setup CFI_INSTRUCTION def_cfa_register $w29
     ; CHECK: CFI_INSTRUCTION def_cfa_offset -8
     frame-setup CFI_INSTRUCTION def_cfa_offset -8
+    ; CHECK: CFI_INSTRUCTION llvm_def_aspace_cfa $w29, 16, 6
+    frame-setup CFI_INSTRUCTION llvm_def_aspace_cfa $w29, 16, 6
     ; CHECK: CFI_INSTRUCTION offset $w30, -8
     frame-setup CFI_INSTRUCTION offset $w30, -8
     ; CHECK: CFI_INSTRUCTION rel_offset $w30, -8
     frame-setup CFI_INSTRUCTION rel_offset $w30, -8
     ; CHECK: CFI_INSTRUCTION adjust_cfa_offset -8
     frame-setup CFI_INSTRUCTION adjust_cfa_offset -8
     CFI_INSTRUCTION restore $w30
     ; CHECK: CFI_INSTRUCTION restore $w30

llvm/test/MC/ELF/cfi-llvm-def-aspace-cfa-errors.s

+// RUN: not llvm-mc -filetype=obj -triple x86_64-pc-linux-gnu %s -o /dev/null 2>&1 | FileCheck %s
+
+// Check that we diagnose malformed .cfi_llvm_def_aspace_cfa directives.
+
+.cfi_startproc
+
+// CHECK: [[#@LINE+1]]:{{[0-9]+}}: error: invalid register name
+.cfi_llvm_def_aspace_cfa foo
+
+// CHECK: [[#@LINE+1]]:{{[0-9]+}}: error: unexpected token in directive
+.cfi_llvm_def_aspace_cfa %rcx .
+
+// CHECK: [[#@LINE+1]]:{{[0-9]+}}: error: expected absolute expression
+.cfi_llvm_def_aspace_cfa %rcx, .+1
+
+// CHECK: [[#@LINE+1]]:{{[0-9]+}}: error: unexpected token in directive
+.cfi_llvm_def_aspace_cfa %rcx, 1 .
+
+// CHECK: [[#@LINE+1]]:{{[0-9]+}}: error: expected absolute expression
+.cfi_llvm_def_aspace_cfa %rcx, 1, .+1
+
+// CHECK: [[#@LINE+1]]:{{[0-9]+}}: error: unexpected token at start of statement
+.cfi_llvm_def_aspace_cfa %rcx, 1, 1,
+
+.cfi_endproc

llvm/test/MC/ELF/cfi-llvm-def-aspace-cfa.s

+# RUN: llvm-mc -filetype=asm -triple x86_64-pc-linux-gnu %s -o - | FileCheck --check-prefix=ASM %s
+# RUN: llvm-mc -filetype=obj -triple x86_64-pc-linux-gnu %s -o %t
+# RUN: llvm-readelf -S -r -x .eh_frame %t | FileCheck --check-prefix=READELF %s
+
+f:
+  .cfi_startproc
+  nop
+  .cfi_llvm_def_aspace_cfa %rcx, 0, 6
+  nop
+  .cfi_endproc
+
+# ASM: f:
+# ASM-NEXT: .cfi_startproc
+# ASM-NEXT: nop
+# ASM-NEXT: .cfi_llvm_def_aspace_cfa %rcx, 0, 6
+# FIXME Why emit an extra empty line?
+# ASM-EMPTY:
+# ASM-NEXT: nop
+# ASM-NEXT: .cfi_endproc
+
+# READELF: Section Headers:
+# READELF: Name              Type            Address          Off    Size   ES Flg Lk Inf Al
+# READELF: .eh_frame         X86_64_UNWIND   0000000000000000 000048 000030 00   A  0   0  8
+
+# READELF: Relocation section '.rela.eh_frame' at offset 0xc0 contains 1 entries:
+# READELF-NEXT:     Offset             Info             Type               Symbol's Value  Symbol's Name + Addend
+# READELF-NEXT: 0000000000000020  0000000200000002 R_X86_64_PC32          0000000000000000 .text + 0
+
+# READELF: Hex dump of section '.eh_frame':
+# READELF-NEXT: 0x00000000 14000000 00000000 017a5200 01781001
+# READELF-NEXT: 0x00000010 1b0c0708 90010000 14000000 1c000000
+# READELF-NEXT: 0x00000020 00000000 02000000 00413002 00060000

llvm/test/tools/llvm-dwarfdump/X86/debug_frame_LLVM_def_aspace_cfa.s

+# RUN: llvm-mc %s -filetype=obj -triple=i686-pc-linux -o %t
+# RUN: llvm-dwarfdump -v %t | FileCheck %s
+
+# CHECK:      .eh_frame contents:
+# CHECK:          FDE
+# CHECK-NEXT:     Format:
+# CHECK-NEXT:     DW_CFA_LLVM_def_aspace_cfa: EDX +0 as6
+# CHECK-NEXT:     DW_CFA_nop:
+
+.text
+.globl foo
+.type  foo,@function
+foo:
+ .cfi_startproc
+ .cfi_llvm_def_aspace_cfa %edx, 0, 6
+ .cfi_endproc

llvm/unittests/DebugInfo/DWARF/DWARFDebugFrameTest.cpp

llvm::DenseSet<uint8_t> ValidExtendedOpcodes = {
       dwarf::DW_CFA_def_cfa,
       dwarf::DW_CFA_def_cfa_register,
       dwarf::DW_CFA_def_cfa_offset,
       dwarf::DW_CFA_def_cfa_expression,
       dwarf::DW_CFA_expression,
       dwarf::DW_CFA_offset_extended_sf,
       dwarf::DW_CFA_def_cfa_sf,
       dwarf::DW_CFA_def_cfa_offset_sf,
+      dwarf::DW_CFA_LLVM_def_aspace_cfa,
+      dwarf::DW_CFA_LLVM_def_aspace_cfa_sf,
       dwarf::DW_CFA_val_offset,
       dwarf::DW_CFA_val_offset_sf,
       dwarf::DW_CFA_val_expression,
       dwarf::DW_CFA_MIPS_advance_loc8,
       dwarf::DW_CFA_GNU_window_save,
       dwarf::DW_CFA_AARCH64_negate_ra_state,
       dwarf::DW_CFA_GNU_args_size};