MS inline asm: produce "ambiguous" error when encountering "<type> ptr <reg name>"

[• zizhar]

Summary

Currently "<type> ptr <reg name>" treated as <reg name> in MS inline asm, ignoring the "<type> ptr" completely and possibly ignoring the intention of the user.
Fixed llvm to produce "ambiguous" error when encountering "<type> ptr <reg name>" operands.

For example: andpd xmm1,xmmword ptr xmm1 --> andpd xmm1, xmm1
though andpd has 2 possible matching formats - andpd xmm, xmm/m128

[rnk]

lib/Target/X86/AsmParser/X86AsmParser.cpp

1770–1771 ↗

(On Diff #40027)

I think a better diagnostic would be something like:
expected memory operand after 'ptr', found register operand instead

test/MC/X86/intel-syntax-ambiguous.s

56 ↗

(On Diff #40027)

Can you add a test case with a GPR operand? This also seems bad:

mov dword ptr eax, ebx

The user should write the following, right?

mov dword ptr [eax], ebx

[• zizhar]

fixed rnk's comments

[rnk]

lgtm

lib/Target/X86/AsmParser/X86AsmParser.cpp

1770 ↗

(On Diff #40271)

The broken string should align with the open paren. clang-format can do this.

[• zizhar]

fixed the error line to how clang-format does it.

llvm/trunk/lib/Target/X86/AsmParser/X86AsmParser.cpp

if (isParsingInlineAsm()) {
     if (AsmTokStr == "length" || AsmTokStr == "LENGTH")
       return ParseIntelOperator(IOK_LENGTH);
     if (AsmTokStr == "size" || AsmTokStr == "SIZE")
       return ParseIntelOperator(IOK_SIZE);
     if (AsmTokStr == "type" || AsmTokStr == "TYPE")
       return ParseIntelOperator(IOK_TYPE);
   }
 
+  bool PtrInOperand = false;
   unsigned Size = getIntelMemOperandSize(Tok.getString());
   if (Size) {
     Parser.Lex(); // Eat operand size (e.g., byte, word).
     if (Tok.getString() != "PTR" && Tok.getString() != "ptr")
       return ErrorOperand(Tok.getLoc(), "Expected 'PTR' or 'ptr' token!");
     Parser.Lex(); // Eat ptr.
+    PtrInOperand = true;
   }
   Start = Tok.getLoc();
 
   // Immediate.
   if (getLexer().is(AsmToken::Integer) || getLexer().is(AsmToken::Minus) ||
       getLexer().is(AsmToken::Tilde) || getLexer().is(AsmToken::LParen)) {
     AsmToken StartTok = Tok;
     IntelExprStateMachine SM(/*Imm=*/0, /*StopOnLBrac=*/true,
if (getSTI().getFeatureBits()[X86::FeatureAVX512] &&
       getLexer().is(AsmToken::LCurly))
     return ParseRoundingModeOp(Start, End);
 
   // Register.
   unsigned RegNo = 0;
   if (!ParseRegister(RegNo, Start, End)) {
     // If this is a segment register followed by a ':', then this is the start
     // of a segment override, otherwise this is a normal register reference.
-    if (getLexer().isNot(AsmToken::Colon))
+    // In case it is a normal register and there is ptr in the operand this 
+    // is an error
+    if (getLexer().isNot(AsmToken::Colon)){
+      if (PtrInOperand){
+        return ErrorOperand(Start, "expected memory operand after "
+                                   "'ptr', found register operand instead");
+      }
       return X86Operand::CreateReg(RegNo, Start, End);
+    }
     
     return ParseIntelSegmentOverride(/*SegReg=*/RegNo, Start, Size);
   }
 
   // Memory operand.
   return ParseIntelMemOperand(/*Disp=*/0, Start, Size);
 }
 
 std::unique_ptr<X86Operand> X86AsmParser::ParseATTOperand() {

llvm/trunk/test/MC/X86/intel-syntax-ambiguous.s

 add byte ptr [eax], eax
 // CHECK: error: invalid operand for instruction
 
 add rax, 3
 // CHECK: error: register %rax is only available in 64-bit mode
 
 fadd   "?half@?0??bar@@YAXXZ@4NA"
 // CHECK: error: ambiguous operand size for instruction 'fadd'
+
+// Instruction line with PTR inside check that they don't accept register as memory.
+
+// CHECK:  error: expected memory operand after 'ptr', found register operand instead
+// CHECK: andps xmm1, xmmword ptr xmm1
+andps xmm1, xmmword ptr xmm1
+// CHECK:  error: expected memory operand after 'ptr', found register operand instead
+// CHECK: andps xmmword ptr xmm1, xmm1
+andps xmmword ptr xmm1, xmm1
+// CHECK:  error: expected memory operand after 'ptr', found register operand instead
+// CHECK: mov dword ptr eax, ebx
+mov dword ptr eax, ebx