This is an archive of the discontinued LLVM Phabricator instance.

[ms] [llvm-ml] Add a draft MASM parser
ClosedPublic

Authored by epastor on Jan 13 2020, 8:37 PM.

Details

Summary

Many directives are unavailable, and support for others may be limited.

This first draft has preliminary support for:

  • conditional directives (including errors),
  • data allocation (unsigned types up to 8 bytes, and ALIGN),
  • equates/variables (numeric and text),
  • and procedure directives (without parameters),

as well as COMMENT, ECHO, INCLUDE, INCLUDELIB, PUBLIC, and EXTERN. Text variables (aka text macros) are expanded in-place wherever the identifier occurs.

We deliberately ignore all ml.exe processor directives.

Prominent features not yet supported:

  • structs
  • macros (both procedures and functions)
  • procedures (with specified parameters)
  • substitution & expansion operators

Conditional directives are complicated by the fact that "ifdef rax" is a valid way to check if a file is being assembled for a 64-bit x86 processor; we add support for "ifdef <register>" in general, which requires adding a tryParseRegister method to all MCTargetAsmParsers. (Some targets require backtracking in the non-register case.)

Diff Detail

Event Timeline

epastor created this revision.Jan 13 2020, 8:37 PM
epastor edited the summary of this revision. (Show Details)

Unit tests: fail. 61613 tests passed, 186 failed and 781 were skipped.

failed: LLVM.CodeGen/X86/2012-01-12-extract-sv.ll
failed: LLVM.CodeGen/X86/2012-01-18-vbitcast.ll
failed: LLVM.CodeGen/X86/2012-07-10-extload64.ll
failed: LLVM.CodeGen/X86/add.ll
failed: LLVM.CodeGen/X86/apm.ll
failed: LLVM.CodeGen/X86/arg-copy-elide-win64.ll
failed: LLVM.CodeGen/X86/avx-intel-ocl.ll
failed: LLVM.CodeGen/X86/avx512-intel-ocl.ll
failed: LLVM.CodeGen/X86/avx512-regcall-Mask.ll
failed: LLVM.CodeGen/X86/avx512-regcall-NoMask.ll
failed: LLVM.CodeGen/X86/catchpad-lifetime.ll
failed: LLVM.CodeGen/X86/catchpad-realign-savexmm.ll
failed: LLVM.CodeGen/X86/catchpad-regmask.ll
failed: LLVM.CodeGen/X86/catchpad-reuse.ll
failed: LLVM.CodeGen/X86/catchret-empty-fallthrough.ll
failed: LLVM.CodeGen/X86/catchret-fallthrough.ll
failed: LLVM.CodeGen/X86/catchret-regmask.ll
failed: LLVM.CodeGen/X86/cfguard-checks.ll
failed: LLVM.CodeGen/X86/cfguard-x86-64-vectorcall.ll
failed: LLVM.CodeGen/X86/cfguard-x86-vectorcall.ll
failed: LLVM.CodeGen/X86/cleanuppad-large-codemodel.ll
failed: LLVM.CodeGen/X86/cleanuppad-realign.ll
failed: LLVM.CodeGen/X86/conditional-tailcall-pgso.ll
failed: LLVM.CodeGen/X86/conditional-tailcall.ll
failed: LLVM.CodeGen/X86/consecutive-load-shuffle.ll
failed: LLVM.CodeGen/X86/eh-unknown.ll
failed: LLVM.CodeGen/X86/finite-libcalls.ll
failed: LLVM.CodeGen/X86/fma-commute-x86.ll
failed: LLVM.CodeGen/X86/fma-intrinsics-canonical.ll
failed: LLVM.CodeGen/X86/fma-intrinsics-x86-upgrade.ll
failed: LLVM.CodeGen/X86/fma-intrinsics-x86.ll
failed: LLVM.CodeGen/X86/fma4-commute-x86.ll
failed: LLVM.CodeGen/X86/fp-strict-libcalls-msvc32.ll
failed: LLVM.CodeGen/X86/funclet-layout.ll
failed: LLVM.CodeGen/X86/gather-addresses.ll
failed: LLVM.CodeGen/X86/h-registers-0.ll
failed: LLVM.CodeGen/X86/label-annotation.ll
failed: LLVM.CodeGen/X86/label-heapallocsite.ll
failed: LLVM.CodeGen/X86/late-address-taken.ll
failed: LLVM.CodeGen/X86/lea-3.ll
failed: LLVM.CodeGen/X86/lea.ll
failed: LLVM.CodeGen/X86/mem-intrin-base-reg.ll
failed: LLVM.CodeGen/X86/movgs.ll
failed: LLVM.CodeGen/X86/movtopush.ll
failed: LLVM.CodeGen/X86/musttail-inalloca.ll
failed: LLVM.CodeGen/X86/musttail-varargs.ll
failed: LLVM.CodeGen/X86/mwaitx.ll
failed: LLVM.CodeGen/X86/nomovtopush.ll
failed: LLVM.CodeGen/X86/noreturn-call-win64.ll
failed: LLVM.CodeGen/X86/noreturn-call.ll
failed: LLVM.CodeGen/X86/scalar-fp-to-i32.ll
failed: LLVM.CodeGen/X86/scalar-fp-to-i64.ll
failed: LLVM.CodeGen/X86/seh-catch-all-win32.ll
failed: LLVM.CodeGen/X86/seh-catch-all.ll
failed: LLVM.CodeGen/X86/seh-catchpad.ll
failed: LLVM.CodeGen/X86/seh-exception-code.ll
failed: LLVM.CodeGen/X86/seh-finally.ll
failed: LLVM.CodeGen/X86/seh-no-invokes.ll
failed: LLVM.CodeGen/X86/seh-safe-div-win32.ll
failed: LLVM.CodeGen/X86/seh-safe-div.ll
failed: LLVM.CodeGen/X86/seh-stack-realign.ll
failed: LLVM.CodeGen/X86/shrink-wrap-chkstk.ll
failed: LLVM.CodeGen/X86/sse-regcall.ll
failed: LLVM.CodeGen/X86/tail-call-mutable-memarg.ll
failed: LLVM.CodeGen/X86/taildup-heapallocsite.ll
failed: LLVM.CodeGen/X86/test-shrink.ll
failed: LLVM.CodeGen/X86/this-return-64.ll
failed: LLVM.CodeGen/X86/unreachable-trap.ll
failed: LLVM.CodeGen/X86/use-add-flags.ll
failed: LLVM.CodeGen/X86/vec_cast.ll
failed: LLVM.CodeGen/X86/win-catchpad-csrs.ll
failed: LLVM.CodeGen/X86/win-catchpad-nested.ll
failed: LLVM.CodeGen/X86/win-catchpad-varargs.ll
failed: LLVM.CodeGen/X86/win-catchpad.ll
failed: LLVM.CodeGen/X86/win-cleanuppad.ll
failed: LLVM.CodeGen/X86/win-funclet-cfi.ll
failed: LLVM.CodeGen/X86/win-mixed-ehpersonality.ll
failed: LLVM.CodeGen/X86/win32-eh.ll
failed: LLVM.CodeGen/X86/win32-seh-catchpad-realign.ll
failed: LLVM.CodeGen/X86/win32-seh-catchpad.ll
failed: LLVM.CodeGen/X86/win32-seh-nested-finally.ll
failed: LLVM.CodeGen/X86/win64-funclet-savexmm.ll
failed: LLVM.CodeGen/X86/win64-jumptable.ll
failed: LLVM.CodeGen/X86/win64_frame.ll
failed: LLVM.CodeGen/X86/win64_sibcall.ll
failed: LLVM.CodeGen/X86/win_chkstk.ll
failed: LLVM.CodeGen/X86/win_coreclr_chkstk.ll
failed: LLVM.CodeGen/X86/win_cst_pool.ll
failed: LLVM.CodeGen/X86/wineh-coreclr.ll
failed: LLVM.CodeGen/X86/wineh-exceptionpointer.ll
failed: LLVM.CodeGen/X86/wineh-no-ehpads.ll
failed: LLVM.CodeGen/X86/xor.ll
failed: LLVM.DebugInfo/COFF/array-odr-violation.ll
failed: LLVM.DebugInfo/COFF/asan-module-ctor.ll
failed: LLVM.DebugInfo/COFF/class-options-common.ll
failed: LLVM.DebugInfo/COFF/comdat.ll
failed: LLVM.DebugInfo/COFF/dlang.ll
failed: LLVM.DebugInfo/COFF/fpo-csrs.ll
failed: LLVM.DebugInfo/COFF/fpo-funclet.ll
failed: LLVM.DebugInfo/COFF/fpo-realign-alloca.ll
failed: LLVM.DebugInfo/COFF/fpo-realign-vframe.ll
failed: LLVM.DebugInfo/COFF/fpo-shrink-wrap.ll
failed: LLVM.DebugInfo/COFF/fpo-stack-protect.ll
failed: LLVM.DebugInfo/COFF/global-constants.ll
failed: LLVM.DebugInfo/COFF/global-type-hashes.ll
failed: LLVM.DebugInfo/COFF/globals-discarded.ll
failed: LLVM.DebugInfo/COFF/globals.ll
failed: LLVM.DebugInfo/COFF/inlining-header.ll
failed: LLVM.DebugInfo/COFF/inlining.ll
failed: LLVM.DebugInfo/COFF/line-zero.ll
failed: LLVM.DebugInfo/COFF/lines-bb-start.ll
failed: LLVM.DebugInfo/COFF/local-variable-gap.ll
failed: LLVM.DebugInfo/COFF/local-variables.ll
failed: LLVM.DebugInfo/COFF/multifile.ll
failed: LLVM.DebugInfo/COFF/multifunction.ll
failed: LLVM.DebugInfo/COFF/nrvo.ll
failed: LLVM.DebugInfo/COFF/parameter-order.ll
failed: LLVM.DebugInfo/COFF/pieces.ll
failed: LLVM.DebugInfo/COFF/pr28747.ll
failed: LLVM.DebugInfo/COFF/pr37492.ll
failed: LLVM.DebugInfo/COFF/register-variables.ll
failed: LLVM.DebugInfo/COFF/simple.ll
failed: LLVM.DebugInfo/COFF/swift.ll
failed: LLVM.DebugInfo/COFF/synthetic.ll
failed: LLVM.DebugInfo/COFF/thunk.ll
failed: LLVM.DebugInfo/COFF/types-basic.ll
failed: LLVM.DebugInfo/COFF/types-data-members.ll
failed: LLVM.DebugInfo/COFF/vframe-csr.ll
failed: LLVM.DebugInfo/COFF/vframe-fpo.ll
failed: LLVM.DebugInfo/X86/dbg-declare-inalloca.ll
failed: LLVM.DebugInfo/X86/dbg-value-regmask-clobber.ll
failed: LLVM.DebugInfo/X86/dbg-value-transfer-order.ll
failed: LLVM.DebugInfo/X86/pieces-4.ll
failed: LLVM.DebugInfo/X86/ref_addr_relocation.ll
failed: LLVM.DebugInfo/X86/spill-nospill.ll
failed: LLVM.ExecutionEngine/RuntimeDyld/X86/COFF_x86_64.s
failed: LLVM.MC/AsmParser/seh-directive-errors.s
failed: LLVM.MC/COFF/assoc-private.s
failed: LLVM.MC/COFF/cross-section-relative.s
failed: LLVM.MC/COFF/cv-compiler-info.ll
failed: LLVM.MC/COFF/cv-def-range-align.s
failed: LLVM.MC/COFF/cv-def-range-gap.s
failed: LLVM.MC/COFF/cv-def-range.s
failed: LLVM.MC/COFF/cv-empty-file-table.s
failed: LLVM.MC/COFF/cv-empty-linetable.s
failed: LLVM.MC/COFF/cv-errors.s
failed: LLVM.MC/COFF/cv-fpo-csrs.s
failed: LLVM.MC/COFF/cv-fpo-errors.s
failed: LLVM.MC/COFF/cv-fpo-realign.s
failed: LLVM.MC/COFF/cv-fpo-setframe.s
failed: LLVM.MC/COFF/cv-inline-linetable-infloop.s
failed: LLVM.MC/COFF/cv-inline-linetable-unlikely.s
failed: LLVM.MC/COFF/cv-inline-linetable-unreachable.s
failed: LLVM.MC/COFF/cv-inline-linetable.s
failed: LLVM.MC/COFF/cv-loc-cross-section.s
failed: LLVM.MC/COFF/cv-loc-unreachable-2.s
failed: LLVM.MC/COFF/cv-loc-unreachable.s
failed: LLVM.MC/COFF/cv-string.s
failed: LLVM.MC/COFF/seh-section.s
failed: LLVM.MC/COFF/seh.s
failed: LLVM.MC/COFF/symbol-alias.s
failed: LLVM.MC/COFF/symbol-fragment-offset-64.s
failed: LLVM.MC/COFF/symbol-fragment-offset.s
failed: LLVM.MC/X86/reloc-directive.s
failed: lld.COFF/associative-comdat.s
failed: lld.COFF/export-stdcall.s
failed: lld.COFF/export-weak-alias.s
failed: lld.COFF/gfids-corrupt.s
failed: lld.COFF/gfids-export.s
failed: lld.COFF/gfids-fallback.s
failed: lld.COFF/gfids-icf.s
failed: lld.COFF/gfids-relocations32.s
failed: lld.COFF/gfids-relocations64.s
failed: lld.COFF/guard-longjmp.s
failed: lld.COFF/guardcf-thunk.s
failed: lld.COFF/icf-pdata.s
failed: lld.COFF/icf-xdata.s
failed: lld.COFF/pdb-debug-f.s
failed: lld.COFF/pdb-global-constants.test
failed: lld.COFF/pdb-inlinees-extrafiles.s
failed: lld.COFF/pdb-inlinees.s
failed: lld.COFF/pdb-unknown-subsection.s
failed: lld.COFF/s_udt.s
failed: lld.COFF/safeseh-no.s
failed: lld.COFF/safeseh-notable.s
failed: lld.COFF/string-tail-merge.s

clang-tidy: unknown.

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

epastor updated this revision to Diff 237950.Jan 14 2020, 5:25 AM

Keep MASM-specific MCAsmInfo separate to avoid overriding CommentString, etc.

Unit tests: unknown.

clang-tidy: unknown.

clang-format: unknown.

Build artifacts: console-log.txt

lenary removed a subscriber: lenary.Jan 14 2020, 5:27 AM
epastor updated this revision to Diff 238333.Jan 15 2020, 12:38 PM

Rebasing after changes to D72679.

Unit tests: unknown.

clang-tidy: unknown.

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-format.patch, CMakeCache.txt, console-log.txt

epastor updated this revision to Diff 239199.Jan 20 2020, 1:55 PM

Label all lit tests for llvm-ml as requiring X86 targets

Unit tests: pass. 62028 tests passed, 0 failed and 783 were skipped.

clang-tidy: unknown.

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

epastor updated this revision to Diff 239690.Jan 22 2020, 1:57 PM

Rebase, and add support for signed data types

Unit tests: pass. 62114 tests passed, 0 failed and 808 were skipped.

clang-tidy: fail. clang-tidy found 1 errors and 65 warnings.

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-tidy.txt, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

This is a big diff. But AsmParser is a big interface, so I'm not sure it can be all that much smaller.

Some parts of MasmParser looks like they're from the other class and not needed. Is the idea to copy most things over for now and then delete dead code later, to keep the git diff clearer?

Maybe the tryParseRegister bits could be in a separate change?

llvm/include/llvm/MC/MCParser/MCAsmParser.h
311

This looks very similar to the line above; maybe add a comment that draws attention to the extra "M".

llvm/lib/MC/MCParser/MasmParser.cpp
676

Maybe say "llvm-ml currently only supports COFF output" and use one branch for all the other file types.

1276–1277

If we need this, should this be in the super class?

1487

Likely don't need this one?

1563

ditto

epastor marked 8 inline comments as done.Jan 24 2020, 2:38 PM

Some parts of MasmParser looks like they're from the other class and not needed. Is the idea to copy most things over for now and then delete dead code later, to keep the git diff clearer?

That was the idea - both for the git diff, and for my own ease of understanding. I've been planning on cleanup passes towards the end, as we see what's really unnecessary - along with refactors that pushes more shared code up into the parent class. If this sort of temporary duplication is frowned upon, I could move that up the priority list.

Maybe the tryParseRegister bits could be in a separate change?

Not a bad thought. I'll look into splitting that out and stacking this on top... though that seems a bit messy to do with git.

llvm/lib/MC/MCParser/MasmParser.cpp
1276–1277

Removed.

1487

Removed.

1563

Nope, we need this one! It could be shared code, but - see the main comment.

epastor updated this revision to Diff 240294.Jan 24 2020, 2:39 PM
epastor marked 3 inline comments as done.

Resolving comments

Unit tests: fail. 62110 tests passed, 5 failed and 807 were skipped.

failed: libc++.std/language_support/cmp/cmp_partialord/partialord.pass.cpp
failed: libc++.std/language_support/cmp/cmp_strongeq/cmp.strongeq.pass.cpp
failed: libc++.std/language_support/cmp/cmp_strongord/strongord.pass.cpp
failed: libc++.std/language_support/cmp/cmp_weakeq/cmp.weakeq.pass.cpp
failed: libc++.std/language_support/cmp/cmp_weakord/weakord.pass.cpp

clang-tidy: fail. clang-tidy found 1 errors and 69 warnings. 0 of them are added as review comments below (why?).

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-tidy.txt, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

Pre-merge checks is in beta. Report issue. Please join beta or enable it for your project.

thakis accepted this revision.Jan 27 2020, 7:42 AM

Make sense. If it's easy to pull out the one part mentioned above, give that a try, else I think it's fine as-is too.

This revision is now accepted and ready to land.Jan 27 2020, 7:42 AM
epastor updated this revision to Diff 240598.Jan 27 2020, 8:11 AM

Split out tryParseRegister addition as a separate change, for easier review.

Unit tests: pass. 62222 tests passed, 0 failed and 816 were skipped.

clang-tidy: fail. clang-tidy found 1 errors and 69 warnings. 0 of them are added as review comments below (why?).

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-tidy.txt, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

Pre-merge checks is in beta. Report issue. Please join beta or enable it for your project.

nhaehnle removed a subscriber: nhaehnle.Jan 29 2020, 1:35 AM
epastor updated this revision to Diff 244880.Feb 16 2020, 8:00 AM

Rebase (hopefully final)

epastor updated this revision to Diff 244885.Feb 16 2020, 8:45 AM

Fix casing & casting errors due to rebase

This revision was automatically updated to reflect the committed changes.
MaskRay added inline comments.Jul 7 2023, 3:17 PM
llvm/lib/MC/MCParser/AsmLexer.cpp
599–608

The code was refactored by D99889. As of today, there is no test for ? at the start of a symbol.

Herald added a project: Restricted Project. · View Herald TranscriptJul 7 2023, 3:17 PM
Herald added a subscriber: pengfei. · View Herald Transcript