Remove added checks

Addressed comments by @skan.

Ping

Thank you for the comments. Indeed this "read-only" is not well-defined. What we actually want to test is if the section is writable or not. Abandoning this diff.

Addressed linter warning

Handle X86AlignForMacroFusion option first.
Override it if X86AlignBranchWithin32BBoundaries is set.

@RKSimon, @skan, @MaskRay, @lebedev.ri: I've addressed all comments and suggestions. I suggest that we limit this discussion to NeverAlign fragment, and direct all questions and comments regarding automatic macro-fusion alignment to D101817.
Please let me know if there's anything to change here, otherwise I'd gently ask you to approve.

Addressed linter warning

Addressed comments by @skan

Addressed the comment by @skan:

• removed newly added BranchKinds, reused existing ones

@skan:
The big picture is that NeverAlign fragment insertion is controlled by a client (BOLT), while BoundaryAlign insertion is done automatically. This automatic macro-fusion alignment might be useful as a standalone performance optimization.
This diff leverages the infrastructure to detect eligible macro-fusion pairs and insert BoundaryAlign fragment. I've added logic on top of it to perform macro-fusion alignment when that is requested by an option.

@lebedev.ri:

Where is this fragment going to be added? The alignment will differ per CPU.

Sorry, didn't understand this question then. Do you mean handling of X86 subtargets?

Addressed comments by @MaskRay and @lebedev.ri:

1. Directive parsing error messages
2. Combined tests into single file
3. Reworded comment for emitNeverAlignCodeAtEnd

@lebedev.ri:

Where is this fragment going to be added? The alignment will differ per CPU.

This fragment is inserted before first instruction in macro-fusion pair (cmp instruction in cmp+jcc pair). Modern Intel Cores have a macro-fusion restriction that cmp+jcc shouldn't be split by a cache line boundary. It's OK to for cmp instruction to cross cache line boundary. Not all X86 cores have macro-fusion or this restriction, so the insertion policy and alignment is up to the MC client (BOLT or assembly programmer).

Addressed clang-format warnings

Moved up and reused isAgainstBoundary.

Do I need to add some documentation for the new assembler directive?

Shengchen @skan, Fangrui @MaskRay: thank you for detailed comments and suggestions! I don't have commit access, can you please commit this for me?

Added a test case suggested by @skan with NeverAlign followed by RelaxableFragment

Addressed comments by @skan, added test case for errors in .avoid_end_align directive

Lost the original diff in the rebase, fixed that

Rebased

Addressed comments by @MaskRay, added X86 assembler directive, added a testcase using the directive

@skan: I've experimented with adding customization of BoundaryAlign fragment and conducted some tests to see if it's functionally identical and has similar processing time.
I've implemented two options: automatic alignment (similar to JCC erratum mitigation with -x86-branches-within-32B-boundaries), and client insertion: exposed interfaces to insert BoundaryAlign into OS from the client (identical to current use of NeverAlign fragment in BOLT).
Automatic alignment has noticeably higher overhead than current alignment with NeverAlign, which is expected since there are more fragments inserted (BoundaryAlign fragments are inserted on every cmp/test instruction).
However, even with client insertion (BoundaryAlign fragment is inserted using the same logic as NeverAlign), BoundaryAlign still has higher overhead. One test binary has up to ~40% increase in processing time (1:09.36elapsed for NeverAlign vs 1:38.23 for BoundaryAlign).
The overhead comes from the fact that BoundaryAlign relies on relaxation to determine its size and invalidates all following fragments, while NeverAlign uses simpler logic similar to FT_Align which doesn't trigger invalidation. At this point I don't think we should pursue this direction further as the processing time is important for BOLT. I can submit a diff with automatic alignment for macro-fusion for a review, and it might be of use in a compiler/assembler (maybe as -O3 optimization?), but it won't be used by BOLT.
Thank you for suggesting to try to reuse BoundaryAlign and pointing out various correctness issues handled by BoundaryAlign.

@MaskRay, @skan, appreciate your comments and suggestions!
@MaskRay:

The patch should come with with tests.

Agree, still trying to find a way to test added functionality outside of BOLT's use case. Thank you for reviewing this patch and BOLT upstreaming repo!

We've considered two options to integrate this functionality into LLVM:

1. Similar to JCC errata/BoundaryAlign fragment:

Automatically insert NeverAlign into the section on cmp instruction, check if fuseable instruction follows it, and remove the fragment otherwise.

2. Similar to the current usage in BOLT: https://github.com/facebookincubator/BOLT/blob/68abc968b706b55585b1b8be315aef5d3bf90b1c/bolt/src/BinaryEmitter.cpp#L445

Check basic block instructions in advance, only emit NeverAlign fragment if fuseable sequence is found.

@hoy, thank you for your suggestions! Yes, it's one of the first BOLT patches, but not the first :)
We've decided with @maksfb to upstream our alignment-for-macrofusion logic as well, will include it into the follow-up revision. This would enable us to produce a test case.

Formatting fixes