Add builtins for aligning and checking alignment of pointers and integers

[arichardson]

Summary

This change introduces three new builtins (which work on both pointers
and integers) that can be used instead of common bitwise arithmetic:
builtin_align_up(x, alignment), builtin_align_down(x, alignment) and
__builtin_is_aligned(x, alignment).

I originally added these builtins to the CHERI fork of LLVM a few years ago
to handle the slightly different C semantics that we use for CHERI [1].
Until recently these builtins (or sequences of other builtins) were
required to generate correct code. I have since made changes to the default
C semantics so that they are no longer strictly necessary (but using them
does generate slightly more efficient code). However, based on our experience
using them in various projects over the past few years, I believe that adding
these builtins to clang would be useful.

These builtins have the following benefit over bit-manipulation and casts
via uintptr_t:

• The named builtins clearly convey the semantics of the operation. While checking alignment using builtin_is_aligned(x, 16) versus ((x & 15) == 0) is probably not a huge win in readably, I personally find builtin_align_up(x, N) a lot easier to read than (x+(N-1))&~(N-1).
• They preserve the type of the argument (including const qualifiers). When using casts via uintptr_t, it is easy to cast to the wrong type or strip qualifiers such as const.
• If the alignment argument is a constant value, clang can check that it is a power-of-two and within the range of the type. Since the semantics of these builtins is well defined compared to arbitraty bit-manipulation, it is possible to add a UBSAN checker that the run-time value is a valid power-of-two. I intend to add this as a follow-up to this change.
• The builtins avoids int-to-pointer casts both in C and LLVM IR. In the future (i.e. once most optimizations handle it), we could use the new llvm.ptrmask intrinsic to avoid the ptrtoint instruction that would normally be generated.
• They can be used to round up/down to the next aligned value for both integers and pointers without requiring two separate macros.
• In many projects the alignment operations are already wrapped in macros (e.g. roundup2 and rounddown2 in FreeBSD), so by replacing the macro implementation with a builtin call, we get improved diagnostics for many call-sites while only having to change a few lines.

[1] In our CHERI compiler we have compilation mode where all pointers are
implemented as capabilities (essentially unforgeable 128-bit fat pointers).
In our original model, casts from uintptr_t (which is a 128-bit capability)
to an integer value returned the "offset" of the capability (i.e. the
difference between the virtual address and the base of the allocation).
This causes problems for cases such as checking the alignment: for example, the
expression if ((uintptr_t)ptr & 63) == 0 is generally used to check if the
pointer is aligned to a multiple of 64 bytes. The problem with offsets is that
any pointer to the beginning of an allocation will have an offset of zero, so
this check always succeeds in that case (even if the address is not correctly
aligned). The same issues also exist when aligning up or down. Using the
alignment builtins ensures that the address is used instead of the offset. While
I have since changed the default C semantics to return the address instead of
the offset when casting, this offset compilation mode can still be used by
passing a command-line flag.

[merge_guards_bot]

Unit tests: fail. 60874 tests passed, 1 failed and 726 were skipped.

failed: Clang.SemaCXX/builtin-align-cxx.cpp

clang-format: pass.

Build artifacts: console-log.txt, CMakeCache.txt, test-results.xml, diff.json

[lebedev.ri]

clang/lib/CodeGen/CGBuiltin.cpp

14328

Is sufficient amount of passes, analyses know about this intrinsic?

[arichardson]

clang/lib/CodeGen/CGBuiltin.cpp

14328

Good question. In the simple test cases that I looked at the code generation was equivalent.

In our fork we still use ptrtoint+inttoptr since I implemented them before the new intrinsic existed. But since the ptrmask instrinsic exists now I thought I'd use it for upstreaming.
I'll investigate if this results in worse codegen for more complex uses.

[lebedev.ri]

clang/lib/CodeGen/CGBuiltin.cpp

14328

(TLDR: before producing it in more cases in clang, i think it should be first ensured
that everything in middle-end is fully aware of said intrinsic. (i.e. using it vs it's
exploded form results in no differences in final assembly on a sufficient test coverage))

[rsmith]

To me these seem sufficiently useful to be worth having. Please add documentation for them.

clang/include/clang/Basic/DiagnosticSemaKinds.td

2926

"the result of checking whether [...] is always true" would be better.

clang/lib/AST/ExprConstant.cpp

10636–10638

There are also cases where the first argument is a pointer for which we should be able to constant-fold __builtin_is_aligned (if we can evaluate it to a specific offset within a specific object, and we know the alignment for that object as a whole). See the handling for __builtin_assume_aligned, and factor out the common part.

10636–10640

These should be evaluated in the same EvalInfo as the enclosing evaluation, so they can use (eg) the values of local state that's already been computed in a constexpr function.

10643

You need to produce a diagnostic on failure.

10648

Would it make more sense to convert integer arguments to intptr_t when checking the call to the builtin?

clang/lib/Sema/SemaChecking.cpp

222

You should actually decay the argument to a pointer here. More generally, the right pattern to use here is:

1. Compute the parameter type you want the builtin to have for this call, then
2. Perform a copy-initialization of a parameter with that type from the argument, then
3. Update TheCall's corresponding argument to the converted form.

You can find this pattern in various builtin checking functions in this file if you want something to crib from. (Search for InitializeParameter.)

[fhahn]

clang/lib/CodeGen/CGBuiltin.cpp

14328

I'll investigate if this results in worse codegen for more complex uses.

The findings would be interesting indeed.

One thing to watch out for is that ptrmask should give better alias analysis results than ptrtoint/inttoptr. Also, IIRC some instcombine transformations for ptrtoint/inttoptr are not strictly valid according to the LangRef. Not sure if that changed yet.

[theraven]

clang/lib/CodeGen/CGBuiltin.cpp

14328

There is currently an open review on the semantics of inttoptr / ptrtoint. The current LangRef underspecifies it. On most architectures, both are bitcasts. On CHERI (including ARM's Morello system), it is not safe to round trip a pointer via an integer at the IR or machine-code level (in C, [u]intptr_t is represented as i8* in the IR). A few architectures do some arithmetic to cast between pointer and integer. The mid-level optimisers are slowly getting better at avoiding the patterns that rely on the underspecified behaviour, but removing this need for them would be a benefit.

[arichardson]

• Add support for constant-evaluating pointer values
• Add errors messages for constant evaluation
• Extend tests for constant evaluation
• Use ptrtoint+inttoptr instead of ptrmask as suggested by @lebedev.ri
• Fix SemaChecking.cpp

[merge_guards_bot]

Unit tests: pass. 60961 tests passed, 0 failed and 726 were skipped.

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or apply this patch.

Build artifacts: diff.json, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

[arichardson]

Fix align32array[0] -> &align32array[0] typo in test

[merge_guards_bot]

Unit tests: pass. 60963 tests passed, 0 failed and 726 were skipped.

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or apply this patch.

Build artifacts: diff.json, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

[arichardson]

• Add documentation for the new alignment builtins

[merge_guards_bot]

Unit tests: pass. 61055 tests passed, 0 failed and 728 were skipped.

clang-tidy: fail. Please fix clang-tidy findings.

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-tidy.txt, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

[arichardson]

Ping.

[aaron.ballman]

clang/docs/LanguageExtensions.rst

2565

Should explicitly specify whether the arguments are expressed in bytes or bits.

2574

determined -> determine
is be not -> is not

clang/lib/AST/ExprConstant.cpp

8179

Can go with const auto * where the type is spelled out in the initialization like this (here and elsewhere).

8181–8183

No else after a return.

8198

Should drop the top-level const qualifier as that isn't an idiom we typically use for locals.

10707

Comment missing a full stop at the end.

10742

This assertion doesn't add much given the above line of code.

10756

Same here as above.

clang/lib/CodeGen/CGBuiltin.cpp

14272

Can elide braces.

clang/lib/Sema/SemaChecking.cpp

224

Comment should probably be a FIXME unless you intend to address it as part of this patch. Also, the comment is missing a full stop (please check all comments in the patch as many of them are missing one).

[arichardson]

• Address feedback

[merge_guards_bot]

Unit tests: pass. 61165 tests passed, 0 failed and 728 were skipped.

clang-tidy: fail. Please fix clang-tidy findings.

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-tidy.txt, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

[aaron.ballman]

LGTM with a few nits to be fixed.

clang/lib/AST/ExprConstant.cpp

8181–8183

You can still drop this else (and use a regular if) because of the preceding return.

10707

The comment is still missing its full stop at the end.

clang/lib/Sema/SemaChecking.cpp

246–259

There are a bunch of else after return issues in this ladder. I would rearrange to put the error cases first and the warning case last.

[arichardson]

• Address remaining comments

[merge_guards_bot]

Unit tests: pass. 61172 tests passed, 0 failed and 728 were skipped.

clang-tidy: fail. Please fix clang-tidy findings.

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-tidy.txt, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

[lebedev.ri]

clang/lib/CodeGen/CGBuiltin.cpp

14327

Until we are there, can we still emit sane IR, without inttoptr?
https://godbolt.org/z/atBtSx

[arichardson]

clang/lib/CodeGen/CGBuiltin.cpp

14327

It seems like avoiding the select might be better?
https://godbolt.org/z/UdPjZk

I've done that in the current version.

[arichardson]

Address feedback: Avoid inttoptr by using ptrtoint + GEP instead.

Using two GEPs for align_up seems to generate better code than select + a single GEP.
See https://godbolt.org/z/UdPjZk

[arichardson]

• Also update the other codegen test

[merge_guards_bot]

Unit tests: fail. 61172 tests passed, 1 failed and 728 were skipped.

failed: Clang.CodeGen/builtin-align-array.c

clang-tidy: fail. Please fix clang-tidy findings.

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-tidy.txt, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

[merge_guards_bot]

Unit tests: pass. 61173 tests passed, 0 failed and 728 were skipped.

clang-tidy: fail. Please fix clang-tidy findings.

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-tidy.txt, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

[lebedev.ri]

In D71499#1800636, @arichardson wrote:

Address feedback: Avoid inttoptr by using ptrtoint + GEP instead.

Using two GEPs for align_up seems to generate better code than select + a single GEP.
See https://godbolt.org/z/UdPjZk

Avoiding select is good, true. But we can do better with a single gep still:
https://godbolt.org/z/u7YHKw

[lebedev.ri]

clang/lib/CodeGen/CGBuiltin.cpp

14323

But this isn't what we are doing, negation != inversion.

[arichardson]

Improved code generation with a single GEP

The pointer case of align_up is now generates the expected add+mask assembly even without the llvm.ptrmask intrinsic.

[merge_guards_bot]

Unit tests: pass. 61172 tests passed, 0 failed and 728 were skipped.

clang-tidy: fail. Please fix clang-tidy findings.

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-tidy.txt, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

[lebedev.ri]

Looks ok to me now in principle.
I have one more question about pointer variants though (see inline)

clang/lib/CodeGen/CGBuiltin.cpp

14326

I honestly wonder if (at least for align up) that would result in worse results,
but we'll see if/when such patch appears i guess..

14332

What's the semantics of these aligning builtins for pointers?
Is this GEP supposed to comply to the C17 6.5.6p8, C++ [expr.add]?
(TLDR: both the old pointer, and the newly-aligned pointer
must both point to the same array (allocated memory region))

I.e. can this GEP be inbounds? If it can, it'd be most great for optimizations.

[arichardson]

In D71499#1801104, @lebedev.ri wrote:

Looks ok to me now in principle.
I have one more question about pointer variants though (see inline)

I am not sure the GEP can be inbounds since I have seen some cases where aligning pointers is used to get a pointer to a different object.
I most cases it should be in-bounds (even when used to implement malloc()), but I have seen some cases where aligning pointers is used to get a pointer to a different object.
For example, some versions WebKit align pointers down by 64k to get a pointer to a structure that holds metadata for all objects allocated inside that region.

I am not sure what happens for those cases if we add inbounds (miscompilation?), so I haven't added it here.
I guess we could add it if alignment is a constant and is less than the object size, but there might already be a pass to infer if a GEP is inbounds?

[lebedev.ri]

(would be good for @nlopes to comment, maybe i'm overweighting this..)

In D71499#1801119, @arichardson wrote:

In D71499#1801104, @lebedev.ri wrote:

Looks ok to me now in principle.
I have one more question about pointer variants though (see inline)

What i'm asking is:

• Are these builtins designed (as per clang/docs/LanguageExtensions.rst) to only be passed pointers in-bounds to the allocated memory chunk (Logical pointer*), or any random bag of bits casted to pointer type (Physical Pointer*)?
• If Logical pointer, are they designed to also produce Logical pointer? Or Physical Pointer?

I am not sure the GEP can be inbounds since I have seen some cases
where aligning pointers is used to get a pointer to a different object.
I most cases it should be in-bounds (even when used to implement malloc()),
but I have seen some cases where aligning pointers is used to get a pointer
to a different object.

Object as in C++ object?
I'm specifically talking about memory region/chunk, as in what is returned by malloc().

For example, some versions WebKit align pointers down by 64k to get a pointer to a structure that holds metadata for all objects allocated inside that region.

But that entire region is still a single memory region/chunk,
not just some other random memory chunk that *happens* to be close nearby?

I am not sure what happens for those cases if we add inbounds (miscompilation?), so I haven't added it here.
I guess we could add it if alignment is a constant and is less than the object size, but there might already be a pass to infer if a GEP is inbounds?

I'm pushing on this because these intrinsics are supposed to be better than hand-rolled variants
(and in their current form with a single non-inbounds GEP they already are infinitly better
than ptrtoint+inttoptr pair), but if we go with current design (return Physical pointer),
(which is less optimizer friendly than gep inbounds which would requre/produce Logical pointers),
we'll be stuck with it..

Since there is no such builtin currently (certainly not in clang,
i don't see one in GCC), we do get to dictate it's semantics.
We don't have to define it to be most lax/UB-free (ptrtoint+inttoptr or non-inbounds gep),
but we can define it to be most optimizer-friendly (gep inbounds).

• https://web.ist.utl.pt/nuno.lopes/pubs/llvmmem-oopsla18.pdf

[arichardson]

What i'm asking is:

• Are these builtins designed (as per clang/docs/LanguageExtensions.rst) to only be passed pointers in-bounds to the allocated memory chunk (Logical pointer*), or any random bag of bits casted to pointer type (Physical Pointer*)?
• If Logical pointer, are they designed to also produce Logical pointer? Or Physical Pointer?

In my view you both input and output should be a Logical pointer. If you want random values, you should be aligning uintptr_t instead.
For CHERI we only need to support passing and creating valid pointers (i.e. with the capability tag bit set and valid bounds information).
Since the source pointer contains bounds information, it will never be possible to use the __builtin_align_up/down result to access memory from a different underlying allocation.

I am not sure the GEP can be inbounds since I have seen some cases
where aligning pointers is used to get a pointer to a different object.
I most cases it should be in-bounds (even when used to implement malloc()),
but I have seen some cases where aligning pointers is used to get a pointer
to a different object.

Object as in C++ object?
I'm specifically talking about memory region/chunk, as in what is returned by malloc().

For example, some versions WebKit align pointers down by 64k to get a pointer to a structure that holds metadata for all objects allocated inside that region.

But that entire region is still a single memory region/chunk,
not just some other random memory chunk that *happens* to be close nearby?

Since we can run that code on CHERI-MIPS, I realize it must be the same memory region.
I had look at the code again and the aligned structure is constructed in a 64K region that is created by a single call to fastMalloc(blockSize), i.e. malloc()/mmap().

I am not sure what happens for those cases if we add inbounds (miscompilation?), so I haven't added it here.
I guess we could add it if alignment is a constant and is less than the object size, but there might already be a pass to infer if a GEP is inbounds?

I'm pushing on this because these intrinsics are supposed to be better than hand-rolled variants
(and in their current form with a single non-inbounds GEP they already are infinitly better
than ptrtoint+inttoptr pair), but if we go with current design (return Physical pointer),
(which is less optimizer friendly than gep inbounds which would requre/produce Logical pointers),
we'll be stuck with it..

Since there is no such builtin currently (certainly not in clang,
i don't see one in GCC), we do get to dictate it's semantics.
We don't have to define it to be most lax/UB-free (ptrtoint+inttoptr or non-inbounds gep),
but we can define it to be most optimizer-friendly (gep inbounds).

• https://web.ist.utl.pt/nuno.lopes/pubs/llvmmem-oopsla18.pdf

It seems like we should be able to add inbounds to the GEP when aligning pointer values.

However, I think this means that for the downstream CHERI codegen there is one more case where we need to be careful (but that shouldn't matter for upstream).
For us uintptr_t is represented as i8 addrspace(200)*.
As uintptr_t can hold values that are plain integers and not pointers, we would need to add the inbounds only if the AST type is a pointer, and use a non-inbounds GEP for (u)intptr_t.
Does that sound correct or did I misunderstand something?

Note: Creating out-of-bounds pointers (capabilities) is fine for CHERI in hardware, they just can't be dereferenced without trapping. You can bring them back in bounds and then use them again (as long as they weren't *too* far out of bounds, since the compression scheme can't represent all possible values).

Adding @brooks in case there is a use case in the OS kernel that may need to create out-of-bounds pointers that I forgot about.

[jrtc27]

clang/docs/LanguageExtensions.rst

2565

This should clearly state whether an already-aligned argument is left unmodified or incremented/decremented by the alignment amount. As it stands it reads more like such arguments will be modified, but the implementation is to round, ie the former.

[lebedev.ri]

(temporarily undoing my review since it looks like things may change)

In D71499#1801302, @arichardson wrote:

...

Aha! :)

In D71499#1801302, @arichardson wrote:

However, I think this means that for the downstream CHERI codegen
there is one more case where we need to be careful
(but that shouldn't matter for upstream).
For us uintptr_t is represented as i8 addrspace(200)*.
As uintptr_t can hold values that are plain integers and not pointers,
we would need to add the inbounds only if the AST type is a pointer,
and use a non-inbounds GEP for (u)intptr_t.

I may be missing some details, but won't we get that for free,
since we only do gep for pointers already?

In D71499#1801302, @arichardson wrote:

Does that sound correct or did I misunderstand something?

To me this all sounds great.
Since we get to dictate^W define semantics, i'd think we can require

In D71499#1801302, @arichardson wrote:

both input and output should be a Logical pointer.
If you want random values, you should be aligning uintptr_t instead.

[arichardson]

• Generate inbounds GEP. Also mention that values must point to the same allocation in the documentation.
• Clarify that correctly aligned values do not change.

[arichardson]

In D71499#1801421, @lebedev.ri wrote:

(temporarily undoing my review since it looks like things may change)

In D71499#1801302, @arichardson wrote:

...

Aha! :)

In D71499#1801302, @arichardson wrote:

However, I think this means that for the downstream CHERI codegen
there is one more case where we need to be careful
(but that shouldn't matter for upstream).
For us uintptr_t is represented as i8 addrspace(200)*.
As uintptr_t can hold values that are plain integers and not pointers,
we would need to add the inbounds only if the AST type is a pointer,
and use a non-inbounds GEP for (u)intptr_t.

I may be missing some details, but won't we get that for free,
since we only do gep for pointers already?

I believe we would need something like this downstream instead of always creating an inbounds GEP:

if (E->getType()->isPointerType()) {
  // The result must point to the same underlying allocation. This means we
  // can use an inbounds GEP to enable better optimization.
  Result = Builder.CreateInBoundsGEP(EmitCastToVoidPtr(Args.Src),
                                     Difference, "aligned_result");
} else {
  // However, for when performing operations on __intcap_t (which is also
  // a pointer type in LLVM IR), we cannot set the inbounds flag as the
  // result could be either an arbitrary integer value or a valid pointer.
  // Setting the inbounds flag for the arbitrary integer case is not safe.
  assert(E->getType()->isIntCapType());
  Result = Builder.CreateGEP(EmitCastToVoidPtr(Args.Src), Difference,
                             "aligned_result");
}

In D71499#1801302, @arichardson wrote:

Does that sound correct or did I misunderstand something?

To me this all sounds great.
Since we get to dictate^W define semantics, i'd think we can require

In D71499#1801302, @arichardson wrote:

both input and output should be a Logical pointer.
If you want random values, you should be aligning uintptr_t instead.

Thank you very much for the useful suggestion!

[arichardson]

• Fix typos caused by dodgy n key on my keyboard.

[merge_guards_bot]

Unit tests: pass. 61176 tests passed, 0 failed and 729 were skipped.

clang-tidy: fail. Please fix clang-tidy findings.

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-tidy.txt, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

[lebedev.ri]

Hmm, i keep coming up with things here, sorry :/

I think we should take one more step - explicitly call out that the result of __builtin_align_* is, well, aligned :)
For that, we need to add __attribute__((alloc_align(2))) attribute.
Despite the name, it does not imply anything about provenance/aliasing/etc, only about alignment:
https://godbolt.org/z/9bAjxK

clang/docs/LanguageExtensions.rst

2575–2576

Might be worth explicitly calling out C17 6.5.6p8, C++ [expr.add]?

clang/lib/CodeGen/CGBuiltin.cpp

14334–14335

I would suspect this should follow suit of the rest of clang codegen, i.e. do

Value* Base = EmitCastToVoidPtr(Args.Src);
if (CGF.getLangOpts().isSignedOverflowDefined())
  Result = Builder.CreateGEP(Base, Difference, "aligned_result");
else
  Result = CGF.EmitCheckedInBoundsGEP(Base, Difference,
                 /*SignedIndices=*/true, /*isSubtraction*/=false,
                 E->getExprLoc(), "aligned_result");

[merge_guards_bot]

Unit tests: pass. 61176 tests passed, 0 failed and 729 were skipped.

clang-tidy: fail. Please fix clang-tidy findings.

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-tidy.txt, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

[lebedev.ri]

For future reference, if anyone needs here's the C versions of these builtins:
https://godbolt.org/z/oHeAh-

^ looks like we are missing some backend-level folds for round-down variant, filed https://bugs.llvm.org/show_bug.cgi?id=44448

[lebedev.ri]

In D71499#1802134, @lebedev.ri wrote:

For future reference, if anyone needs here's the C versions of these builtins:
https://godbolt.org/z/oHeAh-

^ looks like we are missing some backend-level folds for round-down variant, filed https://bugs.llvm.org/show_bug.cgi?id=44448

Appled some backend codegen fixes (8dab0a4a7d691f2704f1079538e0ef29548db159, 3d492d7503d197246115eb38e7b1b61143d0c99f, 0727e2b90c7b11d5c6be55919c443628d8e2bc6e),
__builtin_align_down() codegen should now be as optimal as __builtin_align_up() is. (at least on X86)

[nlopes]

@lebedev.ri I agree with you that the semantics of these alignment builtins should only return a pointer that is of the same object as the one given as input.
Otherwise, these builtins would be even worst that ptr2int/int2ptr, since their result could alias with any other pointer in the program, not just the escaped pointers.

I gave a glance through the patch, especially the documentation section, and the semantics look right to me.
Would be nice to see this using ptrmask on a subsequent patch.

[arichardson]

• Add a test that the new alignment is propagated to e.g. llvm.memcpy

To do this emit a llvm.assume for pointer types and mark the builtin
as having an alloc_align attribute (although that does not seem to do anything)

• Update docs to mention C17 6.5.6p8, C++ [expr.add]

• Use getLangOpts().isSignedOverflowDefined() in codegen.

[merge_guards_bot]

Unit tests: pass. 61248 tests passed, 0 failed and 736 were skipped.

clang-tidy: fail. Please fix clang-tidy findings.

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-tidy.txt, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

[lebedev.ri]

Nice. Thank you for working on this.
I think this finally fully looks good to me.

Not sure whether we should be really adding an attribute to AST,
or just calling EmitAlignmentAssumption() like it is done now.
I'll defer to @erichkeane on that.

[lebedev.ri]

Land this? (branching is near..)

[arichardson]

In D71499#1810526, @lebedev.ri wrote:

Land this? (branching is near..)

I was planning to wait until Sunday for @erichkeane to comment whether the AST attribute is needed and if I don't hear back until then land I would commit the current version. Does that seem sensible or is it too close to the branch point?

Otherwise I can commit it tomorrow morning and change it postcommit if necessary.
What do you think?

[erichkeane]

Sorry, I didn't realize you were waiting for me. This seems fine as it is, I don't see the attribute buying us anything.

clang/docs/LanguageExtensions.rst

 point is the location of the caller. When the builtins appear as part of a
 default member initializer, the invocation point is the location of the
 constructor or aggregate initialization used to create the object. Otherwise
 the invocation point is the same as the location of the builtin.
 
 When the invocation point of ``__builtin_FUNCTION`` is not a function scope the
 empty string is returned.
 
+Alignment builtins
+------------------
+Clang provides builtins to support checking and adjusting alignment of
+pointers and integers.
+These builtins can be used to avoid relying on implementation-defined behavior
+of arithmetic on integers derived from pointers.
+Additionally, these builtins retain type information and, unlike bitwise
+arithmentic, they can perform semantic checking on the alignment value.
+
+**Syntax**:
+
+.. code-block:: c
+
+  Type __builtin_align_up(Type value, size_t alignment);
+  Type __builtin_align_down(Type value, size_t alignment);
+  bool __builtin_is_aligned(Type value, size_t alignment);
+
+
+**Example of use**:
+
+.. code-block:: c++
+
+  char* global_alloc_buffer;
+  void* my_aligned_allocator(size_t alloc_size, size_t alignment) {
+    char* result = __builtin_align_up(global_alloc_buffer, alignment);
+    // result now contains the value of global_alloc_buffer rounded up to the
+    // next multiple of alignment.
+    global_alloc_buffer = result + alloc_size;
+    return result;
+  }
+
+  void* get_start_of_page(void* ptr) {
+    return __builtin_align_down(ptr, PAGE_SIZE);
+  }
+
+  void example(char* buffer) {
+     if (__builtin_is_aligned(buffer, 64)) {
+       do_fast_aligned_copy(buffer);
+     } else {
+       do_unaligned_copy(buffer);
+     }
+  }
+
+  // In addition to pointers, the builtins can also be used on integer types
+  // and are evaluatable inside constant expressions.
+  static_assert(__builtin_align_up(123, 64) == 128, "");
+  static_assert(__builtin_align_down(123u, 64) == 64u, "");
+  static_assert(!__builtin_is_aligned(123, 64), "");
+
+
+**Description**:
+
+The builtins ``__builtin_align_up``, ``__builtin_align_down``, return their
+first argument aligned up/down to the next multiple of the second argument.
+If the value is already sufficiently aligned, it is returned unchanged.
+The builtin ``__builtin_is_aligned`` returns whether the first argument is
+aligned to a multiple of the second argument.
+All of these builtins expect the alignment to be expressed as a number of bytes.
+
+These builtins can be used for all integer types as well as (non-function)
+pointer types. For pointer types, these builtins operate in terms of the integer
+address of the pointer and return a new pointer of the same type (including
+qualifiers such as ``const``) with an adjusted address.
+When aligning pointers up or down, the resulting value must be within the same
+underlying allocation or one past the end (see C17 6.5.6p8, C++ [expr.add]).
+This means that arbitrary integer values stored in pointer-type variables must
+not be passed to these builtins. For those use cases, the builtins can still be
+used, but the operation must be performed on the pointer cast to ``uintptr_t``.
+
+If Clang can determine that the alignment is not a power of two at compile time,
+it will result in a compilation failure. If the alignment argument is not a
+power of two at run time, the behavior of these builtins is undefined.
+
 Non-standard C++11 Attributes
 =============================
 
 Clang's non-standard C++11 attributes live in the ``clang`` attribute
 namespace.
 
 Clang supports GCC's ``gnu`` attribute namespace. All GCC attributes which
 are accepted with the ``__attribute__((foo))`` syntax are also accepted as

clang/include/clang/Basic/Builtins.def

 // Clang builtins (not available in GCC).
 BUILTIN(__builtin_addressof, "v*v&", "nct")
 BUILTIN(__builtin_operator_new, "v*z", "tc")
 BUILTIN(__builtin_operator_delete, "vv*", "tn")
 BUILTIN(__builtin_char_memchr, "c*cC*iz", "n")
 BUILTIN(__builtin_dump_struct, "ivC*v*", "tn")
 BUILTIN(__builtin_preserve_access_index, "v.", "t")
 
+// Alignment builtins (uses custom parsing to support pointers and integers)
+BUILTIN(__builtin_is_aligned, "bvC*z", "nct")
+BUILTIN(__builtin_align_up, "v*vC*z", "nct")
+BUILTIN(__builtin_align_down, "v*vC*z", "nct")
+
 // Safestack builtins
 BUILTIN(__builtin___get_unsafe_stack_start, "v*", "Fn")
 BUILTIN(__builtin___get_unsafe_stack_bottom, "v*", "Fn")
 BUILTIN(__builtin___get_unsafe_stack_top, "v*", "Fn")
 BUILTIN(__builtin___get_unsafe_stack_ptr, "v*", "Fn")
 
 // Nontemporal loads/stores builtins
 BUILTIN(__builtin_nontemporal_store, "v.", "t")

clang/include/clang/Basic/DiagnosticASTKinds.td

def note_constexpr_inherited_ctor_call_here : Note<
   "in implicit initialization for inherited constructor of %0">;
 def note_constexpr_baa_insufficient_alignment : Note<
   "%select{alignment of|offset of the aligned pointer from}0 the base pointee "
   "object (%1 %plural{1:byte|:bytes}1) is %select{less than|not a multiple of}0 the "
   "asserted %2 %plural{1:byte|:bytes}2">;
 def note_constexpr_baa_value_insufficient_alignment : Note<
   "value of the aligned pointer (%0) is not a multiple of the asserted %1 "
   "%plural{1:byte|:bytes}1">;
+def note_constexpr_invalid_alignment : Note<
+  "requested alignment %0 is not a positive power of two">;
+def note_constexpr_alignment_too_big : Note<
+  "requested alignment must be %0 or less for type %1; %2 is invalid">;
+def note_constexpr_alignment_compute : Note<
+  "cannot constant evaluate whether run-time alignment is at least %0">;
+def note_constexpr_alignment_adjust : Note<
+  "cannot constant evaluate the result of adjusting alignment to %0">;
 def note_constexpr_destroy_out_of_lifetime : Note<
   "destroying object '%0' whose lifetime has already ended">;
 def note_constexpr_unsupported_destruction : Note<
   "non-trivial destruction of type %0 in a constant expression is not supported">;
 def note_constexpr_unsupported_tempoarary_nontrivial_dtor : Note<
   "non-trivial destruction of lifetime-extended temporary with type %0 "
   "used in the result of a constant expression is not yet supported">;
 def note_constexpr_unsupported_unsized_array : Note<

clang/include/clang/Basic/DiagnosticSemaKinds.td

def err_alignment_too_small : Error<
   "requested alignment must be %0 or greater">;
 def err_alignment_too_big : Error<
   "requested alignment must be %0 or smaller">;
 def err_alignment_not_power_of_two : Error<
   "requested alignment is not a power of 2">;
 def err_alignment_dependent_typedef_name : Error<
   "requested alignment is dependent but declaration is not dependent">;
 
+def warn_alignment_builtin_useless : Warning<
+  "%select{aligning a value|the result of checking whether a value is aligned}0"
+  " to 1 byte is %select{a no-op|always true}0">, InGroup<TautologicalCompare>;
 def err_attribute_aligned_too_great : Error<
   "requested alignment must be %0 bytes or smaller">;
 def warn_assume_aligned_too_great
     : Warning<"requested alignment must be %0 bytes or smaller; maximum "
               "alignment assumed">,
       InGroup<DiagGroup<"builtin-assume-aligned-alignment">>;
 def warn_redeclaration_without_attribute_prev_attribute_ignored : Warning<
   "%q0 redeclared without %1 attribute: previous %1 ignored">,

clang/lib/AST/ExprConstant.cpp

static CharUnits GetAlignOfExpr(EvalInfo &Info, const Expr *E,
 
   if (const MemberExpr *ME = dyn_cast<MemberExpr>(E))
     return Info.Ctx.getDeclAlign(ME->getMemberDecl(),
                                  /*RefAsPointee*/true);
 
   return GetAlignOfType(Info, E->getType(), ExprKind);
 }
 
+static CharUnits getBaseAlignment(EvalInfo &Info, const LValue &Value) {
+  if (const auto *VD = Value.Base.dyn_cast<const ValueDecl *>())
+    return Info.Ctx.getDeclAlign(VD);
+  if (const auto *E = Value.Base.dyn_cast<const Expr *>())
+    return GetAlignOfExpr(Info, E, UETT_AlignOf);
+  return GetAlignOfType(Info, Value.Base.getTypeInfoType(), UETT_AlignOf);
+}
+
+/// Evaluate the value of the alignment argument to __builtin_align_{up,down},
+/// __builtin_is_aligned and __builtin_assume_aligned.
+static bool getAlignmentArgument(const Expr *E, QualType ForType,
+                                 EvalInfo &Info, APSInt &Alignment) {
+  if (!EvaluateInteger(E, Alignment, Info))
+    return false;
+  if (Alignment < 0 || !Alignment.isPowerOf2()) {
+    Info.FFDiag(E, diag::note_constexpr_invalid_alignment) << Alignment;
+    return false;
+  }
+  unsigned SrcWidth = Info.Ctx.getIntWidth(ForType);
+  APSInt MaxValue(APInt::getOneBitSet(SrcWidth, SrcWidth - 1));
+  if (APSInt::compareValues(Alignment, MaxValue) > 0) {
+    Info.FFDiag(E, diag::note_constexpr_alignment_too_big)
+        << MaxValue << ForType << Alignment;
+    return false;
+  }
+  // Ensure both alignment and source value have the same bit width so that we
+  // don't assert when computing the resulting value.
+  APSInt ExtAlignment =
+      APSInt(Alignment.zextOrTrunc(SrcWidth), /*isUnsigned=*/true);
+  assert(APSInt::compareValues(Alignment, ExtAlignment) == 0 &&
+         "Alignment should not be changed by ext/trunc");
+  Alignment = ExtAlignment;
+  assert(Alignment.getBitWidth() == SrcWidth);
+  return true;
+}
+
 // To be clear: this happily visits unsupported builtins. Better name welcomed.
 bool PointerExprEvaluator::visitNonBuiltinCallExpr(const CallExpr *E) {
   if (ExprEvaluatorBaseTy::VisitCallExpr(E))
     return true;
 
   if (!(InvalidBaseOK && getAllocSizeAttr(E)))
     return false;
 
case Builtin::BI__builtin_assume_aligned: {
     // We need to be very careful here because: if the pointer does not have the
     // asserted alignment, then the behavior is undefined, and undefined
     // behavior is non-constant.
     if (!evaluatePointer(E->getArg(0), Result))
       return false;
 
     LValue OffsetResult(Result);
     APSInt Alignment;
-    if (!EvaluateInteger(E->getArg(1), Alignment, Info))
+    if (!getAlignmentArgument(E->getArg(1), E->getArg(0)->getType(), Info,
+                              Alignment))
       return false;
     CharUnits Align = CharUnits::fromQuantity(Alignment.getZExtValue());
 
     if (E->getNumArgs() > 2) {
       APSInt Offset;
       if (!EvaluateInteger(E->getArg(2), Offset, Info))
         return false;
 
       int64_t AdditionalOffset = -Offset.getZExtValue();
       OffsetResult.Offset += CharUnits::fromQuantity(AdditionalOffset);
     }
 
     // If there is a base object, then it must have the correct alignment.
     if (OffsetResult.Base) {
-      CharUnits BaseAlignment;
-      if (const ValueDecl *VD =
-          OffsetResult.Base.dyn_cast<const ValueDecl*>()) {
-        BaseAlignment = Info.Ctx.getDeclAlign(VD);
-      } else if (const Expr *E = OffsetResult.Base.dyn_cast<const Expr *>()) {
-        BaseAlignment = GetAlignOfExpr(Info, E, UETT_AlignOf);
-      } else {
-        BaseAlignment = GetAlignOfType(
-            Info, OffsetResult.Base.getTypeInfoType(), UETT_AlignOf);
-      }
+      CharUnits BaseAlignment = getBaseAlignment(Info, OffsetResult);
 
       if (BaseAlignment < Align) {
         Result.Designator.setInvalid();
         // FIXME: Add support to Diagnostic for long / long long.
         CCEDiag(E->getArg(0),
                 diag::note_constexpr_baa_insufficient_alignment) << 0
           << (unsigned)BaseAlignment.getQuantity()
           << (unsigned)Align.getQuantity();
if (OffsetResult.Offset.alignTo(Align) != OffsetResult.Offset) {
                      diag::note_constexpr_baa_value_insufficient_alignment))
         << (int)OffsetResult.Offset.getQuantity()
         << (unsigned)Align.getQuantity();
       return false;
     }
 
     return true;
   }
+  case Builtin::BI__builtin_align_up:
+  case Builtin::BI__builtin_align_down: {
+    if (!evaluatePointer(E->getArg(0), Result))
+      return false;
+    APSInt Alignment;
+    if (!getAlignmentArgument(E->getArg(1), E->getArg(0)->getType(), Info,
+                              Alignment))
+      return false;
+    CharUnits BaseAlignment = getBaseAlignment(Info, Result);
+    CharUnits PtrAlign = BaseAlignment.alignmentAtOffset(Result.Offset);
+    // For align_up/align_down, we can return the same value if the alignment
+    // is known to be greater or equal to the requested value.
+    if (PtrAlign.getQuantity() >= Alignment)
+      return true;
+
+    // The alignment could be greater than the minimum at run-time, so we cannot
+    // infer much about the resulting pointer value. One case is possible:
+    // For `_Alignas(32) char buf[N]; __builtin_align_down(&buf[idx], 32)` we
+    // can infer the correct index if the requested alignment is smaller than
+    // the base alignment so we can perform the computation on the offset.
+    if (BaseAlignment.getQuantity() >= Alignment) {
+      assert(Alignment.getBitWidth() <= 64 &&
+             "Cannot handle > 64-bit address-space");
+      uint64_t Alignment64 = Alignment.getZExtValue();
+      CharUnits NewOffset = CharUnits::fromQuantity(
+          BuiltinOp == Builtin::BI__builtin_align_down
+              ? llvm::alignDown(Result.Offset.getQuantity(), Alignment64)
+              : llvm::alignTo(Result.Offset.getQuantity(), Alignment64));
+      Result.adjustOffset(NewOffset - Result.Offset);
+      // TODO: diagnose out-of-bounds values/only allow for arrays?
+      return true;
+    }
+    // Otherwise, we cannot constant-evaluate the result.
+    Info.FFDiag(E->getArg(0), diag::note_constexpr_alignment_adjust)
+        << Alignment;
+    return false;
+  }
   case Builtin::BI__builtin_operator_new:
     return HandleOperatorNewCall(Info, E, Result);
   case Builtin::BI__builtin_launder:
     return evaluatePointer(E->getArg(0), Result);
   case Builtin::BIstrchr:
   case Builtin::BIwcschr:
   case Builtin::BImemchr:
   case Builtin::BIwmemchr:

 
 bool IntExprEvaluator::VisitCallExpr(const CallExpr *E) {
   if (unsigned BuiltinOp = E->getBuiltinCallee())
     return VisitBuiltinCallExpr(E, BuiltinOp);
 
   return ExprEvaluatorBaseTy::VisitCallExpr(E);
 }
 
+static bool getBuiltinAlignArguments(const CallExpr *E, EvalInfo &Info,
+                                     APValue &Val, APSInt &Alignment) {
+  QualType SrcTy = E->getArg(0)->getType();
+  if (!getAlignmentArgument(E->getArg(1), SrcTy, Info, Alignment))
+    return false;
+  // Even though we are evaluating integer expressions we could get a pointer
+  // argument for the __builtin_is_aligned() case.
+  if (SrcTy->isPointerType()) {
+    LValue Ptr;
+    if (!EvaluatePointer(E->getArg(0), Ptr, Info))
+      return false;
+    Ptr.moveInto(Val);
+  } else if (!SrcTy->isIntegralOrEnumerationType()) {
+    Info.FFDiag(E->getArg(0));
+    return false;
+  } else {
+    APSInt SrcInt;
+    if (!EvaluateInteger(E->getArg(0), SrcInt, Info))
+      return false;
+    assert(SrcInt.getBitWidth() >= Alignment.getBitWidth() &&
+           "Bit widths must be the same");
+    Val = APValue(SrcInt);
+  }
+  assert(Val.hasValue());
+  return true;
+}
+
 bool IntExprEvaluator::VisitBuiltinCallExpr(const CallExpr *E,
                                             unsigned BuiltinOp) {
   switch (unsigned BuiltinOp = E->getBuiltinCallee()) {
   default:
     return ExprEvaluatorBaseTy::VisitCallExpr(E);
 
   case Builtin::BI__builtin_dynamic_object_size:
   case Builtin::BI__builtin_object_size: {
bool IntExprEvaluator::VisitBuiltinCallExpr(const CallExpr *E,
   }
 
   case Builtin::BI__builtin_os_log_format_buffer_size: {
     analyze_os_log::OSLogBufferLayout Layout;
     analyze_os_log::computeOSLogBufferLayout(Info.Ctx, E, Layout);
     return Success(Layout.size().getQuantity(), E);
   }
 
+  case Builtin::BI__builtin_is_aligned: {
+    APValue Src;
+    APSInt Alignment;
+    if (!getBuiltinAlignArguments(E, Info, Src, Alignment))
+      return false;
+    if (Src.isLValue()) {
+      // If we evaluated a pointer, check the minimum known alignment.
+      LValue Ptr;
+      Ptr.setFrom(Info.Ctx, Src);
+      CharUnits BaseAlignment = getBaseAlignment(Info, Ptr);
+      CharUnits PtrAlign = BaseAlignment.alignmentAtOffset(Ptr.Offset);
+      // We can return true if the known alignment at the computed offset is
+      // greater than the requested alignment.
+      assert(PtrAlign.isPowerOfTwo());
+      assert(Alignment.isPowerOf2());
+      if (PtrAlign.getQuantity() >= Alignment)
+        return Success(1, E);
+      // If the alignment is not known to be sufficient, some cases could still
+      // be aligned at run time. However, if the requested alignment is less or
+      // equal to the base alignment and the offset is not aligned, we know that
+      // the run-time value can never be aligned.
+      if (BaseAlignment.getQuantity() >= Alignment &&
+          PtrAlign.getQuantity() < Alignment)
+        return Success(0, E);
+      // Otherwise we can't infer whether the value is sufficiently aligned.
+      // TODO: __builtin_is_aligned(__builtin_align_{down,up{(expr, N), N)
+      //  in cases where we can't fully evaluate the pointer.
+      Info.FFDiag(E->getArg(0), diag::note_constexpr_alignment_compute)
+          << Alignment;
+      return false;
+    }
+    assert(Src.isInt());
+    return Success((Src.getInt() & (Alignment - 1)) == 0 ? 1 : 0, E);
+  }
+  case Builtin::BI__builtin_align_up: {
+    APValue Src;
+    APSInt Alignment;
+    if (!getBuiltinAlignArguments(E, Info, Src, Alignment))
+      return false;
+    if (!Src.isInt())
+      return Error(E);
+    APSInt AlignedVal =
+        APSInt((Src.getInt() + (Alignment - 1)) & ~(Alignment - 1),
+               Src.getInt().isUnsigned());
+    assert(AlignedVal.getBitWidth() == Src.getInt().getBitWidth());
+    return Success(AlignedVal, E);
+  }
+  case Builtin::BI__builtin_align_down: {
+    APValue Src;
+    APSInt Alignment;
+    if (!getBuiltinAlignArguments(E, Info, Src, Alignment))
+      return false;
+    if (!Src.isInt())
+      return Error(E);
+    APSInt AlignedVal =
+        APSInt(Src.getInt() & ~(Alignment - 1), Src.getInt().isUnsigned());
+    assert(AlignedVal.getBitWidth() == Src.getInt().getBitWidth());
+    return Success(AlignedVal, E);
+  }
+
   case Builtin::BI__builtin_bswap16:
   case Builtin::BI__builtin_bswap32:
   case Builtin::BI__builtin_bswap64: {
     APSInt Val;
     if (!EvaluateInteger(E->getArg(0), Val, Info))
       return false;
 
     return Success(Val.byteSwap(), E);

clang/lib/CodeGen/CGBuiltin.cpp

case Builtin::BI__builtin_addressof:
     return RValue::get(EmitLValue(E->getArg(0)).getPointer(*this));
   case Builtin::BI__builtin_operator_new:
     return EmitBuiltinNewDeleteCall(
         E->getCallee()->getType()->castAs<FunctionProtoType>(), E, false);
   case Builtin::BI__builtin_operator_delete:
     return EmitBuiltinNewDeleteCall(
         E->getCallee()->getType()->castAs<FunctionProtoType>(), E, true);
 
+  case Builtin::BI__builtin_is_aligned:
+    return EmitBuiltinIsAligned(E);
+  case Builtin::BI__builtin_align_up:
+    return EmitBuiltinAlignTo(E, true);
+  case Builtin::BI__builtin_align_down:
+    return EmitBuiltinAlignTo(E, false);
+
   case Builtin::BI__noop:
     // __noop always evaluates to an integer literal zero.
     return RValue::get(ConstantInt::get(IntTy, 0));
   case Builtin::BI__builtin_call_with_static_chain: {
     const CallExpr *Call = cast<CallExpr>(E->getArg(0));
     const Expr *Chain = E->getArg(1);
     return EmitCall(Call->getCallee()->getType(),
                     EmitCallee(Call->getCallee()), Call, ReturnValue,
for (unsigned i = 0; i < MI.NumEltsD; ++i)
           CharUnits::fromQuantity(4));
     return Result;
   }
   default:
     return nullptr;
   }
 }
 
+struct BuiltinAlignArgs {
+  llvm::Value *Src = nullptr;
+  llvm::Type *SrcType = nullptr;
+  llvm::Value *Alignment = nullptr;
+  llvm::Value *Mask = nullptr;
+  llvm::IntegerType *IntType = nullptr;
+
+  BuiltinAlignArgs(const CallExpr *E, CodeGenFunction &CGF) {
+    QualType AstType = E->getArg(0)->getType();
+    if (AstType->isArrayType())
+      Src = CGF.EmitArrayToPointerDecay(E->getArg(0)).getPointer();
+    else
+      Src = CGF.EmitScalarExpr(E->getArg(0));
+    SrcType = Src->getType();
+    if (SrcType->isPointerTy()) {
+      IntType = IntegerType::get(
+          CGF.getLLVMContext(),
+          CGF.CGM.getDataLayout().getIndexTypeSizeInBits(SrcType));
+    } else {
+      assert(SrcType->isIntegerTy());
+      IntType = cast<llvm::IntegerType>(SrcType);
+    }
+    Alignment = CGF.EmitScalarExpr(E->getArg(1));
+    Alignment = CGF.Builder.CreateZExtOrTrunc(Alignment, IntType, "alignment");
+    auto *One = llvm::ConstantInt::get(IntType, 1);
+    Mask = CGF.Builder.CreateSub(Alignment, One, "mask");
+  }
+};
+
+/// Generate (x & (y-1)) == 0.
+RValue CodeGenFunction::EmitBuiltinIsAligned(const CallExpr *E) {
+  BuiltinAlignArgs Args(E, *this);
+  llvm::Value *SrcAddress = Args.Src;
+  if (Args.SrcType->isPointerTy())
+    SrcAddress =
+        Builder.CreateBitOrPointerCast(Args.Src, Args.IntType, "src_addr");
+  return RValue::get(Builder.CreateICmpEQ(
+      Builder.CreateAnd(SrcAddress, Args.Mask, "set_bits"),
+      llvm::Constant::getNullValue(Args.IntType), "is_aligned"));
+}
+
+/// Generate (x & ~(y-1)) to align down or ((x+(y-1)) & ~(y-1)) to align up.
+/// Note: For pointer types we can avoid ptrtoint/inttoptr pairs by using the
+/// llvm.ptrmask instrinsic (with a GEP before in the align_up case).
+/// TODO: actually use ptrmask once most optimization passes know about it.
+RValue CodeGenFunction::EmitBuiltinAlignTo(const CallExpr *E, bool AlignUp) {
+  BuiltinAlignArgs Args(E, *this);
+  llvm::Value *SrcAddr = Args.Src;
+  if (Args.Src->getType()->isPointerTy())
+    SrcAddr = Builder.CreatePtrToInt(Args.Src, Args.IntType, "intptr");
+  llvm::Value *SrcForMask = SrcAddr;
+  if (AlignUp) {
+    // When aligning up we have to first add the mask to ensure we go over the
+    // next alignment value and then align down to the next valid multiple.
+    // By adding the mask, we ensure that align_up on an already aligned
+    // value will not change the value.
+    SrcForMask = Builder.CreateAdd(SrcForMask, Args.Mask, "over_boundary");
+  }
+  // Invert the mask to only clear the lower bits.
+  llvm::Value *InvertedMask = Builder.CreateNot(Args.Mask, "inverted_mask");
+  llvm::Value *Result =
+      Builder.CreateAnd(SrcForMask, InvertedMask, "aligned_result");
+  if (Args.Src->getType()->isPointerTy()) {
+    /// TODO: Use ptrmask instead of ptrtoint+gep once it is optimized well.
+    // Result = Builder.CreateIntrinsic(
+    //  Intrinsic::ptrmask, {Args.SrcType, SrcForMask->getType(), Args.IntType},
+    //  {SrcForMask, NegatedMask}, nullptr, "aligned_result");
+    Result->setName("aligned_intptr");
+    llvm::Value *Difference = Builder.CreateSub(Result, SrcAddr, "diff");
+    // The result must point to the same underlying allocation. This means we
+    // can use an inbounds GEP to enable better optimization.
+    Value *Base = EmitCastToVoidPtr(Args.Src);
+    if (getLangOpts().isSignedOverflowDefined())
+      Result = Builder.CreateGEP(Base, Difference, "aligned_result");
+    else
+      Result = EmitCheckedInBoundsGEP(Base, Difference,
+                                      /*SignedIndices=*/true,
+                                      /*isSubtraction=*/!AlignUp,
+                                      E->getExprLoc(), "aligned_result");
+    Result = Builder.CreatePointerCast(Result, Args.SrcType);
+    // Emit an alignment assumption to ensure that the new alignment is
+    // propagated to loads/stores, etc.
+    EmitAlignmentAssumption(Result, E, E->getExprLoc(), Args.Alignment);
+  }
+  assert(Result->getType() == Args.SrcType);
+  return RValue::get(Result);
+}
+
 Value *CodeGenFunction::EmitWebAssemblyBuiltinExpr(unsigned BuiltinID,
                                                    const CallExpr *E) {
   switch (BuiltinID) {
   case WebAssembly::BI__builtin_wasm_memory_size: {
     llvm::Type *ResultType = ConvertType(E->getType());
     Value *I = EmitScalarExpr(E->getArg(0));
     Function *Callee = CGM.getIntrinsic(Intrinsic::wasm_memory_size, ResultType);
     return Builder.CreateCall(Callee, I);

clang/lib/CodeGen/CodeGenFunction.h

public:
   RValue EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
                          const CallExpr *E, ReturnValueSlot ReturnValue);
 
   RValue emitRotate(const CallExpr *E, bool IsRotateRight);
 
   /// Emit IR for __builtin_os_log_format.
   RValue emitBuiltinOSLogFormat(const CallExpr &E);
 
+  /// Emit IR for __builtin_is_aligned.
+  RValue EmitBuiltinIsAligned(const CallExpr *E);
+  /// Emit IR for __builtin_align_up/__builtin_align_down.
+  RValue EmitBuiltinAlignTo(const CallExpr *E, bool AlignUp);
+
   llvm::Function *generateBuiltinOSLogHelperFunction(
       const analyze_os_log::OSLogBufferLayout &Layout,
       CharUnits BufferAlignment);
 
   RValue EmitBlockCallExpr(const CallExpr *E, ReturnValueSlot ReturnValue);
 
   /// EmitTargetBuiltinExpr - Emit the given builtin call. Returns 0 if the call
   /// is unhandled by the current target.

clang/lib/Sema/SemaChecking.cpp

 static bool SemaBuiltinPreserveAI(Sema &S, CallExpr *TheCall) {
   if (checkArgCount(S, TheCall, 1))
     return true;
 
   TheCall->setType(TheCall->getArg(0)->getType());
   return false;
 }
 
+/// Check that the value argument for __builtin_is_aligned(value, alignment) and
+/// __builtin_aligned_{up,down}(value, alignment) is an integer or a pointer
+/// type (but not a function pointer) and that the alignment is a power-of-two.
+static bool SemaBuiltinAlignment(Sema &S, CallExpr *TheCall, unsigned ID) {
+  if (checkArgCount(S, TheCall, 2))
+    return true;
+
+  clang::Expr *Source = TheCall->getArg(0);
+  bool IsBooleanAlignBuiltin = ID == Builtin::BI__builtin_is_aligned;
+
+  auto IsValidIntegerType = [](QualType Ty) {
+    return Ty->isIntegerType() && !Ty->isEnumeralType() && !Ty->isBooleanType();
+  };
+  QualType SrcTy = Source->getType();
+  // We should also be able to use it with arrays (but not functions!).
+  if (SrcTy->canDecayToPointerType() && SrcTy->isArrayType()) {
+    SrcTy = S.Context.getDecayedType(SrcTy);
+  }
+  if ((!SrcTy->isPointerType() && !IsValidIntegerType(SrcTy)) ||
+      SrcTy->isFunctionPointerType()) {
+    // FIXME: this is not quite the right error message since we don't allow
+    // floating point types, or member pointers.
+    S.Diag(Source->getExprLoc(), diag::err_typecheck_expect_scalar_operand)
+        << SrcTy;
+    return true;
+  }
+
+  clang::Expr *AlignOp = TheCall->getArg(1);
+  if (!IsValidIntegerType(AlignOp->getType())) {
+    S.Diag(AlignOp->getExprLoc(), diag::err_typecheck_expect_int)
+        << AlignOp->getType();
+    return true;
+  }
+  Expr::EvalResult AlignResult;
+  unsigned MaxAlignmentBits = S.Context.getIntWidth(SrcTy) - 1;
+  // We can't check validity of alignment if it is type dependent.
+  if (!AlignOp->isInstantiationDependent() &&
+      AlignOp->EvaluateAsInt(AlignResult, S.Context,
+                             Expr::SE_AllowSideEffects)) {
+    llvm::APSInt AlignValue = AlignResult.Val.getInt();
+    llvm::APSInt MaxValue(
+        llvm::APInt::getOneBitSet(MaxAlignmentBits + 1, MaxAlignmentBits));
+    if (AlignValue < 1) {
+      S.Diag(AlignOp->getExprLoc(), diag::err_alignment_too_small) << 1;
+      return true;
+    }
+    if (llvm::APSInt::compareValues(AlignValue, MaxValue) > 0) {
+      S.Diag(AlignOp->getExprLoc(), diag::err_alignment_too_big)
+          << MaxValue.toString(10);
+      return true;
+    }
+    if (!AlignValue.isPowerOf2()) {
+      S.Diag(AlignOp->getExprLoc(), diag::err_alignment_not_power_of_two);
+      return true;
+    }
+    if (AlignValue == 1) {
+      S.Diag(AlignOp->getExprLoc(), diag::warn_alignment_builtin_useless)
+          << IsBooleanAlignBuiltin;
+    }
+  }
+
+  ExprResult SrcArg = S.PerformCopyInitialization(
+      InitializedEntity::InitializeParameter(S.Context, SrcTy, false),
+      SourceLocation(), Source);
+  if (SrcArg.isInvalid())
+    return true;
+  TheCall->setArg(0, SrcArg.get());
+  ExprResult AlignArg =
+      S.PerformCopyInitialization(InitializedEntity::InitializeParameter(
+                                      S.Context, AlignOp->getType(), false),
+                                  SourceLocation(), AlignOp);
+  if (AlignArg.isInvalid())
+    return true;
+  TheCall->setArg(1, AlignArg.get());
+  // For align_up/align_down, the return type is the same as the (potentially
+  // decayed) argument type including qualifiers. For is_aligned(), the result
+  // is always bool.
+  TheCall->setType(IsBooleanAlignBuiltin ? S.Context.BoolTy : SrcTy);
+  return false;
+}
+
 static bool SemaBuiltinOverflow(Sema &S, CallExpr *TheCall) {
   if (checkArgCount(S, TheCall, 3))
     return true;
 
   // First two arguments should be integers.
   for (unsigned I = 0; I < 2; ++I) {
     ExprResult Arg = TheCall->getArg(I);
     QualType Ty = Arg.get()->getType();
#include "clang/Basic/Builtins.def"
   case Builtin::BI__builtin_annotation:
     if (SemaBuiltinAnnotation(*this, TheCall))
       return ExprError();
     break;
   case Builtin::BI__builtin_addressof:
     if (SemaBuiltinAddressof(*this, TheCall))
       return ExprError();
     break;
+  case Builtin::BI__builtin_is_aligned:
+  case Builtin::BI__builtin_align_up:
+  case Builtin::BI__builtin_align_down:
+    if (SemaBuiltinAlignment(*this, TheCall, BuiltinID))
+      return ExprError();
+    break;
   case Builtin::BI__builtin_add_overflow:
   case Builtin::BI__builtin_sub_overflow:
   case Builtin::BI__builtin_mul_overflow:
     if (SemaBuiltinOverflow(*this, TheCall))
       return ExprError();
     break;
   case Builtin::BI__builtin_operator_new:
   case Builtin::BI__builtin_operator_delete: {

clang/test/CodeGen/builtin-align-array.c

+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py
+/// Check that the alignment builtins handle array-to-pointer decay
+// RUN: %clang_cc1 -triple=x86_64-unknown-unknown -o - -emit-llvm %s | FileCheck %s
+
+extern int func(char *c);
+
+// CHECK-LABEL: define {{[^@]+}}@test_array() #0
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    [[BUF:%.*]] = alloca [1024 x i8], align 16
+// CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds [1024 x i8], [1024 x i8]* [[BUF]], i64 0, i64 44
+// CHECK-NEXT:    [[INTPTR:%.*]] = ptrtoint i8* [[ARRAYIDX]] to i64
+// CHECK-NEXT:    [[ALIGNED_INTPTR:%.*]] = and i64 [[INTPTR]], -16
+// CHECK-NEXT:    [[DIFF:%.*]] = sub i64 [[ALIGNED_INTPTR]], [[INTPTR]]
+// CHECK-NEXT:    [[ALIGNED_RESULT:%.*]] = getelementptr inbounds i8, i8* [[ARRAYIDX]], i64 [[DIFF]]
+// CHECK-NEXT:    [[PTRINT:%.*]] = ptrtoint i8* [[ALIGNED_RESULT]] to i64
+// CHECK-NEXT:    [[MASKEDPTR:%.*]] = and i64 [[PTRINT]], 15
+// CHECK-NEXT:    [[MASKCOND:%.*]] = icmp eq i64 [[MASKEDPTR]], 0
+// CHECK-NEXT:    call void @llvm.assume(i1 [[MASKCOND]])
+// CHECK-NEXT:    [[CALL:%.*]] = call i32 @func(i8* [[ALIGNED_RESULT]])
+// CHECK-NEXT:    [[ARRAYIDX1:%.*]] = getelementptr inbounds [1024 x i8], [1024 x i8]* [[BUF]], i64 0, i64 22
+// CHECK-NEXT:    [[INTPTR2:%.*]] = ptrtoint i8* [[ARRAYIDX1]] to i64
+// CHECK-NEXT:    [[OVER_BOUNDARY:%.*]] = add i64 [[INTPTR2]], 31
+// CHECK-NEXT:    [[ALIGNED_INTPTR4:%.*]] = and i64 [[OVER_BOUNDARY]], -32
+// CHECK-NEXT:    [[DIFF5:%.*]] = sub i64 [[ALIGNED_INTPTR4]], [[INTPTR2]]
+// CHECK-NEXT:    [[ALIGNED_RESULT6:%.*]] = getelementptr inbounds i8, i8* [[ARRAYIDX1]], i64 [[DIFF5]]
+// CHECK-NEXT:    [[PTRINT7:%.*]] = ptrtoint i8* [[ALIGNED_RESULT6]] to i64
+// CHECK-NEXT:    [[MASKEDPTR8:%.*]] = and i64 [[PTRINT7]], 31
+// CHECK-NEXT:    [[MASKCOND9:%.*]] = icmp eq i64 [[MASKEDPTR8]], 0
+// CHECK-NEXT:    call void @llvm.assume(i1 [[MASKCOND9]])
+// CHECK-NEXT:    [[CALL10:%.*]] = call i32 @func(i8* [[ALIGNED_RESULT6]])
+// CHECK-NEXT:    [[ARRAYIDX11:%.*]] = getelementptr inbounds [1024 x i8], [1024 x i8]* [[BUF]], i64 0, i64 16
+// CHECK-NEXT:    [[SRC_ADDR:%.*]] = ptrtoint i8* [[ARRAYIDX11]] to i64
+// CHECK-NEXT:    [[SET_BITS:%.*]] = and i64 [[SRC_ADDR]], 63
+// CHECK-NEXT:    [[IS_ALIGNED:%.*]] = icmp eq i64 [[SET_BITS]], 0
+// CHECK-NEXT:    [[CONV:%.*]] = zext i1 [[IS_ALIGNED]] to i32
+// CHECK-NEXT:    ret i32 [[CONV]]
+//
+int test_array(void) {
+  char buf[1024];
+  func(__builtin_align_down(&buf[44], 16));
+  func(__builtin_align_up(&buf[22], 32));
+  return __builtin_is_aligned(&buf[16], 64);
+}
+
+// CHECK-LABEL: define {{[^@]+}}@test_array_should_not_mask() #0
+// CHECK-NEXT:  entry:
+// CHECK-NEXT:    [[BUF:%.*]] = alloca [1024 x i8], align 32
+// CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds [1024 x i8], [1024 x i8]* [[BUF]], i64 0, i64 64
+// CHECK-NEXT:    [[INTPTR:%.*]] = ptrtoint i8* [[ARRAYIDX]] to i64
+// CHECK-NEXT:    [[ALIGNED_INTPTR:%.*]] = and i64 [[INTPTR]], -16
+// CHECK-NEXT:    [[DIFF:%.*]] = sub i64 [[ALIGNED_INTPTR]], [[INTPTR]]
+// CHECK-NEXT:    [[ALIGNED_RESULT:%.*]] = getelementptr inbounds i8, i8* [[ARRAYIDX]], i64 [[DIFF]]
+// CHECK-NEXT:    [[PTRINT:%.*]] = ptrtoint i8* [[ALIGNED_RESULT]] to i64
+// CHECK-NEXT:    [[MASKEDPTR:%.*]] = and i64 [[PTRINT]], 15
+// CHECK-NEXT:    [[MASKCOND:%.*]] = icmp eq i64 [[MASKEDPTR]], 0
+// CHECK-NEXT:    call void @llvm.assume(i1 [[MASKCOND]])
+// CHECK-NEXT:    [[CALL:%.*]] = call i32 @func(i8* [[ALIGNED_RESULT]])
+// CHECK-NEXT:    [[ARRAYIDX1:%.*]] = getelementptr inbounds [1024 x i8], [1024 x i8]* [[BUF]], i64 0, i64 32
+// CHECK-NEXT:    [[INTPTR2:%.*]] = ptrtoint i8* [[ARRAYIDX1]] to i64
+// CHECK-NEXT:    [[OVER_BOUNDARY:%.*]] = add i64 [[INTPTR2]], 31
+// CHECK-NEXT:    [[ALIGNED_INTPTR4:%.*]] = and i64 [[OVER_BOUNDARY]], -32
+// CHECK-NEXT:    [[DIFF5:%.*]] = sub i64 [[ALIGNED_INTPTR4]], [[INTPTR2]]
+// CHECK-NEXT:    [[ALIGNED_RESULT6:%.*]] = getelementptr inbounds i8, i8* [[ARRAYIDX1]], i64 [[DIFF5]]
+// CHECK-NEXT:    [[PTRINT7:%.*]] = ptrtoint i8* [[ALIGNED_RESULT6]] to i64
+// CHECK-NEXT:    [[MASKEDPTR8:%.*]] = and i64 [[PTRINT7]], 31
+// CHECK-NEXT:    [[MASKCOND9:%.*]] = icmp eq i64 [[MASKEDPTR8]], 0
+// CHECK-NEXT:    call void @llvm.assume(i1 [[MASKCOND9]])
+// CHECK-NEXT:    [[CALL10:%.*]] = call i32 @func(i8* [[ALIGNED_RESULT6]])
+// CHECK-NEXT:    ret i32 1
+//
+int test_array_should_not_mask(void) {
+  _Alignas(32) char buf[1024];
+  // TODO: The align_up and align_down calls should be folded to no-ops
+  func(__builtin_align_down(&buf[64], 16));
+  func(__builtin_align_up(&buf[32], 32));
+  // This expression can be constant-evaluated:
+  return __builtin_is_aligned(&buf[64], 32);
+}

clang/test/CodeGen/builtin-align-assumption.c

+/// Check that the new alignment set by the alignment builtins is propagated
+/// to e.g. llvm.memcpy calls.
+// RUN: %clang_cc1 -triple=x86_64-unknown-unknown %s -emit-llvm -O1 -o - | FileCheck %s
+
+// CHECK-LABEL: define {{[^@]+}}@align_up
+// CHECK:         call void @llvm.memcpy.p0i8.p0i8.i64(i8* nonnull align 64 dereferenceable(16) {{%.+}}, i8* nonnull align 1 dereferenceable(16) {{%.+}}, i64 16, i1 false)
+// CHECK-NEXT:    ret void
+//
+void align_up(void* data, int* ptr) {
+  // The call to llvm.memcpy should have an "align 64" on the first argument
+  __builtin_memcpy(__builtin_align_up(ptr, 64), data, 16);
+}

clang/test/CodeGen/builtin-align.c

+/// Check the code generation for the alignment builtins
+/// To make the test case easier to read, run SROA after generating IR to remove the alloca instructions.
+// RUN: %clang_cc1 -triple=x86_64-unknown-unknown -DTEST_VOID_PTR \
+// RUN:   -o - -emit-llvm %s -disable-O0-optnone | opt -S -sroa | \
+// RUN:   FileCheck %s -check-prefixes CHECK,POINTER,ALIGNMENT_EXT \
+// RUN:   -enable-var-scope '-D$PTRTYPE=i8'
+// RUN: %clang_cc1 -triple=x86_64-unknown-unknown -DTEST_FLOAT_PTR \
+// RUN:   -o - -emit-llvm %s -disable-O0-optnone | opt -S -sroa | \
+// RUN:   FileCheck %s -check-prefixes CHECK,POINTER,NON_I8_POINTER,ALIGNMENT_EXT \
+// RUN:   -enable-var-scope '-D$PTRTYPE=f32'
+// RUN: %clang_cc1 -triple=x86_64-unknown-unknown -DTEST_LONG \
+// RUN:   -o - -emit-llvm %s -disable-O0-optnone | opt -S -sroa | \
+// RUN:   FileCheck %s -check-prefixes CHECK,INTEGER,ALIGNMENT_EXT -enable-var-scope
+/// Check that we can handle the case where the alignment parameter is wider
+/// than the source type (generate a trunc on alignment instead of zext)
+// RUN: %clang_cc1 -triple=x86_64-unknown-unknown -DTEST_USHORT \
+// RUN:   -o - -emit-llvm %s -disable-O0-optnone | opt -S -sroa | \
+// RUN:   FileCheck %s -check-prefixes CHECK,INTEGER,ALIGNMENT_TRUNC -enable-var-scope
+
+
+#ifdef TEST_VOID_PTR
+#define TYPE void *
+#elif defined(TEST_FLOAT_PTR)
+#define TYPE float *
+#elif defined(TEST_LONG)
+#define TYPE long
+#elif defined(TEST_CAP)
+#define TYPE void *__capability
+#elif defined(TEST_USHORT)
+#define TYPE unsigned short
+#else
+#error MISSING TYPE
+#endif
+
+/// Check that constant initializers work and are correct
+_Bool aligned_true = __builtin_is_aligned(1024, 512);
+// CHECK: @aligned_true = global i8 1, align 1
+_Bool aligned_false = __builtin_is_aligned(123, 512);
+// CHECK: @aligned_false = global i8 0, align 1
+
+int down_1 = __builtin_align_down(1023, 32);
+// CHECK: @down_1 = global i32 992, align 4
+int down_2 = __builtin_align_down(256, 32);
+// CHECK: @down_2 = global i32 256, align 4
+
+int up_1 = __builtin_align_up(1023, 32);
+// CHECK: @up_1 = global i32 1024, align 4
+int up_2 = __builtin_align_up(256, 32);
+// CHECK: @up_2 = global i32 256, align 4
+
+/// Capture the IR type here to use in the remaining FileCheck captures:
+// CHECK: define {{[^@]+}}@get_type() #0
+// CHECK-NEXT:  entry:
+// POINTER-NEXT:    ret [[$TYPE:.+]] null
+// INTEGER-NEXT:    ret [[$TYPE:.+]] 0
+//
+TYPE get_type(void) {
+  return (TYPE)0;
+}
+
+// CHECK-LABEL: define {{[^@]+}}@is_aligned
+// CHECK-SAME: ([[$TYPE]] {{[^%]*}}[[PTR:%.*]], i32 [[ALIGN:%.*]]) #0
+// CHECK-NEXT:  entry:
+// ALIGNMENT_EXT-NEXT:   [[ALIGNMENT:%.*]] = zext i32 [[ALIGN]] to [[ALIGN_TYPE:i64]]
+// ALIGNMENT_TRUNC-NEXT: [[ALIGNMENT:%.*]] = trunc i32 [[ALIGN]] to [[ALIGN_TYPE:i16]]
+// CHECK-NEXT:           [[MASK:%.*]] = sub [[ALIGN_TYPE]] [[ALIGNMENT]], 1
+// POINTER-NEXT:         [[PTR:%.*]] = ptrtoint [[$TYPE]] %ptr to i64
+// CHECK-NEXT:           [[SET_BITS:%.*]] = and [[ALIGN_TYPE]] [[PTR]], [[MASK]]
+// CHECK-NEXT:           [[IS_ALIGNED:%.*]] = icmp eq [[ALIGN_TYPE]] [[SET_BITS]], 0
+// CHECK-NEXT:           ret i1 [[IS_ALIGNED]]
+//
+_Bool is_aligned(TYPE ptr, unsigned align) {
+  return __builtin_is_aligned(ptr, align);
+}
+
+// CHECK-LABEL: define {{[^@]+}}@align_up
+// CHECK-SAME: ([[$TYPE]] {{[^%]*}}[[PTR:%.*]], i32 [[ALIGN:%.*]]) #0
+// CHECK-NEXT:  entry:
+// ALIGNMENT_EXT-NEXT:   [[ALIGNMENT:%.*]] = zext i32 [[ALIGN]] to [[ALIGN_TYPE:i64]]
+// ALIGNMENT_TRUNC-NEXT: [[ALIGNMENT:%.*]] = trunc i32 [[ALIGN]] to [[ALIGN_TYPE:i16]]
+// CHECK-NEXT:           [[MASK:%.*]] = sub [[ALIGN_TYPE]] [[ALIGNMENT]], 1
+// INTEGER-NEXT:         [[OVER_BOUNDARY:%.*]] = add [[$TYPE]] [[PTR]], [[MASK]]
+// NOTYET-POINTER-NEXT:  [[ALIGNED_RESULT:%.*]] = call [[$TYPE]] @llvm.ptrmask.p0[[$PTRTYPE]].p0i8.i64(i8* [[OVER_BOUNDARY]], [[ALIGN_TYPE]] [[INVERTED_MASK]])
+// POINTER-NEXT:         [[INTPTR:%.*]] = ptrtoint [[$TYPE]] [[PTR]] to [[ALIGN_TYPE]]
+// POINTER-NEXT:         [[OVER_BOUNDARY:%.*]] = add [[ALIGN_TYPE]] [[INTPTR]], [[MASK]]
+// CHECK-NEXT:           [[INVERTED_MASK:%.*]] = xor [[ALIGN_TYPE]] [[MASK]], -1
+// CHECK-NEXT:           [[ALIGNED_RESULT:%.*]] = and [[ALIGN_TYPE]] [[OVER_BOUNDARY]], [[INVERTED_MASK]]
+// POINTER-NEXT:         [[DIFF:%.*]] = sub i64 [[ALIGNED_RESULT]], [[INTPTR]]
+// NON_I8_POINTER-NEXT:  [[PTR:%.*]] = bitcast [[$TYPE]] {{%.*}} to i8*
+// POINTER-NEXT:         [[ALIGNED_RESULT:%.*]] = getelementptr inbounds i8, i8* [[PTR]], i64 [[DIFF]]
+// NON_I8_POINTER-NEXT:  [[ALIGNED_RESULT:%.*]] = bitcast i8* {{%.*}} to [[$TYPE]]
+// POINTER-NEXT:         [[ASSUME_MASK:%.*]] = sub i64 %alignment, 1
+// POINTER-NEXT:         [[ASSUME_INTPTR:%.*]]= ptrtoint [[$TYPE]] [[ALIGNED_RESULT]] to i64
+// POINTER-NEXT:         [[MASKEDPTR:%.*]] = and i64 %ptrint, [[ASSUME_MASK]]
+// POINTER-NEXT:         [[MASKEDCOND:%.*]] = icmp eq i64 [[MASKEDPTR]], 0
+// POINTER-NEXT:         call void @llvm.assume(i1 [[MASKEDCOND]])
+// CHECK-NEXT:           ret [[$TYPE]] [[ALIGNED_RESULT]]
+//
+TYPE align_up(TYPE ptr, unsigned align) {
+  return __builtin_align_up(ptr, align);
+}
+
+// CHECK-LABEL: define {{[^@]+}}@align_down
+// CHECK-SAME: ([[$TYPE]] {{[^%]*}}[[PTR:%.*]], i32 [[ALIGN:%.*]]) #0
+// CHECK-NEXT:  entry:
+// ALIGNMENT_EXT-NEXT:   [[ALIGNMENT:%.*]] = zext i32 [[ALIGN]] to [[ALIGN_TYPE:i64]]
+// ALIGNMENT_TRUNC-NEXT: [[ALIGNMENT:%.*]] = trunc i32 [[ALIGN]] to [[ALIGN_TYPE:i16]]
+// CHECK-NEXT:           [[MASK:%.*]] = sub [[ALIGN_TYPE]] [[ALIGNMENT]], 1
+// NOTYET-POINTER-NEXT:  [[ALIGNED_RESULT:%.*]] = call [[$TYPE]] @llvm.ptrmask.p0[[$PTRTYPE]].p0[[$PTRTYPE]].i64([[$TYPE]] [[PTR]], [[ALIGN_TYPE]] [[INVERTED_MASK]])
+// POINTER-NEXT:         [[INTPTR:%.*]] = ptrtoint [[$TYPE]] [[PTR]] to [[ALIGN_TYPE]]
+// CHECK-NEXT:           [[INVERTED_MASK:%.*]] = xor [[ALIGN_TYPE]] [[MASK]], -1
+// POINTER-NEXT:         [[ALIGNED_INTPTR:%.*]] = and [[ALIGN_TYPE]] [[INTPTR]], [[INVERTED_MASK]]
+// POINTER-NEXT:         [[DIFF:%.*]] = sub i64 [[ALIGNED_INTPTR]], [[INTPTR]]
+// NON_I8_POINTER-NEXT:  [[PTR:%.*]] = bitcast [[$TYPE]] {{%.*}} to i8*
+// POINTER-NEXT:         [[ALIGNED_RESULT:%.*]] = getelementptr inbounds i8, i8* [[PTR]], i64 [[DIFF]]
+// NON_I8_POINTER-NEXT:  [[ALIGNED_RESULT:%.*]] = bitcast i8* {{%.*}} to [[$TYPE]]
+// INTEGER-NEXT:         [[ALIGNED_RESULT:%.*]] = and [[ALIGN_TYPE]] [[PTR]], [[INVERTED_MASK]]
+// POINTER-NEXT:         [[ASSUME_MASK:%.*]] = sub i64 %alignment, 1
+// POINTER-NEXT:         [[ASSUME_INTPTR:%.*]]= ptrtoint [[$TYPE]] [[ALIGNED_RESULT]] to i64
+// POINTER-NEXT:         [[MASKEDPTR:%.*]] = and i64 %ptrint, [[ASSUME_MASK]]
+// POINTER-NEXT:         [[MASKEDCOND:%.*]] = icmp eq i64 [[MASKEDPTR]], 0
+// POINTER-NEXT:         call void @llvm.assume(i1 [[MASKEDCOND]])
+// CHECK-NEXT:           ret [[$TYPE]] [[ALIGNED_RESULT]]
+//
+TYPE align_down(TYPE ptr, unsigned align) {
+  return __builtin_align_down(ptr, align);
+}

clang/test/Sema/builtin-align.c

+// RUN: %clang_cc1 -triple x86_64-linux-gnu -DALIGN_BUILTIN=__builtin_align_down -DRETURNS_BOOL=0 %s -fsyntax-only -verify -Wpedantic
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -DALIGN_BUILTIN=__builtin_align_up   -DRETURNS_BOOL=0 %s -fsyntax-only -verify -Wpedantic
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -DALIGN_BUILTIN=__builtin_is_aligned -DRETURNS_BOOL=1 %s -fsyntax-only -verify -Wpedantic
+
+struct Aggregate {
+  int i;
+  int j;
+};
+enum Enum { EnumValue1,
+            EnumValue2 };
+typedef __SIZE_TYPE__ size_t;
+
+void test_parameter_types(char *ptr, size_t size) {
+  struct Aggregate agg;
+  enum Enum e = EnumValue2;
+  _Bool b = 0;
+
+  // The first parameter can be any pointer or integer type:
+  (void)ALIGN_BUILTIN(ptr, 4);
+  (void)ALIGN_BUILTIN(size, 2);
+  (void)ALIGN_BUILTIN(12345, 2);
+  (void)ALIGN_BUILTIN(agg, 2);    // expected-error {{operand of type 'struct Aggregate' where arithmetic or pointer type is required}}
+  (void)ALIGN_BUILTIN(e, 2);      // expected-error {{operand of type 'enum Enum' where arithmetic or pointer type is required}}
+  (void)ALIGN_BUILTIN(b, 2);      // expected-error {{operand of type '_Bool' where arithmetic or pointer type is required}}
+  (void)ALIGN_BUILTIN((int)e, 2); // but with a cast it is fine
+  (void)ALIGN_BUILTIN((int)b, 2); // but with a cast it is fine
+
+  // The second parameter must be an integer type (but not enum or _Bool):
+  (void)ALIGN_BUILTIN(ptr, size);
+  (void)ALIGN_BUILTIN(ptr, ptr);    // expected-error {{used type 'char *' where integer is required}}
+  (void)ALIGN_BUILTIN(ptr, agg);    // expected-error {{used type 'struct Aggregate' where integer is required}}
+  (void)ALIGN_BUILTIN(ptr, b);      // expected-error {{used type '_Bool' where integer is required}}
+  (void)ALIGN_BUILTIN(ptr, e);      // expected-error {{used type 'enum Enum' where integer is required}}
+  (void)ALIGN_BUILTIN(ptr, (int)e); // but with a cast enums are fine
+  (void)ALIGN_BUILTIN(ptr, (int)b); // but with a cast booleans are fine
+
+  (void)ALIGN_BUILTIN(ptr, size);
+  (void)ALIGN_BUILTIN(size, size);
+}
+
+void test_result_unused(int i, int align) {
+  // -Wunused-result does not trigger for macros so we can't use ALIGN_BUILTIN()
+  // but need to explicitly call each function.
+  __builtin_align_up(i, align);   // expected-warning{{ignoring return value of function declared with const attribute}}
+  __builtin_align_down(i, align); // expected-warning{{ignoring return value of function declared with const attribute}}
+  __builtin_is_aligned(i, align); // expected-warning{{ignoring return value of function declared with const attribute}}
+  ALIGN_BUILTIN(i, align);        // no warning here
+}
+
+#define check_same_type(type1, type2) __builtin_types_compatible_p(type1, type2) && __builtin_types_compatible_p(type1 *, type2 *)
+
+void test_return_type(void *ptr, int i, long l) {
+  char array[32];
+  __extension__ typedef typeof(ALIGN_BUILTIN(ptr, 4)) result_type_ptr;
+  __extension__ typedef typeof(ALIGN_BUILTIN(i, 4)) result_type_int;
+  __extension__ typedef typeof(ALIGN_BUILTIN(l, 4)) result_type_long;
+  __extension__ typedef typeof(ALIGN_BUILTIN(array, 4)) result_type_char_array;
+#if RETURNS_BOOL
+  _Static_assert(check_same_type(_Bool, result_type_ptr), "Should return bool");
+  _Static_assert(check_same_type(_Bool, result_type_int), "Should return bool");
+  _Static_assert(check_same_type(_Bool, result_type_long), "Should return bool");
+  _Static_assert(check_same_type(_Bool, result_type_char_array), "Should return bool");
+#else
+  _Static_assert(check_same_type(void *, result_type_ptr), "Should return void*");
+  _Static_assert(check_same_type(int, result_type_int), "Should return int");
+  _Static_assert(check_same_type(long, result_type_long), "Should return long");
+  // Check that we can use the alignment builtins on on array types (result should decay)
+  _Static_assert(check_same_type(char *, result_type_char_array),
+                 "Using the builtins on an array should yield the decayed type");
+#endif
+}
+
+void test_invalid_alignment_values(char *ptr, long *longptr, size_t align) {
+  int x = 1;
+  (void)ALIGN_BUILTIN(ptr, 2);
+  (void)ALIGN_BUILTIN(longptr, 1024);
+  (void)ALIGN_BUILTIN(x, 32);
+
+  (void)ALIGN_BUILTIN(ptr, 0); // expected-error {{requested alignment must be 1 or greater}}
+  (void)ALIGN_BUILTIN(ptr, 1);
+#if RETURNS_BOOL
+  // expected-warning@-2 {{checking whether a value is aligned to 1 byte is always true}}
+#else
+  // expected-warning@-4 {{aligning a value to 1 byte is a no-op}}
+#endif
+  (void)ALIGN_BUILTIN(ptr, 3); // expected-error {{requested alignment is not a power of 2}}
+  (void)ALIGN_BUILTIN(x, 7);   // expected-error {{requested alignment is not a power of 2}}
+
+  // check the maximum range for smaller types:
+  __UINT8_TYPE__ c = ' ';
+
+  (void)ALIGN_BUILTIN(c, 128);        // this is fine
+  (void)ALIGN_BUILTIN(c, 256);        // expected-error {{requested alignment must be 128 or smaller}}
+  (void)ALIGN_BUILTIN(x, 1ULL << 31); // this is also fine
+  (void)ALIGN_BUILTIN(x, 1LL << 31);  // this is also fine
+  __INT32_TYPE__ i32 = 3;
+  __UINT32_TYPE__ u32 = 3;
+  // Maximum is the same for int32 and uint32
+  (void)ALIGN_BUILTIN(i32, 1ULL << 32);              // expected-error {{requested alignment must be 2147483648 or smaller}}
+  (void)ALIGN_BUILTIN(u32, 1ULL << 32);              // expected-error {{requested alignment must be 2147483648 or smaller}}
+  (void)ALIGN_BUILTIN(ptr, ((__int128)1) << 65);     // expected-error {{requested alignment must be 9223372036854775808 or smaller}}
+  (void)ALIGN_BUILTIN(longptr, ((__int128)1) << 65); // expected-error {{requested alignment must be 9223372036854775808 or smaller}}
+
+  const int bad_align = 8 + 1;
+  (void)ALIGN_BUILTIN(ptr, bad_align); // expected-error {{requested alignment is not a power of 2}}
+}
+
+// Check that it can be used in constant expressions:
+void constant_expression(int x) {
+  _Static_assert(__builtin_is_aligned(1024, 512), "");
+  _Static_assert(!__builtin_is_aligned(256, 512ULL), "");
+  _Static_assert(__builtin_align_up(33, 32) == 64, "");
+  _Static_assert(__builtin_align_down(33, 32) == 32, "");
+
+  // But not if one of the arguments isn't constant:
+  _Static_assert(ALIGN_BUILTIN(33, x) != 100, ""); // expected-error {{static_assert expression is not an integral constant expression}}
+  _Static_assert(ALIGN_BUILTIN(x, 4) != 100, "");  // expected-error {{static_assert expression is not an integral constant expression}}
+}
+
+// Check that it is a constant expression that can be assigned to globals:
+int global1 = __builtin_align_down(33, 8);
+int global2 = __builtin_align_up(33, 8);
+_Bool global3 = __builtin_is_aligned(33, 8);
+
+extern void test_ptr(char *c);
+char *test_array_and_fnptr(void) {
+  char buf[1024];
+  // The builtins should also work on arrays (decaying the return type)
+  (void)(ALIGN_BUILTIN(buf, 16));
+  // But not on functions and function pointers:
+  (void)(ALIGN_BUILTIN(test_array_and_fnptr, 16));  // expected-error{{operand of type 'char *(void)' where arithmetic or pointer type is required}}
+  (void)(ALIGN_BUILTIN(&test_array_and_fnptr, 16)); // expected-error{{operand of type 'char *(*)(void)' where arithmetic or pointer type is required}}
+}

clang/test/SemaCXX/builtin-align-cxx.cpp

+// C++-specific checks for the alignment builtins
+// RUN: %clang_cc1 -triple=x86_64-unknown-unknown -std=c++11 -o - %s -fsyntax-only -verify
+
+// Check that we don't crash when using dependent types in __builtin_align:
+template <typename a, a b>
+void *c(void *d) { // expected-note{{candidate template ignored}}
+  return __builtin_align_down(d, b);
+}
+
+struct x {};
+x foo;
+void test(void *value) {
+  c<int, 16>(value);
+  c<struct x, foo>(value); // expected-error{{no matching function for call to 'c'}}
+}
+
+template <typename T, long Alignment, long ArraySize = 16>
+void test_templated_arguments() {
+  T array[ArraySize];                                                           // expected-error{{variable has incomplete type 'fwddecl'}}
+  static_assert(__is_same(decltype(__builtin_align_up(array, Alignment)), T *), // expected-error{{requested alignment is not a power of 2}}
+                "return type should be the decayed array type");
+  static_assert(__is_same(decltype(__builtin_align_down(array, Alignment)), T *),
+                "return type should be the decayed array type");
+  static_assert(__is_same(decltype(__builtin_is_aligned(array, Alignment)), bool),
+                "return type should be bool");
+  T *x1 = __builtin_align_up(array, Alignment);
+  T *x2 = __builtin_align_down(array, Alignment);
+  bool x3 = __builtin_align_up(array, Alignment);
+}
+
+void test() {
+  test_templated_arguments<int, 32>(); // fine
+  test_templated_arguments<struct fwddecl, 16>();
+  // expected-note@-1{{in instantiation of function template specialization 'test_templated_arguments<fwddecl, 16, 16>'}}
+  // expected-note@-2{{forward declaration of 'fwddecl'}}
+  test_templated_arguments<int, 7>(); // invalid alignment value
+  // expected-note@-1{{in instantiation of function template specialization 'test_templated_arguments<int, 7, 16>'}}
+}
+
+template <typename T>
+void test_incorrect_alignment_without_instatiation(T value) {
+  int array[32];
+  static_assert(__is_same(decltype(__builtin_align_up(array, 31)), int *), // expected-error{{requested alignment is not a power of 2}}
+                "return type should be the decayed array type");
+  static_assert(__is_same(decltype(__builtin_align_down(array, 7)), int *), // expected-error{{requested alignment is not a power of 2}}
+                "return type should be the decayed array type");
+  static_assert(__is_same(decltype(__builtin_is_aligned(array, -1)), bool), // expected-error{{requested alignment must be 1 or greater}}
+                "return type should be bool");
+  __builtin_align_up(array);       // expected-error{{too few arguments to function call, expected 2, have 1}}
+  __builtin_align_up(array, 31);   // expected-error{{requested alignment is not a power of 2}}
+  __builtin_align_down(array, 31); // expected-error{{requested alignment is not a power of 2}}
+  __builtin_align_up(array, 31);   // expected-error{{requested alignment is not a power of 2}}
+  __builtin_align_up(value, 31);   // This shouldn't want since the type is dependent
+  __builtin_align_up(value);       // Same here
+}
+
+// The original fix for the issue above broke some legitimate code.
+// Here is a regression test:
+typedef __SIZE_TYPE__ size_t;
+void *allocate_impl(size_t size);
+template <typename T>
+T *allocate() {
+  constexpr size_t allocation_size =
+      __builtin_align_up(sizeof(T), sizeof(void *));
+  return static_cast<T *>(
+      __builtin_assume_aligned(allocate_impl(allocation_size), sizeof(void *)));
+}
+struct Foo {
+  int value;
+};
+void *test2() {
+  return allocate<struct Foo>();
+}
+
+// Check that pointers-to-members cannot be used:
+class MemPtr {
+public:
+  int data;
+  void func();
+  virtual void vfunc();
+};
+void test_member_ptr() {
+  __builtin_align_up(&MemPtr::data, 64);    // expected-error{{operand of type 'int MemPtr::*' where arithmetic or pointer type is required}}
+  __builtin_align_down(&MemPtr::func, 64);  // expected-error{{operand of type 'void (MemPtr::*)()' where arithmetic or pointer type is required}}
+  __builtin_is_aligned(&MemPtr::vfunc, 64); // expected-error{{operand of type 'void (MemPtr::*)()' where arithmetic or pointer type is required}}
+}
+
+void test_references(Foo &i) {
+  // Check that the builtins look at the referenced type rather than the reference itself.
+  (void)__builtin_align_up(i, 64);                            // expected-error{{operand of type 'Foo' where arithmetic or pointer type is required}}
+  (void)__builtin_align_up(static_cast<Foo &>(i), 64);        // expected-error{{operand of type 'Foo' where arithmetic or pointer type is required}}
+  (void)__builtin_align_up(static_cast<const Foo &>(i), 64);  // expected-error{{operand of type 'const Foo' where arithmetic or pointer type is required}}
+  (void)__builtin_align_up(static_cast<Foo &&>(i), 64);       // expected-error{{operand of type 'Foo' where arithmetic or pointer type is required}}
+  (void)__builtin_align_up(static_cast<const Foo &&>(i), 64); // expected-error{{operand of type 'const Foo' where arithmetic or pointer type is required}}
+  (void)__builtin_align_up(&i, 64);
+}
+
+// Check that constexpr wrapper functions can be constant-evaluated.
+template <typename T>
+constexpr bool wrap_is_aligned(T ptr, long align) {
+  return __builtin_is_aligned(ptr, align);
+  // expected-note@-1{{requested alignment -3 is not a positive power of two}}
+  // expected-note@-2{{requested alignment 19 is not a positive power of two}}
+  // expected-note@-3{{requested alignment must be 128 or less for type 'char'; 4194304 is invalid}}
+}
+template <typename T>
+constexpr T wrap_align_up(T ptr, long align) {
+  return __builtin_align_up(ptr, align);
+  // expected-note@-1{{requested alignment -2 is not a positive power of two}}
+  // expected-note@-2{{requested alignment 18 is not a positive power of two}}
+  // expected-note@-3{{requested alignment must be 2147483648 or less for type 'int'; 8589934592 is invalid}}
+  // expected-error@-4{{operand of type 'bool' where arithmetic or pointer type is required}}
+}
+
+template <typename T>
+constexpr T wrap_align_down(T ptr, long align) {
+  return __builtin_align_down(ptr, align);
+  // expected-note@-1{{requested alignment -1 is not a positive power of two}}
+  // expected-note@-2{{requested alignment 17 is not a positive power of two}}
+  // expected-note@-3{{requested alignment must be 32768 or less for type 'short'; 1048576 is invalid}}
+}
+
+constexpr int a1 = wrap_align_up(22, 32);
+static_assert(a1 == 32, "");
+constexpr int a2 = wrap_align_down(22, 16);
+static_assert(a2 == 16, "");
+constexpr bool a3 = wrap_is_aligned(22, 32);
+static_assert(!a3, "");
+static_assert(wrap_align_down(wrap_align_up(22, 16), 32) == 32, "");
+static_assert(wrap_is_aligned(wrap_align_down(wrap_align_up(22, 16), 32), 32), "");
+static_assert(!wrap_is_aligned(wrap_align_down(wrap_align_up(22, 16), 32), 64), "");
+
+constexpr long const_value(long l) { return l; }
+// Check some invalid values during constant-evaluation
+static_assert(wrap_align_down(1, const_value(-1)), ""); // expected-error{{not an integral constant expression}}
+// expected-note@-1{{in call to 'wrap_align_down(1, -1)'}}
+static_assert(wrap_align_up(1, const_value(-2)), ""); // expected-error{{not an integral constant expression}}
+// expected-note@-1{{in call to 'wrap_align_up(1, -2)'}}
+static_assert(wrap_is_aligned(1, const_value(-3)), ""); // expected-error{{not an integral constant expression}}
+// expected-note@-1{{in call to 'wrap_is_aligned(1, -3)'}}
+static_assert(wrap_align_down(1, const_value(17)), ""); // expected-error{{not an integral constant expression}}
+// expected-note@-1{{in call to 'wrap_align_down(1, 17)'}}
+static_assert(wrap_align_up(1, const_value(18)), ""); // expected-error{{not an integral constant expression}}
+// expected-note@-1{{in call to 'wrap_align_up(1, 18)'}}
+static_assert(wrap_is_aligned(1, const_value(19)), ""); // expected-error{{not an integral constant expression}}
+// expected-note@-1{{in call to 'wrap_is_aligned(1, 19)'}}
+
+// Check invalid values for smaller types:
+static_assert(wrap_align_down(static_cast<short>(1), const_value(1 << 20)), ""); // expected-error{{not an integral constant expression}}
+// expected-note@-1{{in call to 'wrap_align_down(1, 1048576)'}}
+// Check invalid boolean type
+static_assert(wrap_align_up(static_cast<int>(1), const_value(1ull << 33)), ""); // expected-error{{not an integral constant expression}}
+// expected-note@-1{{in call to 'wrap_align_up(1, 8589934592)'}}
+static_assert(wrap_is_aligned(static_cast<char>(1), const_value(1 << 22)), ""); // expected-error{{not an integral constant expression}}
+// expected-note@-1{{in call to 'wrap_is_aligned(1, 4194304)'}}
+
+// Check invalid boolean type
+static_assert(wrap_align_up(static_cast<bool>(1), const_value(1 << 21)), ""); // expected-error{{not an integral constant expression}}
+// expected-note@-1{{in instantiation of function template specialization 'wrap_align_up<bool>' requested here}}
+
+// Check constant evaluation for pointers:
+_Alignas(32) char align32array[128];
+static_assert(&align32array[0] == &align32array[0], "");
+// __builtin_align_up/down can be constant evaluated as a no-op for values
+// that are known to have greater alignment:
+static_assert(__builtin_align_up(&align32array[0], 32) == &align32array[0], "");
+static_assert(__builtin_align_up(&align32array[0], 4) == &align32array[0], "");
+static_assert(__builtin_align_down(&align32array[0], 4) == __builtin_align_up(&align32array[0], 8), "");
+// But it can not be evaluated if the alignment is greater than the minimum
+// known alignment, since in that case the value might be the same if it happens
+// to actually be aligned to 64 bytes at run time.
+static_assert(&align32array[0] == __builtin_align_up(&align32array[0], 64), ""); // expected-error{{not an integral constant expression}}
+// expected-note@-1{{cannot constant evaluate the result of adjusting alignment to 64}}
+static_assert(__builtin_align_up(&align32array[0], 64) == __builtin_align_up(&align32array[0], 64), ""); // expected-error{{not an integral constant expression}}
+// expected-note@-1{{cannot constant evaluate the result of adjusting alignment to 64}}
+
+// However, we can compute in case the requested alignment is less than the
+// base alignment:
+static_assert(__builtin_align_up(&align32array[0], 4) == &align32array[0], "");
+static_assert(__builtin_align_up(&align32array[1], 4) == &align32array[4], "");
+static_assert(__builtin_align_up(&align32array[2], 4) == &align32array[4], "");
+static_assert(__builtin_align_up(&align32array[3], 4) == &align32array[4], "");
+static_assert(__builtin_align_up(&align32array[4], 4) == &align32array[4], "");
+static_assert(__builtin_align_up(&align32array[5], 4) == &align32array[8], "");
+static_assert(__builtin_align_up(&align32array[6], 4) == &align32array[8], "");
+static_assert(__builtin_align_up(&align32array[7], 4) == &align32array[8], "");
+static_assert(__builtin_align_up(&align32array[8], 4) == &align32array[8], "");
+
+static_assert(__builtin_align_down(&align32array[0], 4) == &align32array[0], "");
+static_assert(__builtin_align_down(&align32array[1], 4) == &align32array[0], "");
+static_assert(__builtin_align_down(&align32array[2], 4) == &align32array[0], "");
+static_assert(__builtin_align_down(&align32array[3], 4) == &align32array[0], "");
+static_assert(__builtin_align_down(&align32array[4], 4) == &align32array[4], "");
+static_assert(__builtin_align_down(&align32array[5], 4) == &align32array[4], "");
+static_assert(__builtin_align_down(&align32array[6], 4) == &align32array[4], "");
+static_assert(__builtin_align_down(&align32array[7], 4) == &align32array[4], "");
+static_assert(__builtin_align_down(&align32array[8], 4) == &align32array[8], "");
+
+// Achiving the same thing using casts to uintptr_t is not allowed:
+static_assert((char *)((__UINTPTR_TYPE__)&align32array[7] & ~3) == &align32array[4], ""); // expected-error{{not an integral constant expression}}
+
+static_assert(__builtin_align_down(&align32array[1], 4) == &align32array[0], "");
+static_assert(__builtin_align_down(&align32array[1], 64) == &align32array[0], ""); // expected-error{{not an integral constant expression}}
+// expected-note@-1{{cannot constant evaluate the result of adjusting alignment to 64}}
+
+// Add some checks for __builtin_is_aligned:
+static_assert(__builtin_is_aligned(&align32array[0], 32), "");
+static_assert(__builtin_is_aligned(&align32array[4], 4), "");
+// We cannot constant evaluate whether the array is aligned to > 32 since this
+// may well be true at run time.
+static_assert(!__builtin_is_aligned(&align32array[0], 64), ""); // expected-error{{not an integral constant expression}}
+// expected-note@-1{{cannot constant evaluate whether run-time alignment is at least 64}}
+
+// However, if the alignment being checked is less than the minimum alignment of
+// the base object we can check the low bits of the alignment:
+static_assert(__builtin_is_aligned(&align32array[0], 4), "");
+static_assert(!__builtin_is_aligned(&align32array[1], 4), "");
+static_assert(!__builtin_is_aligned(&align32array[2], 4), "");
+static_assert(!__builtin_is_aligned(&align32array[3], 4), "");
+static_assert(__builtin_is_aligned(&align32array[4], 4), "");
+
+// TODO: this should evaluate to true even though we can't evaluate the result
+//  of __builtin_align_up() to a concrete value
+static_assert(__builtin_is_aligned(__builtin_align_up(&align32array[0], 64), 64), ""); // expected-error{{not an integral constant expression}}
+// expected-note@-1{{cannot constant evaluate the result of adjusting alignment to 64}}
+
+// Check different source and alignment type widths are handled correctly.
+static_assert(!__builtin_is_aligned(static_cast<signed long>(7), static_cast<signed short>(4)), "");
+static_assert(!__builtin_is_aligned(static_cast<signed short>(7), static_cast<signed long>(4)), "");
+// Also check signed -- unsigned mismatch.
+static_assert(!__builtin_is_aligned(static_cast<signed long>(7), static_cast<signed long>(4)), "");
+static_assert(!__builtin_is_aligned(static_cast<unsigned long>(7), static_cast<unsigned long>(4)), "");
+static_assert(!__builtin_is_aligned(static_cast<signed long>(7), static_cast<unsigned long>(4)), "");
+static_assert(!__builtin_is_aligned(static_cast<unsigned long>(7), static_cast<signed long>(4)), "");
+static_assert(!__builtin_is_aligned(static_cast<signed long>(7), static_cast<unsigned short>(4)), "");
+static_assert(!__builtin_is_aligned(static_cast<unsigned short>(7), static_cast<signed long>(4)), "");