Both gold and ld do align entries to the size of platform pointer.
https://sourceware.org/bugzilla/show_bug.cgi?id=2657#c3
https://sourceware.org/ml/binutils/2006-05/msg00242.html
That seems to be consistent with spec that says both CIE/FDE has
"Padding
Extra bytes to align the CIE structure to an addressing unit size boundary."
https://refspecs.linuxfoundation.org/LSB_3.0.0/LSB-PDA/LSB-PDA/ehframechpt.html