This is an archive of the discontinued LLVM Phabricator instance.

[flang] Fix heap overflow in Real formatting.
AbandonedPublic

Authored by Meinersbur on Oct 4 2020, 4:15 AM.

Details

Summary

The reinterpret_cast is problematic because 1. it violates the strict aliasing assumption and 2. the types can be of different size. In particular, BinaryFloatingPointNumber is always a power-of-2 whereas the source type can be smaller (like for 80-bit real). Therefore, when dereferencing the BinaryFloatingPointNumber, the accesses exceeds the source type's allocation size.

Fix by using memcpy to a zero-initialized buffer of the larger size.

Found using AddressSanitizer.

Diff Detail

Event Timeline

Meinersbur created this revision.Oct 4 2020, 4:15 AM
Herald added a project: Restricted Project. · View Herald Transcript
Meinersbur requested review of this revision.Oct 4 2020, 4:15 AM

This works, but maybe the memcpy should be in a constructor for BinaryFloatingPointNumber -- it already has a memcpy in its default constructor, after an assertion that the size is not smaller (which would be removed).

Copy integer part-wise instead of memcpy. memcpy/reinterpret_cast assumed platform endianess.

I think a better fix would be to generalize ToUInt64 as a template elsewhere. I'll prepare a patch and add you as the reviewer.

Thanks, looking forward to you patch.

Meinersbur abandoned this revision.Oct 19 2020, 8:40 AM

Superseded by D89435