This is an archive of the discontinued LLVM Phabricator instance.

Fix undefined behavior in __tree
ClosedPublic

Authored by EricWF on May 29 2016, 10:08 PM.

Details

Summary

This patch attempts to fix the undefined behavior in __tree by changing the node pointer types used throughout. The pointer types are changed for raw pointers in the current ABI and for fancy pointers in ABI V2 (since the fancy pointer types may not be ABI compatible).

The UB in __tree arises because tree downcasts the embedded end node and then deferences that pointer. Currently there are 3 node types in __tree.

  • __tree_end_node which contains the __left_ pointer. This node is embedded within the container.
  • __tree_node_base which contains __right_, __parent_ and __is_black. This node is used throughout the tree rebalancing algorithms.
  • __tree_node which contains __value_.

Currently __tree stores the start of the tree, __begin_node_, as a pointer to a __tree_node. Additionally the iterators store their position as a pointer to a __tree_node. In both of these cases the pointee can be the end node. This is fixed by changing them to store __tree_end_node pointers instead.

To make this change I introduced an __iter_pointer typedef which is defined to be a pointer to either __tree_end_node in the new ABI or __tree_node in the current one.
Both __tree::__begin_node_ and iterator pointers are now stored as __iter_pointers.

The other situation where __tree_end_node is stored as the wrong type is in __tree_node_base::__parent_. Currently __left_, __right_, and __parent_ are all __tree_node_base pointers. Since the end node will only be stored in __parent_ the fix is to change __parent_ to be a pointer to __tree_end_node.

To make this change I introduced a __parent_pointer typedef which is defined to be a pointer to either __tree_end_node in the new ABI or __tree_node_base in the current one.

Note that in the new ABI __iter_pointer and __parent_pointer are the same type (but not in the old one). The confusion between these two types is unfortunate but it was the best solution I could come up with that maintains the ABI.

The typedef changes force a ton of explicit type casts to correct pointer types and to make current code compatible with both the old and new pointer typedefs. This is the bulk of the change and it's really messy. Unfortunately I don't know how to avoid it.

Please let me know what you think.

Diff Detail

Event Timeline

EricWF updated this revision to Diff 58930.May 29 2016, 10:08 PM
EricWF retitled this revision from to Fix undefined behavior in __tree.
EricWF updated this object.
EricWF added a reviewer: mclow.lists.
EricWF added a subscriber: cfe-commits.
EricWF updated this revision to Diff 58931.May 29 2016, 10:33 PM

Fix __tree algorithm tests.

EricWF updated this revision to Diff 58933.May 29 2016, 10:40 PM

Fix some code I broke last iteration.

EricWF updated this object.May 30 2016, 12:58 AM

@mclow.lists What would you like to see in order to help this patch proceed?

EricWF updated this revision to Diff 63267.Jul 8 2016, 11:29 AM

Fix merge conflicts in __config.

EricWF added a subscriber: howard.hinnant.

Adding Howard to the reviewers. @howard.hinnant Feel free to take a look if your interested.

@mclow.lists Any comments?

mclow.lists accepted this revision.Jul 18 2016, 7:14 AM
mclow.lists edited edge metadata.

I'm not really happy with this situation, but I don't see how to improve this patch.

This revision is now accepted and ready to land.Jul 18 2016, 7:14 AM
EricWF updated this revision to Diff 64525.Jul 19 2016, 11:02 AM
EricWF edited edge metadata.

Add test for PR28469 and remove __tree from the UBSAN blacklist.

EricWF closed this revision.Jul 19 2016, 11:03 AM