I am going on vacation in a few hours and will not have time to finish this. Feel free to take over.
It turns out that at least on relocations that have produce a 0 at runtime, we cannot assume it is local. If we do, the architecture dependent part will assume it can always compute 0 from a position in the file. While 0 might always be in range, it will have a different offset in different executions.
If this goes in as is, there will be some code that is redundant and can be cleaned up.