I'm planning on handling intrinsics that will benefit from checking
the address space enums. Don't bother moving the address collection
for now, since those won't need th enums.
In general, we need to tell the value use, says save a generic pointer to somewhere, of a pointer from its address use. Only the later needs address-space resolving or inferring. It would be better to add the extra opnd index to help the target checks that case and only rewrite the address use.
InferAddressSpaces doesn't track this based on the operand index. This will expect any operands reported by collectRewritableIntrinsicOperands to be replaced, so it shouldn't be reporting any non-address uses