The bitcast may be interfering with other combines or vectorization as shown in PR16739:
I haven't looked at GEPs very much, so I'm not sure if I've missed any safety checks here. I'm also not sure if this warning (still) applies:
"This hasn’t always been forcefully disallowed, though it’s not recommended. It leads to awkward special cases in the optimizers, and fundamental inconsistency in the IR. In the future, it will probably be outright disallowed."
...but it's 5 years since the bug was filed, and the IR looks the same.