The loop here calls ConstantFoldConstantExpression before knowing if all the operands are constant, which can waste a lot of time. Cuts time spent in ConstantFoldInstruction (and callees) by ~20% in our tests. Should be NFC.
LGTM with nits.
I don't think you need the llvm:: qualifier.