When the coefficients of dividend are negative, the gcd may be negative
which will change the sign of dividend and overflow denominator.
Details
Details
Diff Detail
Diff Detail
- Repository
- rG LLVM Github Monorepo
Event Timeline
mlir/lib/Analysis/Presburger/Utils.cpp | ||
---|---|---|
40–46 | I would prefer taking absolute of dividend[i] here. Also, this change still does not cover everything properly. gcd inside the loop can be -1 and will not take the return condition when it should have. Taking absolute of dividend[i] should also fix this. |
mlir/lib/Analysis/Presburger/Utils.cpp | ||
---|---|---|
40–46 | I see, thanks. I have made the necessary changes. |
I would prefer taking absolute of dividend[i] here.
Also, this change still does not cover everything properly. gcd inside the loop can be -1 and will not take the return condition when it should have. Taking absolute of dividend[i] should also fix this.