llvm.org GIT mirror llvm / 74f53f3
[instsimplify] Fix incorrect folding of an ordered fcmp with a vector of all NaN. This patch fixes a crash caused by an incorrect folding of an ordered comparison between a packed floating point vector and a splat vector of NaN. An ordered comparison between a vector and a constant vector of NaN, should always be folded into a constant vector where each element is i1 false. Since revision 266175, SimplifyFCmpInst folds the ordered fcmp into a scalar 'false'. Later on, this would cause an assertion failure, since the value type of the folded value doesn't match the expected value type of the uses of the original instruction: "Assertion failed: New->getType() == getType() && "replaceAllUses of value with new value of different type!". This patch fixes the issue and adds a test case to the already existing test InstSimplify/floating-point-compares.ll. Differential Revision: https://reviews.llvm.org/D24143 git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@280488 91177308-0d34-0410-b5e6-96231b3b80d8 Andrea Di Biagio 3 years ago
2 changed file(s) with 9 addition(s) and 1 deletion(s). Raw diff Collapse all Expand all
32343234 // If the constant is a nan, see if we can fold the comparison based on it.
32353235 if (CFP->getValueAPF().isNaN()) {
32363236 if (FCmpInst::isOrdered(Pred)) // True "if ordered and foo"
3237 return ConstantInt::getFalse(CFP->getContext());
3237 return ConstantInt::get(GetCompareTy(LHS), 0);
32383238 assert(FCmpInst::isUnordered(Pred) &&
32393239 "Comparison must be either ordered or unordered!");
32403240 // True if unordered.
122122 %cmp = fcmp nnan ord double %in1, %in2
123123 ret i1 %cmp
124124 }
125
126 define <2 x i1> @orderedCompareWithNaNVector(<2 x double> %A) {
127 ; CHECK-LABEL: @orderedCompareWithNaNVector(
128 ; CHECK: ret <2 x i1> zeroinitializer
129 %cmp = fcmp olt <2 x double> %A,
130 ret <2 x i1> %cmp
131 }
132